Dataloader

Dataloader Creator

class ezflow.data.dataloader.dataloader_creator.DataloaderCreator(batch_size, pin_memory=False, shuffle=True, num_workers=0, drop_last=True, init_seed=False, append_valid_mask=False, is_prediction=False, distributed=False, world_size=None)[source]

A class to configure a data loader for optical flow datasets. Multiple datasets can be added to configure a data loader for training and validation.

Parameters

  • batch_size (int) – Number of samples per batch to load

  • pin_memory (bool, default : False) – If True, the data loader will copy Tensors into CUDA pinned memory before returning them

  • shuffle (bool, default : True) – If True, data is reshuffled at every epoch

  • num_workers (int, default : 4) – Number of subprocesses to use for data loading

  • drop_last (bool, default : True) – If True, the last incomplete batch is dropped

  • init_seed (bool, default : False) – If True, sets random seed to worker

  • append_valid_mask (bool, default : False) – If True, appends the valid flow mask to the original flow mask at dim=0

  • is_prediction (bool, default : False) – If True, only image data are loaded for prediction otherwise both images and flow data are loaded

  • distributed (bool, default : False) – If True, initializes DistributedSampler for Distributed Training

  • world_size (int, default : None) – The total number of GPU devices per node for Distributed Training

add_AutoFlow(root_dir, augment=False, **kwargs)[source]

Adds the AutoFLow dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the Monkaa dataset in SceneFlow

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_Driving(root_dir, augment=False, **kwargs)[source]

Adds the Driving dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the Driving dataset in SceneFlow

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_FlyingChairs(root_dir, split='training', augment=False, **kwargs)[source]

Adds the Flying Chairs dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the flying chairs dataset

  • split (str, default : "training") – specify the training or validation split

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_FlyingThings3D(root_dir, split='training', dstype='frames_cleanpass', augment=False, **kwargs)[source]

Adds the Flying Things 3D dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the flying things 3D dataset in SceneFlow

  • split (str, default : "training") – specify the training or validation split

  • dstype (str, default : "frames_cleanpass") – specify dataset type

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_FlyingThings3DSubset(root_dir, split='training', augment=False, **kwargs)[source]

Adds the Flying Things 3D Subset dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the flying things 3D Subset dataset in SceneFlow

  • split (str, default : "training") – specify the training or validation split

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_HD1K(root_dir, augment=False, **kwargs)[source]

Adds the HD1K dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the MPI Sintel dataset

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_Kitti(root_dir, split='training', augment=False, **kwargs)[source]

Adds the KITTI dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the MPI Sintel dataset

  • split (str, default : "training") – specify the training or validation split

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_Kubric(root_dir, split='training', augment=False, **kwargs)[source]

Adds the Kubric dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the Monkaa dataset in SceneFlow

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_MPISintel(root_dir, split='training', dstype='clean', augment=False, **kwargs)[source]

Adds the MPI Sintel dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the MPI Sintel dataset

  • split (str, default : "training") – specify the training or validation split

  • dstype (str, default : "clean") – specify dataset type

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_Monkaa(root_dir, augment=False, **kwargs)[source]

Adds the Monkaa dataset to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the Monkaa dataset in SceneFlow

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

add_SceneFlow(root_dir, augment=False, **kwargs)[source]

Adds FlyingThings3D, Driving and Monkaa datasets to the DataloaderCreator object.

Parameters

  • root_dir (str) – path of the root directory for the SceneFlow dataset

  • augment (bool, default : True) – If True, applies data augmentation

  • **kwargs – Arbitrary keyword arguments for augmentation specifying crop_size and the probability of color, eraser and spatial transformation

get_dataloader(rank=0)[source]

Gets the Dataloader for the added datasets.

rankint, default0

The process id within a group for Distributed Training

Returns

PyTorch DataLoader object

Return type

torch.utils.data.DataLoader

Device Dataloader

class ezflow.data.dataloader.device_dataloader.DeviceDataLoader(data_loader, device)[source]

A data loader wrapper to move data to a specific compute device.

Parameters

  • data_loader (DataLoader) – The PyTorch DataLoader from torch.utils.data.dataloader

  • device (torch.device) – The compute device