Data Augmentation

Augmentor

class ezflow.functional.data_augmentation.augmentor.FlowAugmentor(crop_size, eraser_aug_params={'aug_prob': 0.5, 'bounds': [50, 100], 'enabled': False}, noise_aug_params={'aug_prob': 0.5, 'enabled': False, 'noise_std_range': 0.06}, flip_aug_params={'enabled': False, 'h_flip_prob': 0.5, 'v_flip_prob': 0.1}, color_aug_params={'asymmetric_color_aug_prob': 0.2, 'brightness': 0.4, 'contrast': 0.4, 'enabled': False, 'hue': 0.15915494309189535, 'saturation': 0.4}, spatial_aug_params={'aug_prob': 0.8, 'enabled': False, 'max_scale': 1.0, 'max_stretch': 0.2, 'min_scale': - 0.1, 'stretch_prob': 0.8}, advanced_spatial_aug_params={'enable_out_of_boundary_crop': False, 'enabled': False, 'rotate': 0.4, 'scale1': 0.3, 'scale2': 0.1, 'stretch': 0.3, 'translate': 0.4})[source]

Class for appyling a series of augmentations to a pair of images and a flow field

Parameters

  • crop_size (int) – Size of the crop to be applied to the images.

  • eraser_aug_params (dict) – Parameters for the eraser augmentation.

  • noise_aug_params (dict) – Parameters for the noise augmentation.

  • flip_aug_params (dict) – Parameters for the flip augmentation.

  • color_aug_params (dict) – Parameters for the color augmentation.

  • spatial_aug_params (dict) – Parameters for the spatial augmentation.

  • advanced_spatial_aug_params (dict) – Parameters for the spatial augmentation. If both spatial and advanced_spatial augmentations are enabled, the advanced spatial augmentations are used.

class ezflow.functional.data_augmentation.augmentor.SparseFlowAugmentor(crop_size, eraser_aug_params={'aug_prob': 0.5, 'bounds': [50, 100], 'enabled': False}, noise_aug_params={'aug_prob': 0.5, 'enabled': False, 'noise_std_range': 0.06}, flip_aug_params={'enabled': False, 'h_flip_prob': 0.5, 'v_flip_prob': 0.1}, color_aug_params={'asymmetric_color_aug_prob': 0.2, 'brightness': 0.4, 'contrast': 0.4, 'enabled': False, 'hue': 0.15915494309189535, 'saturation': 0.4}, spatial_aug_params={'aug_prob': 0.8, 'enabled': False, 'max_scale': 1.0, 'max_stretch': 0.2, 'min_scale': - 0.1, 'stretch_prob': 0.8}, advanced_spatial_aug_params={'enable_out_of_boundary_crop': False, 'enabled': False, 'rotate': 0.4, 'scale1': 0.3, 'scale2': 0.1, 'stretch': 0.3, 'translate': 0.4})[source]

Class for appyling a series of augmentations to a pair of images, flow field and valid flow field.

Parameters

  • crop_size (int) – Size of the crop to be applied to the images.

  • color_aug_params (dict) – Parameters for the color augmentation.

  • eraser_aug_params (dict) – Parameters for the eraser augmentation.

  • spatial_aug_params (dict) – Parameters for the spatial augmentation.

  • flip_aug_params (dict) – Parameters for the flip augmentation.

Operations

class ezflow.functional.data_augmentation.operations.AdvancedSpatialTransform(crop, enabled=False, scale1=0.3, scale2=0.1, rotate=0.4, translate=0.4, stretch=0.3, h_flip_prob=0.5, schedule_coeff=1, order=1, enable_out_of_boundary_crop=False)[source]

Advanced set of spatial transformations borrowed from:

  1. VCN: https://github.com/gengshan-y/VCN/blob/master/dataloader/flow_transforms.py

  2. Autoflow: https://github.com/google-research/opticalflow-autoflow/blob/main/src/dataset_lib/augmentations/spatial_aug.py

This set of augmentations include random scaling, stretch, rotation, translation and out-of-boundary cropping.

Parameters

  • crop_size (list of int) – Size of the crop

  • enabled (bool, default: False) – If True, applies flip transform

  • scale1 (float, default : 0.1) – Scale factor 1

  • scale1 – Scale factor 2

  • rotate (float, default : 0.4) – Rotate factor

  • translate (float, default : 0.4) – Translate factor

  • stretch (float, default : 0.3) – Stretch factor

  • h_flip_prob (float, default=0.5) – Probability of applying the horizontal flip transform

Returns

  • img1 (PIL Image or numpy.ndarray) – Flipped image 1

  • img2 (PIL Image or numpy.ndarray) – Flipped image 2

  • flow (numpy.ndarray) – Flipped flow field

class ezflow.functional.data_augmentation.operations.Normalize(use=False, mean=[0, 0, 0], std=[255.0, 255.0, 255.0])[source]

A class to return Normalized Image.

Parameters

  • use (boolean) – Whether to normalize image or not

  • mean (list) – The list of mean values to be substracted from each image channel

  • std (list) – The list of std values with which to divide each image channel by

ezflow.functional.data_augmentation.operations.color_transform(img1, img2, enabled=False, asymmetric_color_aug_prob=0.2, brightness=0.4, contrast=0.4, saturation=0.4, hue=0.1592356687898089)[source]

Photometric augmentation borrowed from RAFT https://github.com/princeton-vl/RAFT/blob/master/core/utils/augmentor.py

Parameters

  • img1 (PIL Image or numpy.ndarray) – First of the pair of images

  • img2 (PIL Image or numpy.ndarray) – Second of the pair of images

  • enabled (bool, default: False) – If True, applies color transform

  • asymmetric_color_aug_prob (float) – Probability of applying asymetric color jitter augmentation

  • brightness (float) – Brightness augmentation factor

  • contrast (float) – Contrast augmentation factor

  • saturation (float) – Saturation augmentation factor

  • hue (float) – Hue augmentation factor

Returns

  • img1 (PIL Image or numpy.ndarray) – Augmented image 1

  • img2 (PIL Image or numpy.ndarray) – Augmented image 2

ezflow.functional.data_augmentation.operations.crop(img1, img2, flow, crop_size=(256, 256), crop_type='center', sparse_transform=False, valid=None)[source]

Function to crop the images and flow field

Parameters

  • img1 (PIL Image or numpy.ndarray) – First of the pair of images

  • img2 (PIL Image or numpy.ndarray) – Second of the pair of images

  • flow (numpy.ndarray) – Flow field

  • valid (numpy.ndarray) – Valid flow mask

  • crop_size (tuple) – Size of the crop

  • crop_type (str) – Type of cropping

  • sparse_transform (bool) – Whether to apply sparse transform

Returns

  • img1 (PIL Image or numpy.ndarray) – Augmented image 1

  • img2 (PIL Image or numpy.ndarray) – Augmented image 2

  • flow (numpy.ndarray) – Augmented flow field

  • valid (numpy.ndarray) – Valid flow mask

ezflow.functional.data_augmentation.operations.eraser_transform(img1, img2, enabled=False, bounds=[50, 100], aug_prob=0.5)[source]

Occlusion augmentation borrowed from RAFT https://github.com/princeton-vl/RAFT/blob/master/core/utils/augmentor.py

Parameters

  • img1 (PIL Image or numpy.ndarray) – First of the pair of images

  • img2 (PIL Image or numpy.ndarray) – Second of the pair of images

  • enabled (bool, default: False) – If True, applies eraser transform

  • bounds (list of int) – Bounds of the eraser

  • aug_prob (float) – Probability of applying the augmentation

Returns

  • img1 (PIL Image or numpy.ndarray) – Augmented image 1

  • img2 (PIL Image or numpy.ndarray) – Augmented image 2

ezflow.functional.data_augmentation.operations.flip_transform(img1, img2, flow, valid=None, enabled=False, h_flip_prob=0.5, v_flip_prob=0.1)[source]

Flip augmentation borrowed from RAFT https://github.com/princeton-vl/RAFT/blob/master/core/utils/augmentor.py

Parameters

  • img1 (PIL Image or numpy.ndarray) – First of the pair of images

  • img2 (PIL Image or numpy.ndarray) – Second of the pair of images

  • flow (numpy.ndarray) – Flow field

  • valid (numpy.ndarray, default: None) – Valid Flow field

  • enabled (bool, default: False) – If True, applies flip transform

  • h_flip_prob (float, default=0.5) – Probability of applying the horizontal flip transform

  • v_flip_prob (float, default=0.1) – Probability of applying the vertical flip transform

Returns

  • img1 (PIL Image or numpy.ndarray) – Flipped image 1

  • img2 (PIL Image or numpy.ndarray) – Flipped image 2

  • flow (numpy.ndarray) – Flipped flow field

  • valid (numpy.ndarray, default: None) – Valid Flow field

ezflow.functional.data_augmentation.operations.noise_transform(img1, img2, enabled=False, aug_prob=0.5, noise_std_range=0.06)[source]

Applies random noise augmentation from a gaussian distribution borrowed from VCN: https://github.com/gengshan-y/VCN/blob/master/dataloader/flow_transforms.py

Parameters

  • img1 (PIL Image or numpy.ndarray) – First of the pair of images

  • img2 (PIL Image or numpy.ndarray) – Second of the pair of images

  • enabled (bool, default: False) – If True, applies noise transform

  • aug_prob (float) – Probability of applying the augmentation

  • noise_std_range (float) – Standard deviation of the noise

Returns

  • img1 (PIL Image or numpy.ndarray) – Augmented image 1

  • img2 (PIL Image or numpy.ndarray) – Augmented image 2

ezflow.functional.data_augmentation.operations.resize_sparse_flow_map(flow, valid, fx=1.0, fy=1.0)[source]

Resize flow field and valid flow by the scaling factor of fx and fy

Parameters

  • flow (numpy.ndarray) – Flow field

  • valid (numpy.ndarray) – Valid Flow field

  • fx (float) – Scaling factor along x

  • fy (float) – Scaling factor along y

Returns

  • flow (numpy.ndarray) – Flow field

  • valid (numpy.ndarray) – Valid Flow field

ezflow.functional.data_augmentation.operations.sparse_spatial_transform(img1, img2, flow, valid, crop_size, enabled=False, aug_prob=0.8, min_scale=- 0.2, max_scale=0.5, flip=True, h_flip_prob=0.5)[source]

Sparse spatial augmentation.

Parameters

  • img1 (PIL Image or numpy.ndarray) – First of the pair of images

  • img2 (PIL Image or numpy.ndarray) – Second of the pair of images

  • flow (numpy.ndarray) – Flow field

  • valid (numpy.ndarray) – Valid flow field

  • crop_size (list of int) – Size of the crop

  • aug_prob (float) – Probability of applying the augmentation

  • min_scale (float) – Minimum scale factor

  • max_scale (float) – Maximum scale factor

  • flip (bool) – Whether to apply the flip transform

  • h_flip_prob (float) – Probability of applying the horizontal flip transform

  • v_flip_prob (float) – Probability of applying the vertical flip transform

Returns

  • img1 (PIL Image or numpy.ndarray) – Augmented image 1

  • img2 (PIL Image or numpy.ndarray) – Augmented image 2

  • flow (numpy.ndarray) – Augmented flow field

  • valid (numpy.ndarray) – Valid flow field

ezflow.functional.data_augmentation.operations.spatial_transform(img1, img2, flow, crop_size, enabled=False, aug_prob=0.8, stretch_prob=0.8, max_stretch=0.2, min_scale=- 0.2, max_scale=0.5)[source]

Simple set of spatial augmentation borrowed from RAFT https://github.com/princeton-vl/RAFT/blob/master/core/utils/augmentor.py

Includes random scaling and stretch.

Parameters

  • img1 (PIL Image or numpy.ndarray) – First of the pair of images

  • img2 (PIL Image or numpy.ndarray) – Second of the pair of images

  • flow (numpy.ndarray) – Flow field

  • crop_size (list of int) – Size of the crop

  • enabled (bool, default: False) – If True, applies spatial transform

  • aug_prob (float) – Probability of applying the augmentation

  • stretch_prob (float) – Probability of applying the stretch transform

  • max_stretch (float) – Maximum stretch factor

  • min_scale (float) – Minimum scale factor

  • max_scale (float) – Maximum scale factor

Returns

  • img1 (PIL Image or numpy.ndarray) – Augmented image 1

  • img2 (PIL Image or numpy.ndarray) – Augmented image 2

  • flow (numpy.ndarray) – Augmented flow field