Correlation

Correlation Layer

class ezflow.similarity.correlation.layer.CorrelationLayer(pad_size=4, max_displacement=4)[source]

Correlation layer in pure PyTorch Only supports specific values of the following parameters: - kernel_size: 1 - stride_1: 1 - stride_2: 1 - corr_multiply: 1

Parameters

  • pad_size (int) – Padding size for the correlation layer

  • max_displacement (int) – Maximum displacement for the correlation computation

forward(features1, features2)[source]

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Pairwise Correlation

class ezflow.similarity.correlation.pairwise.MutliScalePairwise4DCorr(fmap1, fmap2, num_levels=4, corr_radius=4)[source]

Pairwise 4D correlation at multiple scales. Used in RAFT (https://arxiv.org/abs/2003.12039)

Parameters

  • fmap1 (torch.Tensor) – First feature map

  • fmap2 (torch.Tensor) – Second feature map

  • num_levels (int) – Number of levels in the feature pyramid

  • corr_radius (int) – Radius of the correlation window

Correlation Sampler

class ezflow.similarity.correlation.sampler.IterSpatialCorrelationSampler(kernel_size: Union[int, Tuple[int, int]] = 1, patch_size: Union[int, Tuple[int, int]] = 1, stride: Union[int, Tuple[int, int]] = 1, padding: Union[int, Tuple[int, int]] = 0, dilation: Union[int, Tuple[int, int]] = 1, dilation_patch: Union[int, Tuple[int, int]] = 1)[source]

Spatial correlation sampling from two inputs using iteration in PyTorch

Parameters

  • kernel_size (Union[int, Tuple[int, int]], default 1) – Total size of your correlation kernel, in pixels

  • patch_size (Union[int, Tuple[int, int]], default 1) – Total size of your patch, determining how many different shifts will be applied.

  • stride (Union[int, Tuple[int, int]], default 1) – Stride of the spatial sampler, will modify output height and width.

  • padding (Union[int, Tuple[int, int]], default 0) – Padding applied to input1 and input2 before applying the correlation sampling, will modify output height and width.

  • dilation (Union[int, Tuple[int, int]], default 1) – Similar to dilation in convolution.

  • dilation_patch (Union[int, Tuple[int, int]], default 1) – Step for every shift in patch.

forward(input1: torch.Tensor, input2: torch.Tensor) → torch.Tensor[source]

Compute the correlation sampling from input1 to input2

Parameters

  • input1 (torch.Tensor) – The origin feature map.

  • input2 (torch.Tensor) – The target feature map.

Returns

Result of correlation sampling

Return type

torch.Tensor

ezflow.similarity.correlation.sampler.iter_spatial_correlation_sample(input1: torch.Tensor, input2: torch.Tensor, kernel_size: Union[int, Tuple[int, int]] = 1, patch_size: Union[int, Tuple[int, int]] = 1, stride: Union[int, Tuple[int, int]] = 1, padding: Union[int, Tuple[int, int]] = 0, dilation: Union[int, Tuple[int, int]] = 1, dilation_patch: Union[int, Tuple[int, int]] = 1) → torch.Tensor[source]

Apply spatial correlation sampling from input1 to input2 using iteration in PyTorch. This docstring is taken and adapted from the original package. Every parameter except input1 and input2 can be either single int or a pair of int. For more information about Spatial Correlation Sampling, see this page. https://lmb.informatik.uni-freiburg.de/Publications/2015/DFIB15/

Parameters

  • input1 (torch.Tensor) – The origin feature map.

  • input2 (torch.Tensor) – The target feature map.

  • kernel_size (Union[int, Tuple[int, int]], default 1) – Total size of your correlation kernel, in pixels

  • patch_size (Union[int, Tuple[int, int]], default 1) – Total size of your patch, determining how many different shifts will be applied.

  • stride (Union[int, Tuple[int, int]], default 1) – Stride of the spatial sampler, will modify output height and width.

  • padding (Union[int, Tuple[int, int]], default 0) – Padding applied to input1 and input2 before applying the correlation sampling, will modify output height and width.

  • dilation (Union[int, Tuple[int, int]], default 1) – Similar to dilation in convolution.

  • dilation_patch (Union[int, Tuple[int, int]], default 1) – Step for every shift in patch.

Returns

Result of correlation sampling.

Return type

torch.Tensor

Raises

  • NotImplementedError – If kernel_size != 1.

  • NotImplementedError – If dilation != 1.