Correlation
Correlation Layer
- class ezflow.similarity.correlation.layer.CorrelationLayer(pad_size=4, max_displacement=4)[source]
Correlation layer in pure PyTorch Only supports specific values of the following parameters: - kernel_size: 1 - stride_1: 1 - stride_2: 1 - corr_multiply: 1
- Parameters
pad_size (int) – Padding size for the correlation layer
max_displacement (int) – Maximum displacement for the correlation computation
- forward(features1, features2)[source]
Defines the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Module
instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
Pairwise Correlation
- class ezflow.similarity.correlation.pairwise.MutliScalePairwise4DCorr(fmap1, fmap2, num_levels=4, corr_radius=4)[source]
Pairwise 4D correlation at multiple scales. Used in RAFT (https://arxiv.org/abs/2003.12039)
- Parameters
fmap1 (torch.Tensor) – First feature map
fmap2 (torch.Tensor) – Second feature map
num_levels (int) – Number of levels in the feature pyramid
corr_radius (int) – Radius of the correlation window
Correlation Sampler
- class ezflow.similarity.correlation.sampler.IterSpatialCorrelationSampler(kernel_size: Union[int, Tuple[int, int]] = 1, patch_size: Union[int, Tuple[int, int]] = 1, stride: Union[int, Tuple[int, int]] = 1, padding: Union[int, Tuple[int, int]] = 0, dilation: Union[int, Tuple[int, int]] = 1, dilation_patch: Union[int, Tuple[int, int]] = 1)[source]
Spatial correlation sampling from two inputs using iteration in PyTorch
- Parameters
kernel_size (Union[int, Tuple[int, int]], default 1) – Total size of your correlation kernel, in pixels
patch_size (Union[int, Tuple[int, int]], default 1) – Total size of your patch, determining how many different shifts will be applied.
stride (Union[int, Tuple[int, int]], default 1) – Stride of the spatial sampler, will modify output height and width.
padding (Union[int, Tuple[int, int]], default 0) – Padding applied to input1 and input2 before applying the correlation sampling, will modify output height and width.
dilation (Union[int, Tuple[int, int]], default 1) – Similar to dilation in convolution.
dilation_patch (Union[int, Tuple[int, int]], default 1) – Step for every shift in patch.
- forward(input1: torch.Tensor, input2: torch.Tensor) torch.Tensor [source]
Compute the correlation sampling from input1 to input2
- Parameters
input1 (torch.Tensor) – The origin feature map.
input2 (torch.Tensor) – The target feature map.
- Returns
Result of correlation sampling
- Return type
torch.Tensor
- ezflow.similarity.correlation.sampler.iter_spatial_correlation_sample(input1: torch.Tensor, input2: torch.Tensor, kernel_size: Union[int, Tuple[int, int]] = 1, patch_size: Union[int, Tuple[int, int]] = 1, stride: Union[int, Tuple[int, int]] = 1, padding: Union[int, Tuple[int, int]] = 0, dilation: Union[int, Tuple[int, int]] = 1, dilation_patch: Union[int, Tuple[int, int]] = 1) torch.Tensor [source]
Apply spatial correlation sampling from input1 to input2 using iteration in PyTorch. This docstring is taken and adapted from the original package. Every parameter except input1 and input2 can be either single int or a pair of int. For more information about Spatial Correlation Sampling, see this page. https://lmb.informatik.uni-freiburg.de/Publications/2015/DFIB15/
- Parameters
input1 (torch.Tensor) – The origin feature map.
input2 (torch.Tensor) – The target feature map.
kernel_size (Union[int, Tuple[int, int]], default 1) – Total size of your correlation kernel, in pixels
patch_size (Union[int, Tuple[int, int]], default 1) – Total size of your patch, determining how many different shifts will be applied.
stride (Union[int, Tuple[int, int]], default 1) – Stride of the spatial sampler, will modify output height and width.
padding (Union[int, Tuple[int, int]], default 0) – Padding applied to input1 and input2 before applying the correlation sampling, will modify output height and width.
dilation (Union[int, Tuple[int, int]], default 1) – Similar to dilation in convolution.
dilation_patch (Union[int, Tuple[int, int]], default 1) – Step for every shift in patch.
- Returns
Result of correlation sampling.
- Return type
torch.Tensor
- Raises
NotImplementedError – If kernel_size != 1.
NotImplementedError – If dilation != 1.