Abstract Augmentation Types

class tormentor.DeterministicImageAugmentation(aug_id=None, seed=None)

Deterministic augmentation functor and its factory.

This class is the base class realising an augmentation. In order to create a create_persistent augmentation the forward_sample_img method and the factory class-function have to be defined. If a constructor is defined it must call super().__init__(**kwargs) .

The **kwargs on the constructor define all needed variables for generating random augmentations. The factory method returns a lambda that instanciates augmentatations. Any parameter about the augmentations randomness should be passed by the factory to the constructor and used inside forward_sample_img’s definition.

augment_image(image_tensor: FloatTensor)

Augments an image or a batch of images.

This method enforces determinism for image data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. All subclasses should implement forward_image instead. Images can have any number of channels although some augmentations eg. those manipulating the color-space might expect specific channel counts.

Parameters

image_tensor – a float tensor of [batch x channels x height x width] or [channels x height x width].

Returns

An image or a batch of tensors sized [batch x channels x height x width] or [channels x height x width]

augment_image(image_tensor: FloatTensor)

Augments an image or a batch of images.

This method enforces determinism for image data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. All subclasses should implement forward_image instead. Images can have any number of channels although some augmentations eg. those manipulating the color-space might expect specific channel counts.

Parameters

image_tensor – a float tensor of [batch x channels x height x width] or [channels x height x width].

Returns

An image or a batch of tensors sized [batch x channels x height x width] or [channels x height x width]

augment_mask(mask_tensor: Tensor)

Augments an mask or a batch of masks.

Masks differ from images as they are interpreted to answer a pixel-wise “where” question. Although technically they can be indistinguishable from images they are interpreted differently. A typical example would be a dense segmentation mask such containing class one-hot encoding along the channel dimension. This method enforces determinism for mask data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. A subclasses should implement forward_mask instead.

Parameters

mask_tensor – a float tensor of [batch x channels x height x width] or [channels x height x width].

Returns

An mask or a batch of tensors sized [batch x channels x height x width] or [channels x height x width]

augment_pointcloud(pc: Union[List[Tuple[FloatTensor, FloatTensor]], Tuple[FloatTensor, FloatTensor]], image_tensor: FloatTensor, compute_img: bool)

Augments pointclouds over an image or a batch.

Pointclouds are defined to be in pixel coordinates in contextualised by an image or at least an image size. The pointcloud for a single image is a tuple of 1D float tensors (vectors) one with the X coordinates and one with the Y coordinates. If the image_tensor is a batch, then a list of pointclouds is associated with the batch, one for every image in the batch. Pointcloud augmentation shares a lot of the heavy computation with augmenting its reference image tensor, both are employing an augmented sampling field. This method should be treated as final and should not be redefined in subclasses. A subclasses should implement forward_pointcloud instead.

Parameters

• pc – Either a tuple of vectors with X Y coordinates, or a list of many such tuples.

• image_tensor – A 3D tensor [channel x height x width] or a 4D tensor [batch x channel x height x width].

• compute_img – If True, the reference image will be augmented and returned, if false the reference image will be returned unaugmented.

Returns

A tuple with a pointcloud or a list of pointclouds, and a 3D or 4D tensor. The image tensor is either the original image_tensor or the same exact augmentation applied the point cloud.

augment_sampling_field(sf: Tuple[FloatTensor, FloatTensor])

Augments a sampling field for an image or samplingfileds for batches.

Sampling fields are the way to see how augmentations move things around. A sampling field can be generated with create_sampling_field and be used to resample image data with apply_sampling_field

This method enforces determinism for image data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. A subclasses should implement forward_sampling_field instead.

Parameters

sf – a tuple with 2 float tensors of the same size. Either [batch x height x width] or [height x width]

Returns

A tuple of 2 tensors sized [batch x new_height x new_width] or [new_height x new_width]

forward_bboxes(bboxes: FloatTensor, image_tensor=None, width_height=None) → FloatTensor

Applies a transformation on Image coordinate defined bounding boxes.

Bounding Boxes are encoded as [Left, Top, Right, Bottom]

Parameters

• bboxes (torch.FloatTensor) – A tensor with bboxes for a sample [N x 4] or a batch [S x N x 4]

• image_tensor (torch.FloatTensor) – A valid batch image tensor [S x C x H x C] or sample image tensor [C x H x W]. In both cases it only used to normalise bbox coordinates and can be omitted if width_height is specified.

• width_height (int, int) – Values used to normalise bbox coordinates to [-1,1] and back, should be omitted if image tensor is passed

Returns: a tensor with the bounding boxes of the transformed bounding box.

forward_img(batch_tensor: FloatTensor) → FloatTensor

Distorts a batch of one or more images.

Parameters

batch_tensor – Images are 4D tensors of [batch_size, #channels, height, width] size.

Returns

A create_persistent 4D tensor [batch_size, #channels, height, width] with the create_persistent image.

forward_img_path_probabilities(batch_tensor: FloatTensor) → FloatTensor

Returns the probability of a specific augmentation’s path being sampled

Normally only augmentation choice and cascade are not leafs in the execution graph.

Parameters

batch_tensor – A tensor in order to infer the batchsize

Returns

A 1D tensor with the probabillity of each batch-sample’s augmentation path probabillity.

forward_mask(batch_tensor: Tensor) → LongTensor

Distorts a a batch of one or more masks.

Parameters

batch_tensor – Images are 4D tensors of [batch_size, #channels, height, width] size.

Returns

A create_persistent 4D tensor [batch_size, #channels, height, width] with the create_persistent image.

forward_pointcloud(pcl: List[Tuple[FloatTensor, FloatTensor]], batch_tensor: FloatTensor, compute_img: bool) → Tuple[List[Tuple[FloatTensor, FloatTensor]], FloatTensor]

Applies a transformation on normalised coordinate points.

:param pcl, a pointcloud for every image in the batch pointclouds are given in pixel coordinates. The list must have the same size as the batch_tensor. Each pointcloud is a tuple consisting of the vectors

Parameters

• batch_tensor (torch.FloatTensor) – The images to which each of the pointclouds refers [BxCxHxW]

• compute_img (bool) – If False the only the pointcloud will be computed, if True the images in the batch_tensor will also be augmented.

Returns

augmented pointclouds and input image tensor or the augmented image tensor depending on compute_img.

Return type

PointCloudsImages

forward_sampling_field(coords: Tuple[FloatTensor, FloatTensor]) → Tuple[FloatTensor, FloatTensor]

Defines a spatial transform.

Parameters

() (coords) – a tuple with two 3D float tensors each having a size of [Batch x Height x Width]. X and Y coordinates are in reference the range [-1, 1].

Returns

The augmented samplint field.

generate_batch_state(batch_tensor: Tensor) → Tuple[Tensor, ...]

Generates deterministic state for each augmentation.

Returns: a tuple of tensors representing the complete state of the augmentaion so that a

like_me()

Returns a new augmentation following the same distributions as self.

Returns

An instance of the same class as self.

class tormentor.SpatialImageAugmentation(aug_id=None, seed=None)

Parent class for augmentations that move things around.

Every class were image pixels move around rather that just change should be a descendant of this class. All classes that do not descend from this class are expected to be neutral for pointclouds, and sampling fields and should be descendants of StaticImageAugmentation.

class tormentor.StaticImageAugmentation(aug_id=None, seed=None)

Parent class for augmentations that don’t move things around.

All classes that do descend from this are expected to be neutral for pointclouds, sampling fields although they might be erasing regions of the images so they are not gurantied to be neutral to masks. Every class were image pixels move around rather that just stay static, should be a descendant of SpatialImageAugmentation.

class tormentor.ColorAugmentation(aug_id=None, seed=None)

Abstract class for all augmentations manipulating the colorspace.

All augmentations inheriting ColorAugmentation, expect 3-channel inputs that can be interpreted as RGB in the range [0., 1.]. If the channels are neither 3 or 1, the augmentation becomes an identity. The subclasses should only define generate_batch_state(self, batch: torch.FloatTensor) and classmethod functional_image(cls, batch: torch.FloatTensor, *batch_state).