Welcome to Tormentor’s documentation!

tormentor

Sampling Fields

tormentor.create_sampling_field(width: int, height: int, batch_size: int = 1, device: device = 'cpu') → Tuple[FloatTensor, FloatTensor]

Creates a SamplingField.

A SamplingField is a tuple of 3D tensors of the same size. Sampling fields are augmentable by all augmentations although many augmentations (Non-spatial) have no effect on them. The can be used to resample images, pointclouds, masks. When sampling, for both axes, the input image is interpreted to lie on the region [-1, 1]. The output image when resampling will have the width and height of the sampling field. A sampling field can also refer to a single image rather than a batch in whitch case the tensors are 2D. The first dimension is the batch size. The second dimension is the width of the output image after sampling. The third dimension is the width of the output image after sampling. The created sampling fields are normalised in the range [-1,1] regardless of their size. Although not enforced, it is expected that augmentations are homomorphisms. Sampling fields are expected to operate identically on all channels and dont have a channel dimension.

Parameters

• width – The sampling fields width.

• height – The sampling fields height.

• batch_size – If 0, the sampling field refers to a single image. Otherwise the first dimension of the tensors. Created sampling fileds are simply repeated over the batch dimension. Default value is 1.

• device – the device on which the sampling filed will be created.

Returns

A tuple of 3D or 2D tensors with values ranged in [-1,1]

tormentor.apply_sampling_field(input_img: Tensor, coords: Tuple[FloatTensor, FloatTensor])

Resamples one or more images by applying sampling fields.

Bilinear interpolation is employed.

Parameters

• input_img – A 4D float tensor [batch x channel x height x width] or a 3D tensor [channel x height x width]. Containing the image or batch from which the image is sampled.

• coords – A sampling field with 3D [batch x out_height x out_width] or 2D [out_height x out_width]. The dimensions of the sampling fields must be one less that the input_img.

Returns

A tensor of as many dimensions [batch x channel x out_height x out_width] or [channel x out_height x out_width]] as the input.

Abstract Augmentation Types

class tormentor.DeterministicImageAugmentation(aug_id=None, seed=None)

Deterministic augmentation functor and its factory.

This class is the base class realising an augmentation. In order to create a create_persistent augmentation the forward_sample_img method and the factory class-function have to be defined. If a constructor is defined it must call super().__init__(**kwargs) .

The **kwargs on the constructor define all needed variables for generating random augmentations. The factory method returns a lambda that instanciates augmentatations. Any parameter about the augmentations randomness should be passed by the factory to the constructor and used inside forward_sample_img’s definition.

augment_image(image_tensor: FloatTensor)

Augments an image or a batch of images.

This method enforces determinism for image data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. All subclasses should implement forward_image instead. Images can have any number of channels although some augmentations eg. those manipulating the color-space might expect specific channel counts.

Parameters

image_tensor – a float tensor of [batch x channels x height x width] or [channels x height x width].

Returns

An image or a batch of tensors sized [batch x channels x height x width] or [channels x height x width]

augment_image(image_tensor: FloatTensor)

Augments an image or a batch of images.

This method enforces determinism for image data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. All subclasses should implement forward_image instead. Images can have any number of channels although some augmentations eg. those manipulating the color-space might expect specific channel counts.

Parameters

image_tensor – a float tensor of [batch x channels x height x width] or [channels x height x width].

Returns

An image or a batch of tensors sized [batch x channels x height x width] or [channels x height x width]

augment_mask(mask_tensor: Tensor)

Augments an mask or a batch of masks.

Masks differ from images as they are interpreted to answer a pixel-wise “where” question. Although technically they can be indistinguishable from images they are interpreted differently. A typical example would be a dense segmentation mask such containing class one-hot encoding along the channel dimension. This method enforces determinism for mask data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. A subclasses should implement forward_mask instead.

Parameters

mask_tensor – a float tensor of [batch x channels x height x width] or [channels x height x width].

Returns

An mask or a batch of tensors sized [batch x channels x height x width] or [channels x height x width]

augment_pointcloud(pc: Union[List[Tuple[FloatTensor, FloatTensor]], Tuple[FloatTensor, FloatTensor]], image_tensor: FloatTensor, compute_img: bool)

Augments pointclouds over an image or a batch.

Pointclouds are defined to be in pixel coordinates in contextualised by an image or at least an image size. The pointcloud for a single image is a tuple of 1D float tensors (vectors) one with the X coordinates and one with the Y coordinates. If the image_tensor is a batch, then a list of pointclouds is associated with the batch, one for every image in the batch. Pointcloud augmentation shares a lot of the heavy computation with augmenting its reference image tensor, both are employing an augmented sampling field. This method should be treated as final and should not be redefined in subclasses. A subclasses should implement forward_pointcloud instead.

Parameters

• pc – Either a tuple of vectors with X Y coordinates, or a list of many such tuples.

• image_tensor – A 3D tensor [channel x height x width] or a 4D tensor [batch x channel x height x width].

• compute_img – If True, the reference image will be augmented and returned, if false the reference image will be returned unaugmented.

Returns

A tuple with a pointcloud or a list of pointclouds, and a 3D or 4D tensor. The image tensor is either the original image_tensor or the same exact augmentation applied the point cloud.

augment_sampling_field(sf: Tuple[FloatTensor, FloatTensor])

Augments a sampling field for an image or samplingfileds for batches.

Sampling fields are the way to see how augmentations move things around. A sampling field can be generated with create_sampling_field and be used to resample image data with apply_sampling_field

This method enforces determinism for image data. Only the batch dimension is guarantied to be preserved. Channels, width, and height dimensions can differ on the outputs. This method should be treated as final and should not be redefined in subclasses. A subclasses should implement forward_sampling_field instead.

Parameters

sf – a tuple with 2 float tensors of the same size. Either [batch x height x width] or [height x width]

Returns

A tuple of 2 tensors sized [batch x new_height x new_width] or [new_height x new_width]

forward_bboxes(bboxes: FloatTensor, image_tensor=None, width_height=None) → FloatTensor

Applies a transformation on Image coordinate defined bounding boxes.

Bounding Boxes are encoded as [Left, Top, Right, Bottom]

Parameters

• bboxes (torch.FloatTensor) – A tensor with bboxes for a sample [N x 4] or a batch [S x N x 4]

• image_tensor (torch.FloatTensor) – A valid batch image tensor [S x C x H x C] or sample image tensor [C x H x W]. In both cases it only used to normalise bbox coordinates and can be omitted if width_height is specified.

• width_height (int, int) – Values used to normalise bbox coordinates to [-1,1] and back, should be omitted if image tensor is passed

Returns: a tensor with the bounding boxes of the transformed bounding box.

forward_img(batch_tensor: FloatTensor) → FloatTensor

Distorts a batch of one or more images.

Parameters

batch_tensor – Images are 4D tensors of [batch_size, #channels, height, width] size.

Returns

A create_persistent 4D tensor [batch_size, #channels, height, width] with the create_persistent image.

forward_img_path_probabilities(batch_tensor: FloatTensor) → FloatTensor

Returns the probability of a specific augmentation’s path being sampled

Normally only augmentation choice and cascade are not leafs in the execution graph.

Parameters

batch_tensor – A tensor in order to infer the batchsize

Returns

A 1D tensor with the probabillity of each batch-sample’s augmentation path probabillity.

forward_mask(batch_tensor: Tensor) → LongTensor

Distorts a a batch of one or more masks.

Parameters

batch_tensor – Images are 4D tensors of [batch_size, #channels, height, width] size.

Returns

A create_persistent 4D tensor [batch_size, #channels, height, width] with the create_persistent image.

forward_pointcloud(pcl: List[Tuple[FloatTensor, FloatTensor]], batch_tensor: FloatTensor, compute_img: bool) → Tuple[List[Tuple[FloatTensor, FloatTensor]], FloatTensor]

Applies a transformation on normalised coordinate points.

:param pcl, a pointcloud for every image in the batch pointclouds are given in pixel coordinates. The list must have the same size as the batch_tensor. Each pointcloud is a tuple consisting of the vectors

Parameters

• batch_tensor (torch.FloatTensor) – The images to which each of the pointclouds refers [BxCxHxW]

• compute_img (bool) – If False the only the pointcloud will be computed, if True the images in the batch_tensor will also be augmented.

Returns

augmented pointclouds and input image tensor or the augmented image tensor depending on compute_img.

Return type

PointCloudsImages

forward_sampling_field(coords: Tuple[FloatTensor, FloatTensor]) → Tuple[FloatTensor, FloatTensor]

Defines a spatial transform.

Parameters

() (coords) – a tuple with two 3D float tensors each having a size of [Batch x Height x Width]. X and Y coordinates are in reference the range [-1, 1].

Returns

The augmented samplint field.

generate_batch_state(batch_tensor: Tensor) → Tuple[Tensor, ...]

Generates deterministic state for each augmentation.

Returns: a tuple of tensors representing the complete state of the augmentaion so that a

like_me()

Returns a new augmentation following the same distributions as self.

Returns

An instance of the same class as self.

class tormentor.SpatialImageAugmentation(aug_id=None, seed=None)

Parent class for augmentations that move things around.

Every class were image pixels move around rather that just change should be a descendant of this class. All classes that do not descend from this class are expected to be neutral for pointclouds, and sampling fields and should be descendants of StaticImageAugmentation.

class tormentor.StaticImageAugmentation(aug_id=None, seed=None)

Parent class for augmentations that don’t move things around.

All classes that do descend from this are expected to be neutral for pointclouds, sampling fields although they might be erasing regions of the images so they are not gurantied to be neutral to masks. Every class were image pixels move around rather that just stay static, should be a descendant of SpatialImageAugmentation.

class tormentor.ColorAugmentation(aug_id=None, seed=None)

Abstract class for all augmentations manipulating the colorspace.

All augmentations inheriting ColorAugmentation, expect 3-channel inputs that can be interpreted as RGB in the range [0., 1.]. If the channels are neither 3 or 1, the augmentation becomes an identity. The subclasses should only define generate_batch_state(self, batch: torch.FloatTensor) and classmethod functional_image(cls, batch: torch.FloatTensor, *batch_state).

Spatial Augmentations

class tormentor.Perspective(aug_id=None, seed=None)

Applies a perspective transformation on the data by moving the corners of an image.

This augmentation is parametrised by two random variables x_offset and y_offset which are the multipliers of each of the image corners corners (-1, -1), (1, -1), (1, 1), and (-1, 1).

class tormentor.Rotate(aug_id=None, seed=None)

Rotates the image around the center.

class tormentor.Zoom(aug_id=None, seed=None)

Augments by scaling images preserving their aspect ratio.

class tormentor.Scale(aug_id=None, seed=None)

Augmentation by scaling images preserving aspect ratio.

class tormentor.Translate(aug_id=None, seed=None)

Augmentation by translating images.

class tormentor.ScaleTranslate(aug_id=None, seed=None)

Augmentation by scaling and translating images preserving aspect ratio.

class tormentor.Flip(aug_id=None, seed=None)

Implementation of augmentation by flipping the X or Y axis.

class tormentor.ElasticTransform(aug_id=None, seed=None)

Augmentation ElasticTransform.

This augmentation does not guaranty to be a homomorphism. In order for the augmentation to behave as a homomorphism, harmonic_smoothing must be quite low. On the other hand, the complexity of the operation is n**2 with respect to harmonic_smoothing or n log(n) depending of how the gaussian filters are implemented.

class tormentor.Wrap(aug_id=None, seed=None)

Augmentation Wrap.

This augmentation acts like many simultaneous elastic transforms with gaussian sigmas set at varius harmonics.

Distributions:

roughness: Quantification of the local inconsistency of the distortion effect. intensity: Quantification of the intensity of the distortion effect.

class tormentor.Shred(aug_id=None, seed=None)

Color Augmentations

class tormentor.Invert(aug_id=None, seed=None)

Performs color inversion in HSV colorspace for some images randomly selected.

class tormentor.Brightness(aug_id=None, seed=None)

Changes the brightness of the image.

class tormentor.Saturation(aug_id=None, seed=None)

Changes the saturation of the image.

class tormentor.Contrast(aug_id=None, seed=None)

Changes the contrast of the image.

class tormentor.Hue(aug_id=None, seed=None)

Changes the Hue of the image.

class tormentor.ColorJitter(aug_id=None, seed=None)

Changes hue, contrast, saturation, and brightness of the image.

class tormentor.PlasmaBrightness(aug_id=None, seed=None)

Changes the brightness of the image locally.

class tormentor.PlasmaRgbBrightness(aug_id=None, seed=None)

Changes the saturation of the image.

class tormentor.PlasmaContrast(aug_id=None, seed=None)

Changes the contrast of the image locally.

class tormentor.PlasmaShadow(aug_id=None, seed=None)

Lowers the brightness of the image over a random mask.

Document Augmentations

class tormentor.Wrap(aug_id=None, seed=None)

Augmentation Wrap.

This augmentation acts like many simultaneous elastic transforms with gaussian sigmas set at varius harmonics.

Distributions:

roughness: Quantification of the local inconsistency of the distortion effect. intensity: Quantification of the intensity of the distortion effect.

class tormentor.Shred(aug_id=None, seed=None)

Composite Augmentation Types

class tormentor.AugmentationCascade

Select randomly among many augmentations.

Cascade of perspective augmentation followed by plasma-brightness

augmentation_factory = tormentor.RandomPerspective | tormentor.RandomPlasmaBrightness

A more complete usage of AugmentationCascade and AugmentationChoice can be seen in the following listing which produces the following computation graph. In the graph AugmentationCascade can be though of as all arrows that don’t leave an AugmentationChoice

from tormentor import RandomColorJitter, RandomFlip, RandomWrap, \
    RandomPlasmaBrightness, RandomPerspective, \
    RandomGaussianAdditiveNoise, RandomRotate

linear_aug = (RandomFlip ^ RandomPerspective ^ RandomRotate)  | RandomColorJitter
nonlinear_aug = RandomWrap | RandomPlasmaBrightness
final_augmentation = (linear_aug ^ nonlinear_aug) | RandomGaussianAdditiveNoise

epochs, batch_size, n_points, width, height = 10, 5, 20, 320, 240

for _ in range(epochs):
    image_batch = torch.rand(batch_size, 3, height, width)
    segmentation_batch = torch.rand(batch_size, 1, height, width).round()
    augmentation = final_augmentation()
    augmented_images = augmentation(image_batch)
    augmented_gt = augmentation(segmentation_batch)
    # Train and do other things

class tormentor.AugmentationChoice(aug_id=None, seed=None)

Select randomly among many augmentations.

Random choice of perspective and plasma-brightness augmentations

augmentation_factory = tormentor.RandomPerspective ^ tormentor.RandomPlasmaBrightness
augmentation = augmentation_factory()
augmented_image = augmentation(image)

Indices and tables

• Index

• Module Index

• Search Page