Reference

Loading & using CAVs

The main way to use CAVlib is through the cavlib.CAV class.

class cavlib.CAV(id, vector, model_layer, metadata=None)

This class represents a CAV object. CAVs are normally created by loading them from disk with CAV.load() or by training them train_cav().

classmethod load(path)

Load a CAV from file.

Parameters

path (str | Path) – The path to the CAV file to load.

Returns

the CAV object.

Return type

CAV

save(path)

Save the CAV file to disk.

Parameters

path (str | Path) – The path to save to. The file is created if it does not exist, otherwise the existing file is overwritten. We recommend using a file extension of .cav.

score(image_or_activation)

Calculates the CAV score for an image.

Parameters

image_or_activation (CAVableImage | NDArray[np.float32]) – The image to score according to this CAV. Can be any of the supported image formats, or a precomputed activation (e.g. from get_activations())

Training CAVs

cavlib.train_cav(*, positive_images, negative_images, model_layer='googlenet_4d', random_state=None)

Create a new CAV by training it on the given positive and negative samples.

Parameters

  • positive_images (Sequence[CAVableImage | TrainingImage]) – A list of images that represent the concept.

  • negative_images (Sequence[CAVableImage | TrainingImage]) – A list of images that don’t represent the concept. These can either be random images, or, even better, images that represent the opposite of the concept you’re trying to capture.

  • model_layer (ModelLayer) – The model layer that you want the new CAV to use.

  • random_state (Optional[np.random.RandomState]) – A supply of randomness. Training CAVs is nondeterministic, due to the SGDClassifier linear model that CAVlib uses. This is not normally a problem, but if you require consistent, deterministic results (for example, in regression tests), set this to a fresh instance of numpy.random.RandomState that has been seeded to a constant, and ensure that your training images are always in the same order.

The image arguments can either be images (e.g. a path, a PIL.Image or numpy array of pixels), or TrainingImage objects. TrainingImage objects provide more flexibility - they can include weights, or, if you have a precalculated activation, this can be used via a TrainingImage.

class cavlib.TrainingImage(image=None, activations=None, weight=1.0)

This object represents an image used for training a CAV. It’s use in training is optional, but it has a few functions:

  • if you want to supply weights when training, you can do so using the weight attribute.

  • it caches activations in-memory

  • if you want to avoid computing model activations at all, you can create the TrainingImage with a precalculated activations dict.

Parameters

  • image (Optional[CAVableImage]) – the image to be used for training. Can be None, if activations is passed.

  • activations (Optional[Dict[ModelLayer, NDArray[np.float32]]]) – a dictionary of activations, keyed by model_layer. e.g. {"googlenet_4d": np.ndarray(<your activation>)}. Used to supply precomputed activations, if you have them.

  • weight (float) – the weight of this image in training. Defaults to 1. To upweight an image, set it to a larger value, like 4.

Computing activations

cavlib.compute_activations(image, model_layer='googlenet_4d')

Calculates activations for a given image. Activations are the value of each neuron in the network at that layer.

This function can be used to precalculate activations for a set of images, so they don’t have to be calculated every time. Activations can be used with cavlib.CAV.score() and cavlib.train.TrainingImage.

Parameters

  • image (CAVableImage) – The image whose activations we want to calculate. Accepts a path to an image file, an open file-like object of an image, a PIL Image, or a numpy array of RGB data

  • model_layer (str) – The model layer to extract.

Returns

The activation vector, as a 1D numpy array.

Image formats

Images can be supplied to CAVlib in a number of formats, as denoted in the API documentation as CAVableImage.

CAVableImage

Wherever CAVableImage appears, the following data types are accepted:

Images that are not the correct size for the model (224, 244) will be:

  • center cropped so that they are square

  • bilinear resized to the correct dimensions

Model layers

ModelLayer

Specifies the model and layer to use to train/evaluate CAVs.

googlenet_4d

A layer mid-way through GooglenetModel. In our testing, it’s good at detecting distinct visual elements like shapes and patterns.

googlenet_5b

A layer towards the end of GooglenetModel. We have found it more conceptual layer that’s good at detecting diverse patterns, textures and compositions.

mobilenet_12d

A layer within MobilenetModel that’s good at detecting more obvious visual themes like colour and texture.

Low-level

Model API

class cavlib.models.Model(model_path, input_value_range)

Bases: object

Base class for models that can produce activations for CAVs.

property output_layer_names: List[str]

A list of layer names that can be used with get_activation_for_image() and get_multiple_activations_for_image()

get_activation_for_image(image, layer_name)

Returns an activation vector for a single image.

Parameters

  • image (numpy.ndarray) – a numpy array of shape (224, 244, 3), either in uint8 type, in the range 0-255, or in floating point, from 0.0-1.0.

  • layer_name (str) – Layer name to return activations for.

Returns

the activation vector

Return type

numpy.ndarray

get_multiple_activations_for_image(image, layer_names)

Returns multiple layer activations for a single image. Faster than get_activations_for_image because the model is only run once.

Parameters

  • image (numpy.ndarray) – a numpy array of shape (224, 244, 3), either in uint8 type, in the range 0-255, or in floating point, from 0.0-1.0.

  • layer_names (List[str]) – Layer names to return activations for.

Returns

List of dicts, containing {layer_name: activation}

Return type

Dict[str, numpy.ndarray]

class cavlib.models.GooglenetModel

Bases: cavlib.models.Model

The model architecture known as ‘GoogLeNet’ or ‘Inception v1’. This particular model was pretrained on ImageNet, and was released by Google under the name ‘inception5h’.

class cavlib.models.MobilenetModel

Bases: cavlib.models.Model

Mobilenet v1, pretrained on ImageNet.