slideflow.Mosaic¶

slideflow.Mosaic plots tile images onto a map of slides, generating a mosaic map.

The idea of a mosaic map is to visualize image feature variation across slides and among categories, in an attempt to better understand the kinds of image features discriminative models might be using to generate class predictions. They are created by first generating whole-dataset layer features (using slideflow.DatasetFeatures), which are then mapped into two-dimensional space using UMAP dimensionality reduction (slideflow.SlideMap). This resulting SlideMap is then passed to slideflow.Mosaic, which overlays tile images onto the dimensionality-reduced slide map.

An example of a mosaic map can be found in Figure 4 of this paper. It bears some resemblence to the Activation Atlases created by Google and OpenAI, without the use of feature inversion.

See Mosaic Maps for an example of how a mosaic map can be used in the context of a project.

Visualization of plotted image tiles.

Generate a mosaic map, which visualizes plotted image tiles.

Creating a mosaic map requires two components: a set of images and corresponding coordinates. Images and coordinates can either be manually provided, or the mosaic can dynamically read images from TFRecords as needed, reducing memory requirements.

The first argument provides the images, and may be any of the following:

A list or array of images (np.ndarray, HxWxC)
A list of tuples, containing (slide_name, tfrecord_index)
A slideflow.SlideMap object

The second argument provides the coordinates, and may be any of:

A list or array of (x, y) coordinates for each image
None (if the first argument is a SlideMap, which has coordinates)

If images are to be read dynamically from tfrecords (with a SlideMap, or by providing tfrecord indices directly), the keyword argument tfrecords must be specified with paths to tfrecords.

Published examples:

Figure 4: https://doi.org/10.1038/s41379-020-00724-3
Figure 6: https://doi.org/10.1038/s41467-022-34025-x

Examples

Generate a mosaic map from a list of images and coordinates.

# Example data (images are HxWxC, np.ndarray)
images = [np.ndarray(...), ...]
coords = [(0.2, 0.9), ...]

# Generate the mosaic
mosaic = Mosaic(images, coordinates)

Generate a mosaic map from tuples of TFRecord paths and indices.

# Example data
paths = ['/path/to/tfrecord.tfrecords', ...]
idx = [253, 112, ...]
coords = [(0.2, 0.9), ...]
tuples = [(tfr, idx) for tfr, i in zip(paths, idx)]

# Generate mosaic map
mosaic = sf.Mosaic(tuples, coords)

Generate a mosaic map from a SlideMap and list of TFRecord paths.

# Prepare a SlideMap from a project
P = sf.Project('/project/path')
ftrs = P.generate_features('/path/to/model')
slide_map = sf.SlideMap.from_features(ftrs)

# Generate mosaic
mosaic = Mosaic(slide_map, tfrecords=ftrs.tfrecords)

Parameters:

images (list(np.ndarray), tuple, slideflow.SlideMap) – Images from which to generate the mosaic. May be a list or array of images (np.ndarray, HxWxC), a list of tuples, containing (slide_name, tfrecord_index), or a slideflow.SlideMap object.
coords (list(str)) – Coordinates for images. May be a list or array of (x, y) coordinates for each image (of same length as images), or None (if images is a SlideMap object).

Keyword Arguments:

tfrecords (list(str), optional) – TFRecord paths. Required if images is either a SlideMap object or a list of tuples containing (slide_name, tfrecord_index). Defaults to None.
num_tiles_x (int, optional) – Mosaic map grid size. Defaults to 50.
tile_select (str, optional) – ‘first’, ‘nearest’, or ‘centroid’. Determines how to choose a tile for display on each grid space. If ‘first’, will display the first valid tile in a grid space (fastest; recommended). If ‘nearest’, will display tile nearest to center of grid space. If ‘centroid’, for each grid, will calculate which tile is nearest to centroid tile_meta. Defaults to ‘nearest’.
tile_meta (dict, optional) – Tile metadata, used for tile_select. Dictionary should have slide names as keys, mapped to list of metadata (length of list = number of tiles in slide). Defaults to None.
normalizer ((str or slideflow.norm.StainNormalizer), optional) – Normalization strategy to use on image tiles. Defaults to None.
normalizer_source (str, optional) – Stain normalization preset or path to a source image. Valid presets include ‘v1’, ‘v2’, and ‘v3’. If None, will use the default present (‘v3’). Defaults to None.

Methods¶

generate_grid(self, num_tiles_x: int = 50, tile_meta: Dict | None = None, tile_select: str = 'first', max_dist: float | None = None)¶

Generate the mosaic map grid.

Parameters:

num_tiles_x (int, optional) – Mosaic map grid size. Defaults to 50.
tile_meta (dict, optional) – Tile metadata, used for tile_select. Dictionary should have slide names as keys, mapped to list of metadata (length of list = number of tiles in slide). Defaults to None.
tile_select (str, optional) – ‘first’, ‘nearest’, or ‘centroid’. Determines how to choose a tile for display on each grid space. If ‘first’, will display the first valid tile in a grid space (fastest; recommended). If ‘nearest’, will display tile nearest to center of grid space. If ‘centroid’, for each grid, will calculate which tile is nearest to centroid tile_meta. Defaults to ‘nearest’.

plot(self, figsize: Tuple[int, int] = (200, 200), focus: List[str] | None = None, focus_slide: str | None = None, background: str = '#dfdfdf', pool: Any | None = None) → None¶

Initializes figures and places image tiles.

If in a Jupyter notebook, the heatmap will be displayed in the cell output. If running via script or shell, the heatmap can then be shown on screen using matplotlib plt.show():

import slideflow as sf
import matplotlib.pyplot as plt

heatmap = sf.Heatmap(...)
heatmap.plot()
plt.show()

Parameters:

figsize (Tuple[int, int], optional) – Figure size. Defaults to (200, 200).
focus (list, optional) – List of tfrecords (paths) to highlight on the mosaic. Defaults to None.
focus_slide (str, optional) – Highlight tiles from this slide. Defaults to None.

points_at_grid_index(self, x, y)¶

save(self, filename: str, **kwargs: Any) → None¶

Saves the mosaic map figure to the given filename.

Parameters:

filename (str) – Path at which to save the mosiac image.

Keyword Arguments:

figsize (Tuple[int, int], optional) – Figure size. Defaults to (200, 200).
focus (list, optional) – List of tfrecords (paths) to highlight on the mosaic.

save_report(self, filename: str) → None¶: Saves a report of which tiles (and their corresponding slide) were displayed on the Mosaic map, in CSV format.

view(self, slides: List[str] | None = None) → None¶

Open Mosaic in Slideflow Studio.

See Slideflow Studio: Live Visualization for more information.

Parameters:: slides (list(str), optional) – Path to whole-slide images. Used for displaying image tile context when hovering over a mosaic grid. Defaults to None.