Skip to content

Latest commit

 

History

History
763 lines (615 loc) · 41.5 KB

usage.md

File metadata and controls

763 lines (615 loc) · 41.5 KB

raw2features - Usage Guide

raw2features reads a whole-slide image in OME-Zarr / OME-NGFF and writes patch-level foundation-model embeddings to its own *.embeddings.zarr store, with the storage backend and the model(s) independently swappable. Every embedding records the exact slide region it came from, so it maps back to its precise location.

This guide explains each command, what actually happens when you run it, and the behaviours that are easy to get wrong (idempotent re-runs, exact MPP, multi-model fan-out, thumbnails).

Contents

  1. Mental model
  2. Install
  3. Commands at a glance
  4. info - probe a slide
  5. embed - extract embeddings
  6. What embed actually does
  7. Re-runs & idempotency (the "already-embedded" behaviour)
  8. thumbnail - previews & QC overlays
  9. Cohort runs on SLURM
  10. Output reference
  11. FAQ / gotchas

1. Mental model

The pipeline is a chain of swappable seams; each embed run flows left to right:

OME-Zarr  ──reader──▶  segmenter ──▶  patcher ──▶  embedder(s) ──▶  sink ──▶  *.embeddings.zarr
(omezarr)             (otsu/none)   (grid,        (resnet50/dinov2/  (zarr)    + receipt (optional)
                                     exact MPP)    uni/uni2_h, …)
Seam Default What it does
reader omezarr resolves NGFF metadata (MPP, pyramid levels) + does random patch reads
segmenter otsu (or --no-seg) low-res tissue mask so only tissue patches are kept
patcher grid tiles a regular grid at an exact target MPP
embedder resnet50 (repeatable) patch → feature vector; one or many, run together
multiplex_strategy none optional named-channel adaptation before an RGB patch encoder
slide_embedder none patch-feature matrix → optional slide-level vector
sink zarr writes coords/grid_index/mask/features/<model> + provenance

List what's installed for any component: raw2features list embedders (or readers, segmenters, patchers, sinks, slide_embedders, multiplex_strategies).


2. Install

git clone https://github.com/CraigMyles/raw2features && cd raw2features
uv sync --extra zarr --extra image --extra torch --extra models

Build the venv on the machine you will run on - Torch wheels are architecture-specific, so a venv built on one arch (e.g. aarch64) will not import on another (e.g. an x86 cluster). Verify with raw2features version.


3. Commands at a glance

Command Purpose
raw2features info <slide.zarr> print NGFF version, MPP, pyramid levels, and the exact-MPP read plan - extracts nothing
raw2features sample <out.zarr> write a tiny synthetic OME-Zarr slide (quickstart / testing)
raw2features embed <slide.zarr> <out_dir> extract patch embeddings for one slide
raw2features embed-many <slide_dir> <out_dir> extract embeddings for a directory/cohort of slides (sharded, resumable)
raw2features verify <slide.zarr> --receipts-dir DIR --out-dir OUT exit 0 iff the requested output is already complete, source-bound, and validated
raw2features thumbnail <slide.zarr> <out_dir> write a thumbnail PNG (+ optional QC overlay)
raw2features export-spatialdata <store.embeddings.zarr> convert an embedding store to scverse SpatialData (squidpy/napari) - see INTEROP.md
raw2features export-h5 <store.embeddings.zarr> <out_dir> export features to HDF5 (one .h5 per model); needs the [h5] extra
raw2features validate-store <store.embeddings.zarr> check a store conforms to the embeddings-store spec
raw2features models list the model registry (name, dim, gated, source)
raw2features list <component> list installed plugins for a component type (readers, segmenters, patchers, embedders, sinks, slide_embedders, multiplex_strategies)
raw2features version print the version

Every option below is also visible via raw2features <command> --help.


4. info - probe a slide

raw2features info slide.zarr --mpp 1.0 --patch-size 224

Reads only metadata (no pixels extracted) and prints the NGFF version, level-0 MPP, every pyramid level (size, downsample, native MPP), and the read plan for your target MPP - which level it would read, at what pixel size, the resample factor, and the achieved MPP. Use it to sanity-check a slide before a run. Example:

mpp_level0:    0.2500
levels:
  L0: 102917 x 43525    downsample   1.00  mpp 0.2500
  ...
  L2:  25729 x 10881    downsample   4.00  mpp 1.0002
plan @ 1.0 um/px, 224px: read L2 (224px, resample 0.9998) -> achieved 1.0002 um/px [native, no resize]

5. embed - extract embeddings

Simplest run (open model, no token needed):

raw2features embed slide.zarr out/ -m resnet50 --mpp 1.0 --patch-size 224

Multiple extractors (decode each patch once, run every model on it):

raw2features embed slide.zarr out/ -m uni -m resnet50 -m dinov2 --mpp 1.0

Gated models (UNI, UNI2-h) need a HuggingFace token and access granted on the model page:

hf auth login                   # cached login; alternatively set HF_TOKEN
raw2features embed slide.zarr out/ -m uni

--hf-token remains available for automation, but cached login or the HF_TOKEN environment variable avoids exposing a token through shell history or process listings. Any token passed on the command line is redacted from stored command provenance.

Key options

Option Default Meaning
-m, --model resnet50 model name; repeatable for multi-model (-f/--feature-extractor are aliases)
--mpp per-model target µm/px; default = each model's recommended MPP (0.5 for most pathology FMs, 1.0 for scale-agnostic). A value overrides the scale while retaining each model's recommended patch size; combine it with --patch-size to force one shared grid
--source-mpp source metadata level-0 source µm/px override for an uncalibrated OME-Zarr whose spatial axes declare no physical unit; normally leave unset
--patch-size per-model patch side in px; default = each model's recommended size (224 for most / 256 KRONOS2 / 448 CONCH / 512 conch_v1_5). A value overrides the size; combine it with --mpp to force one shared grid
--step = patch stride in output px; < patch gives overlap
--no-seg off tile the whole slide (skip tissue masking)
--segmenter otsu tissue segmenter to use
--tissue-threshold 0.1 keep a grid cell if ≥ this fraction is tissue
--features-dtype float16 stored feature dtype
--stain-norm off stain-normalize each patch before embedding (macenko|reinhard|vahadane; vahadane needs raw2features[stain]); changes the features - use separate output dirs for with/without-norm experiments
--multiplex-strategy none adapt an ordinary RGB model to a named-channel multiplex source; built-in: channelwise
--marker all source channels select a multiplex marker; repeat to set the subset and order for a native model or strategy
--channel-names-file source metadata complete positional .txt, .csv, or .tsv panel when OME channel labels are absent or incomplete
--kronos2-additional-markers none complete KRONOS2 novel-marker CSV; conditionally loads the pinned BioLinkBERT text encoder
--config FILE none YAML extraction plan (extractions: list of {model, mpp?, patch_px?}); supersedes -m/--mpp/--patch-size. The same model may repeat (one grid each - the MPP-ablation case)
-s, --slide-encoder none slide-level encoder(s) run after patch embedding (e.g. titan); reads patch features from the store, no WSI re-read. Repeatable.
--qc none per-patch QC producer(s) writing grids/<key>/qc/<tool>/ (e.g. grandqc); needs the producer's extra (raw2features[grandqc])
--qc-stain-norm off normalize the QC input first (macenko|reinhard|vahadane; vahadane needs raw2features[stain]) - for a stain outside the QC model's domain
--device auto auto (best of cuda→mps→cpu) | cuda | mps | cpu
--devices = --device opt-in in-process multi-GPU, e.g. cuda:0,cuda:1; shards this slide's patches across them and gathers in coord order (output identical to one device). Patch-parallel multi-device execution is not supported with multiplex strategies or native multiplex models; see In-process multi-GPU in §6.
--batch-size 256 patches per forward pass (lower it on small GPUs)
--amp auto precision: auto (each model's card precision) | fp32 | bf16 | fp16
--snap-to-level off read a pyramid level natively (no resample); MPP = that level
--mpp-tolerance 0.001 read a level natively if within this fraction of target MPP
--allow-upsample off permit upsampling when the target is finer than level 0
--emit-geojson off also write <id>.patches.geojson (QuPath patch polygons)
--emit-thumbnail off also write a thumbnail + QC overlay (see §8)
--receipts-dir DIR none enable receipts + skip-if-complete (see §7)
--hf-token $HF_TOKEN token for gated models; cached login or the environment is preferred, and command provenance is redacted

See raw2features models for available extractors and which are gated.

Authenticated remote sources

The reader receives the complete remote URI in memory so query-authenticated OME-Zarr stores can be opened. The persisted source.uri, receipts, output identity, and captured CLI remove userinfo and known AWS, Google, Azure, and generic authentication parameters, while retaining semantic selectors such as object versions, generations, series, and fragments. Credential rotation therefore does not rename the output.

Query credentials must be valid for the Zarr prefix and all child metadata/chunk objects. A presigned S3 or GCS URL for one object usually cannot authorize the whole hierarchy; prefer native s3:// or gs:// credential-provider configuration in that case.


6. What embed actually does

For one slide, in order:

  1. (skip check) If --receipts-dir is set and the source-bound target is already complete for this exact config, the run returns skipped immediately (see §7).
  2. Read metadata via the omezarr reader (NGFF version, MPP, pyramid).
  3. Build the grid at the exact target MPP (below).
  4. Segment the tissue (unless --no-seg) on a low-res level and keep only the grid cells whose tissue fraction ≥ --tissue-threshold.
  5. (optional) Render the thumbnail/overlay if --emit-thumbnail.
  6. Embed, decode-once: for each batch, read the patches once, then run every requested model on that shared batch.
  7. Write the *.embeddings.zarr store + provenance, optional sidecars.
  8. (optional) Write a complete receipt if --receipts-dir is set.

Exact MPP (why patches are comparable)

Foundation-model embeddings are scale-sensitive, so every patch is sampled at the exact MPP you ask for - not "close enough". The patcher:

  • picks the nearest finer-or-equal pyramid level (native MPP ≤ target), so it only ever downsamples - it never invents detail by upscaling;
  • reads read_px = round(patch_px × resample) pixels at that level (resample = target_mpp / level_mpp ≥ 1);
  • downsamples that read to exactly patch_px (the stored patch is patch_px at the target MPP, independent of any model's input size - each model then resizes the patch to its input size internally).

If a level's native MPP is within --mpp-tolerance (default 0.1%) of the target, it is read natively with no resize (the common case for pyramids built at the target resolution), and the recorded achieved_mpp is that level's native MPP. Otherwise the patch is resampled and achieved_mpp == target. --snap-to-level forces a native level read (faster, MPP = that level); --allow-upsample permits going finer than level 0. raw2features info prints the exact plan.

Float source pixels

OME-NGFF float arrays do not necessarily declare whether RGB values use a normalized [0, 1] range or a byte-like [0, 255] range. The OME-Zarr reader therefore chooses one convention when the slide is opened and reuses it for every patch and pyramid level; it never rescales one patch differently because that patch happens to be locally dark. The decision uses the array fill value and a deterministic 3×3 lattice of level-0 chunks (up to the first three channels): any finite observed value above 1 selects the byte-like convention, otherwise the slide is treated as normalized. This bounded sample avoids scanning an entire WSI at open. read_region_channels remains native-dtype and unscaled for multiplex models.

RGB encoders on named-channel multiplex tissue images

--multiplex-strategy channelwise lets a registered RGB patch encoder process an OME-Zarr image with a c axis and positionally matching channel names:

raw2features embed multiplex_slide.ome.zarr multiplex_out \
  -m uni --multiplex-strategy channelwise \
  --marker CD3 --marker CK --marker DAPI \
  --multiplex-aggregation mean -s mean

Each selected channel is processed independently. raw2features resolves a normalization for that channel, repeats the normalized single-channel patch across RGB, and applies the encoder's registered preprocessing and pooling (for example CLS for UNI or CLS+mean-patch for Virchow2). The marker vectors are combined with mean (default) or ordered concat; concat requires an explicit marker order. Without an explicit selection, every physical channel must have a unique name and all channels are used in source order. If metadata contains unnamed positions, supply a complete --channel-names-file or explicitly select the named markers to exclude those positions. Selected names must resolve unambiguously. No channel name has implicit inclusion or exclusion semantics for embedding. Nuclear marker names are resolved separately when segmentation is enabled.

--multiplex-normalization percentile is the default. It records each selected channel's 1st and 99th percentiles from the first image-pyramid level whose longest side is at most 2048 pixels. Set alternative bounds with --multiplex-percentile-low and --multiplex-percentile-high. This is a bounded-memory, reproducible raw2features rule rather than a platform-specific calibration. The level limit can be changed with --multiplex-normalization-max-side-px; a larger value may choose a finer level and use more memory. If no pyramid level fits the cap, raw2features asks for a coarser level or a higher cap instead of silently scanning level 0. --multiplex-normalization dtype instead maps the source dtype range to [0,1]; floating-point sources must already be finite in [0,1]. Percentiles describe the selected pyramid level, so spatial downsampling can smooth extrema relative to level 0; the exact level and measured values are fingerprinted. The selected markers and physical indices, order, normalization values and source level, RGB conversion, base-model fingerprint and pooling, aggregation, and output dimension are stored in the grid's model/panel metadata and output fingerprint.

The RGB vision-encoder baselines evaluated in the public KRONOS paper provide structural inspiration. This strategy adapts ordinary RGB encoders rather than implementing the marker-aware KRONOS architecture. Every marker is normalized independently, repeated to RGB, embedded independently, then aggregated with a fully recorded deterministic recipe.

Channel identity normally comes from omero.channels. If those labels are absent or incomplete, provide a complete positional panel without rewriting the source:

raw2features embed multiplex_slide.ome.zarr multiplex_out \
  -m uni --multiplex-strategy channelwise \
  --channel-names-file channel_names.tsv \
  --marker CD3 --marker CK --marker DAPI

The UTF-8 .txt, .csv, or .tsv file contains exactly one unique name for every physical C-axis position, in order. A conventional one-column header is optional. Any existing non-empty OME labels must agree; conflicts and channel-count mismatches fail before model loading or output mutation. The file defines physical channel identity, while repeated --marker options select and order a subset. Only the resolved ordered panel and ordered marker request affect request identity; the file path and serialization do not. Store provenance also records whether effective labels came from OME metadata or a channel-names file and, when they differ, preserves the original OME labels. As with other non-secret arguments, the recorded CLI invocation can include the file path. One file applies to every slide in an embed-many invocation, so heterogeneous panels should use separate runs.

Multiplex runs use nuclear-channel tissue masking when segmentation is enabled. They require one unambiguous nuclear channel or a recognized same-stain group, such as DNA1/DNA2 or repeated DAPI channels. A multi-channel group is averaged in float32 before thresholding. This differs from v0.1's first-match behaviour and is bound into a new grid identity rather than reusing the old mask. --no-seg is fully supported and tiles the complete image, which is useful for segmentation-free analyses or panels without a suitable nuclear channel; it can also retain more background and require more compute.

Patch geometry remains configurable. If an extracted patch does not match the encoder's registered input size, raw2features resizes it before model preprocessing. Smaller, cell-scale fields are therefore mechanically supported, but resizing does not add image detail and the changed physical field of view should be validated for the intended task.

Strategy-derived outputs use a cohort-stable effective key under features/. Model-agnostic slide poolers (mean, max, meanmax) accept them, while learned slide encoders tied to another patch model are rejected. The first implementation processes one device per slide.

The strategy is a plugin (raw2features.multiplex_strategies), with separate panel/config preparation and slide-specific binding phases. Future versions may add marker-to-R/G/B mappings or learned channel adapters if there is demand; either can use the same panel-binding and fingerprint contract. Third-party strategies can accept a JSON object through --multiplex-params; the built-in channelwise strategy uses its explicit options and does not accept additional strategy parameters.

Native multiplex encoders

Native multiplex encoders consume selected marker channels together rather than converting each channel to RGB. Run KRONOS2 after installing its optional stack and accepting the gated institutional-access terms:

pip install "raw2features[all,kronos2]"
hf auth login

raw2features embed multiplex_slide.ome.zarr multiplex_out \
  -m kronos2 --marker CD3 --marker CD8 --marker DAPI

kronos2 is a marker-aware ViT-B/16 that returns a 768-dimensional CLS vector in fp32. Its default patch is 256 pixels. A custom --patch-size is accepted only when it is a multiple of 16. Marker names are matched exactly after the model's separator-insensitive normalization; KRONOSv1 aliases and compound-name fallbacks are not applied. --marker selects physical channels and fixes their order before completion checks. The full source panel, selected indices, model-resolved names, and order remain in provenance and the output fingerprint.

The released vocabulary contains DAPI and DRAQ5 but not Hoechst or DNA1/DNA2 labels. Those common CODEX nuclear stains are not silently aliased. Exclude one from the encoder with an explicit --marker list while still allowing nuclear segmentation to use it, or register it through --kronos2-additional-markers with dataset-specific statistics. When DAPI is absent and DRAQ5 is selected, raw2features follows the upstream example by passing DRAQ5 as preferred_dapi, which applies the model's DAPI normalization statistics; the recorded policy is not configurable in v0.2.1.

Upstream recommends batches of at least eight for reproducibility. raw2features uses fixed eight-sample forward microbatches and pads a shorter final microbatch before discarding the padded outputs. This runtime contract is recorded. The global --batch-size may be left unchanged; it does not override this internal cap. Bitwise equality across different hardware is not promised. On x86-64 Linux with CPython 3.11 or 3.12, the [kronos2] extra installs the authors' pinned xFormers reference path; this path is validated with Python 3.12. Python 3.13 and other platforms use the upstream PyTorch attention fallback, which is slower and may use more memory. The pinned Torch release does not support Python 3.14. --compile is not supported for KRONOS2 in v0.2.1; use the validated eager path.

The released KRONOS2 repository includes 288 usable marker-metadata entries, 268 marked as pretraining markers, and stores their text vectors in the checkpoint. Normal inference therefore never downloads BioLinkBERT. To register a marker outside that released metadata, supply a complete CSV:

raw2features embed multiplex_slide.ome.zarr multiplex_out \
  -m kronos2 --marker NovelMarker --marker DAPI \
  --kronos2-additional-markers additional_markers.csv

Every CSV row requires marker_name, marker_full_name, compartment, family, family_desc, mean, and std. The mean and standard deviation must be finite, with a positive standard deviation. compartment and family must exactly reuse categories from KRONOS2's pinned marker metadata, and a novel name must not collide with that vocabulary under the model's separator-insensitive matching.

Statistics are supplied in the scaled intensity domain consumed by KRONOS2: uint8 values divided by 255, wider unsigned integers divided by 65535, or floating values divided by 400. Signed-integer sources are rejected. raw2features does not estimate these dataset statistics. The option loads the separately pinned BioLinkBERT model, about 1.33 GB, only for novel-marker text vectors. The canonical CSV content, intensity statistics, BioLinkBERT revision, and text-encoding recipe are included in the model fingerprint.

Run native multiplex and ordinary RGB encoders in separate commands so each receives the correct source-channel representation. An additive second command can still append its features to an existing matching grid. Native multiplex patch execution uses one device per slide; embed-many --devices can distribute different slides across devices.

Segmentation

The default otsu segmenter runs Otsu-on-saturation + morphology on a low-res level (~8 µm/px) to produce a tissue mask, and the patcher keeps a grid cell only if its tissue fraction ≥ --tissue-threshold. --no-seg tiles the whole slide (and omits the mask array). Other segmenters (incl. the opt-in deep grandqc) are pluggable via the segmenters seam - see SEGMENTATION.md for the methods and the GrandQC QC layer.

Multi-model fan-out (decode-once)

With several -m flags (-f is an alias), each patch is read and decoded once and reused across every model in the batch - the slow part (IO + decode) is not repeated per model. Each model applies its own preprocessing (mean/std/input size, sourced from its model card), and writes into its own features/<model> array.

In-process multi-GPU (--devices, opt-in)

By default the pipeline runs on the single --device (CPU or one GPU) and is fully portable. --devices is an opt-in runtime flag that spreads work across several devices in one process; it never changes the embeddings (it is excluded from the config hash), so a store built with it is identical to a single-device one and resumes interchangeably.

Multi-device patch-parallel embed --devices execution does not currently support multiplex strategies or native multiplex models. A multiplex run uses one device per slide. embed-many --devices may still distribute distinct slides across devices.

  • embed (one slide) - patch-parallel, latency mode. embed … --devices cuda:0,cuda:1 shards the slide's patches into contiguous blocks, embeds each block on its own device (one model copy + reader per device), and gathers the features back in exact coord order - so the output is byte-identical to a one-GPU run, just faster for a single large slide.
  • embed-many (a shard of slides) - slide-parallel, throughput mode. embed-many … --devices cuda:0,cuda:1 runs one warm worker per device, each pulling slides off a shared queue; every slide is embedded entirely on one device, so its output equals a one-GPU run. This is the path for a multi-GPU workstation without a scheduler (a SLURM array already parallelises one GPU per task - see §9).
  • benchmark … --devices … measures the patch-parallel single-slide path (wall-time / patches/s stay accurate; the per-stage breakdown is single-device only).

Equivalence is exact on CPU and across distinct GPUs (each patch is processed on one device exactly as in the single-device path). Listing the same GPU twice (cuda:0,cuda:0) is only a single-GPU test convenience and may differ by ~1 ULP due to shared-stream float reordering.


7. Re-runs & idempotency

This section explains the already-embedded behaviour: how re-runs detect existing work and what they do with it. Worth reading before a cohort run.

Adding a model / resuming is automatic (store-aware)

embed inspects any existing <slide>.embeddings.zarr before doing work, so re-runs are additive and idempotent - no --receipts-dir required:

  • A requested model already present and output-validated against its current model fingerprint is skipped.
  • A missing model is embedded and appended in place: features/<model> is added next to the existing arrays when it shares that grid's geometry. Existing models and coords are left untouched.
  • A requested geometry not already in the store is added as another grids/<key>/ group. A difference in --mpp, --patch-size, segmentation, or another geometry field is not an error and does not wipe existing grids or mix incompatible rows in one grid.
  • If all requested models and auxiliary outputs are already present, no patch features are recomputed (a model-only run returns status: skipped).

Use --force only when you intentionally want to rebuild the whole store from scratch.

raw2features embed slide.zarr out/ -m uni              # writes features/uni
raw2features embed slide.zarr out/ -m virchow2         # ADDS features/virchow2; uni untouched
raw2features embed slide.zarr out/ -m uni -m virchow2  # both present -> nothing to do
raw2features embed slide.zarr out/ -m conch_v1_5       # ADDS its 0.5/512 grid; other grids untouched

The shared extraction/storage grid is identified by a grid_hash, recorded in the header. Model-specific settings such as resolved AMP live in each output fingerprint instead, so changing them replaces that model in the same grid. "Present" is strict: a model counts only if features/<model> has the current expected dimension and a matching output fingerprint in both the array and grid header, is fully finite, and has len(coords) == n_patches. For legacy arrays without a post-write fingerprint commit marker, an all-zero final row is additionally treated as an unwritten tail. The fingerprint covers the effective weights, preprocessing, pooling, resolved AMP, and loader construction. A truncated, damaged, legacy-unfingerprinted, or stale-contract array is re-embedded; unrelated model arrays and coordinates remain untouched. A derived slide/<model> fingerprint includes its patch input fingerprint, so stale slide vectors are recomputed too.

Receipts - the fast path for cohort runs

--receipts-dir DIR adds a cheap short-circuit on top of the store inspection above. After confirming that any existing target store belongs to the requested source, embed checks for a validated complete receipt for this exact configuration (including the model set), source, and output target. If found it returns skipped; otherwise it processes and writes a complete receipt at the end. This is what the SLURM array uses to skip finished slides cheaply. Completeness is validated against the actual output store, never just the receipt file.

The receipt's config hash covers the content-affecting settings and the model set:

reader, models (order-independent), segmenter / --no-seg, --mpp, --patch-size,
--source-mpp (when set), --step, --tissue-threshold, --features-dtype,
--stain-norm, --snap-to-level, --mpp-tolerance, --allow-upsample, --amp

For multiplex requests, the hash also covers the ordered physical marker selection and resolved positional channel names. Strategy-derived requests additionally cover their normalization recipe, pyramid-level limit, aggregation, and other strategy parameters. Native model parameters, including KRONOS2 additional-marker metadata, are covered when present. The resolved physical nuclear-channel indices enter grid identity when masking is enabled. The --channel-names-file path and serialization are not hashed; their parsed effective names are.

Runtime-only knobs do not change the hash: --device, --devices, --batch-size, --read-workers, --read-block, --compile, --emit-geojson, --emit-thumbnail, --thumbnail-mpp, --max-px, --slide-encoder, --qc, --qc-stain-norm, --qc-artifact-mpp, --output-zarr-format, --force. --force is the exception for control flow: it deliberately bypasses skipping and rebuilds the target without changing the content identity.

verify - the standalone skip check

raw2features verify <slide> --receipts-dir DIR --out-dir OUT <same content flags> runs exactly the same completeness check and exits 0 (complete) or 1 (incomplete). It is how the SLURM array skips finished slides. A receipt is trusted only when its slide ID, credential-free source, requested output target, and the actual store's root/grid source provenance all agree. Pass --out-dir and the same content-affecting flags you pass to embed, or the check is not target-aware (or the config hash will not match).

For library callers, verify also recognizes the direct single-grid receipt written by the public run_slide primitive. Multi-grid requests use embed_slide and its whole-run receipt.

raw2features verify slide.zarr --receipts-dir receipts/ --out-dir embeddings/ -m uni -m resnet50 --mpp 1.0 --quiet

Forcing a rebuild

Pass --force to ignore an existing store and rebuild it from scratch (all requested models recomputed, the store overwritten). It also bypasses a valid receipt's fast path. Deleting the output store (and any receipt) has the same effect.

Auxiliary outputs are produce-if-missing

An explicit --emit-thumbnail, --emit-geojson, --qc, or -s/--slide-encoder request is checked against the matching grid even when its patch features are already complete. Missing outputs are produced without re-embedding patches; existing valid outputs are left alone. These runtime products do not change the patch-grid identity or receipt content hash. The standalone thumbnail and slide-embed commands remain convenient when you only want to add those products.

In a multi-grid invocation, a slide encoder runs on the grid containing its required patch encoder (for example, TITAN uses the conch_v1_5 grid). Model-agnostic pooling encoders run on each requested grid. Standalone slide-embed can infer an unambiguous grid from the required patch model; otherwise pass --grid (and --patch-model when needed).


8. thumbnail - previews & QC overlays

Two ways to get a thumbnail; both are optional and off by default so they never slow a large feature-extraction run.

Standalone (run before or after the heavy embed run; one cheap coarse read):

raw2features thumbnail slide.zarr out/                 # plain preview
raw2features thumbnail slide.zarr out/ --overlay       # + tissue mask + kept-patch grid

Inline with embeddings (piggybacks on the already-open + segmented slide):

raw2features embed slide.zarr out/ -m uni --emit-thumbnail

What it does / sizing

A thumbnail is an RGB overview read from a coarse pyramid level (never the full-resolution level 0). By default it renders at the segmentation MPP (8 µm/px), chosen by the same nearest-level rule the segmenter uses. That single choice gives two things at once:

  • the thumbnail, the tissue mask, and the patch grid share a pixel grid, so the --overlay needs no resampling - it is a faithful QC view ("did segmentation + patching do the right thing on this slide?");
  • it is also a sensible size (≈3 k px wide for a 0.25 µm/px slide).

Override the resolution with --thumbnail-mpp <µm/px> or --max-px <N> (cap the longest side; overrides --thumbnail-mpp). Pyramid-level indices are never exposed - they aren't portable across slides, and the whole tool derives the level from MPP.

For the standalone --overlay, the segmenter + patcher are re-run (cheaply) so the overlay reflects the grid params you pass (--mpp, --patch-size, --step, --tissue-threshold, --segmenter).

Output

<slide_id>.thumbnail.png            # plain overview
<slide_id>.thumbnail.overlay.png    # tissue tint + kept-patch outlines (with --overlay / --emit-thumbnail)

Inline (--emit-thumbnail) also records the thumbnail metadata in the store's .zattrs (raw2features.thumbnail).


9. Cohort runs on SLURM

slurm/ ships a one-slide-per-task array, a per-batch ("shard") array, and a pre-flight checker - see slurm/README.md for a plain-language overview and which script to use. Each task is idempotent: it verifys the slide first and skips if already complete, so re-submitting the same array safely resumes only the missing/failed slides.

Before embed-many shards or loads models, it requires every resolved cohort input to derive a unique output ID. Ordinary local IDs remain basename-based for v0.1 compatibility, so same-named slides from different directories must be renamed or run as separate commands with different output directories. Repeated identical manifest rows are also rejected; a per-output lock across independent invocations is not implied.

cd raw2features && uv sync --extra zarr --extra image --extra torch --extra models
# Add any model-specific extras / pinned git packages from docs/MODELS.md.
export SLIDE_DIR=/path/to/raw            # dir of *.zarr stores (top-level only)
export OUT_DIR=/path/to/embeddings
export MODELS="uni resnet50"             # space-separated; each becomes a -m flag
export HF_TOKEN=hf_...                    # only if MODELS includes a gated model

bash slurm/preflight.sh                  # validates venv/arch, models/HF, slide count
mkdir -p logs                            # SLURM opens --output here before the job runs
shopt -s nullglob
SLIDES=("$SLIDE_DIR"/*.zarr)
shopt -u nullglob
N=${#SLIDES[@]}
(( N > 0 )) || { echo "no *.zarr slides found" >&2; exit 1; }
sbatch --array=0-$((N-1))%64 --export=ALL,SLIDE_DIR,OUT_DIR,MODELS,HF_TOKEN slurm/embed_array.sbatch

Notes:

  • Set --partition in embed_array.sbatch to your cluster's GPU partition (the template ships a gpu placeholder).
  • Lower BATCH_SIZE on smaller GPUs (UNI2-h is a ViT-H and is memory-hungry).
  • The slide list is a top-level glob ($SLIDE_DIR/*.zarr) - never a recursive find into the stores - and is sorted byte-order (LC_COLLATE=C) so task i maps to the same slide on every node.
  • The pre-flight catches the most common cohort-killer: a venv built on the wrong architecture (it imports Torch to check).
  • No scheduler? On a multi-GPU workstation, embed-many … --devices cuda:0,cuda:1,… is the equivalent of the array: one warm worker per GPU, slides distributed across them, per-slide receipts for idempotent resume (see In-process multi-GPU in §6).

10. Output reference

<slide_id>.embeddings.zarr/
├── .zattrs                     # root header: source, provenance + a grids index
└── grids/<key>/                # one per geometry (mpp, patch_px) - usually just one
    ├── .zattrs                 # this grid's full header (below)
    ├── coords/                 # (N,2) int32  level-0 (x,y) top-left of each patch
    ├── grid_index/             # (N,2) int32  (row,col) on the tiling grid
    ├── mask/                   # (n_rows,n_cols) uint8  fraction of each cell that is tissue; /255 for [0,1] (omitted with --no-seg)
    ├── features/<model>/       # (N,dim) float16  one array per extractor
    └── slide/<output-key>/     # (1,dim) float32  optional slide-level vectors

<slide_id>.patches.geojson          # optional (--emit-geojson): per-patch polygons for QuPath
<slide_id>.thumbnail.png            # optional (--emit-thumbnail / thumbnail cmd)
<slide_id>.thumbnail.overlay.png    # optional QC overlay
<slide_id>.<key>.patches.geojson              # corresponding sidecar for each additional grid
<slide_id>.<key>.thumbnail.overlay.png        # corresponding overlay for each additional grid

A store nests its patch sets under grids/<key>/ (one child per extraction geometry - mpp0.5_px224, etc.); a one-geometry run is just a single grid. Open one grid with the reader helper (raw2features.core.store.open_grid(store) returns the sole grid, or pass a key), and export/info default to the sole grid (a multi-grid store wants --grid). The first-created grid keeps the backward-compatible unsuffixed GeoJSON/overlay names; later grids use their key as shown above. The plain thumbnail is shared by the slide.

Spatial-provenance contract (within a grid). Row i refers to the same patch across every array of that grid: coords[i], grid_index[i], and features/<model>[i] are aligned 1:1 in identical order. Combined with patching.level0_patch (the patch's level-0 pixel extent), any embedding is invertible to the exact slide region [x, y, x+level0_patch, y+level0_patch] at level 0.

Each grid's .zattrs["raw2features"] records the full, self-describing header: schema_version, provenance (version, CLI, git SHA, host, GPU), thumbnail (if emitted), source (uri, ngff_version, reader, mpp_level0, dimensions/downsamples, source axes, per-axis physical scale, source translation/origin, and for multiplex sources the effective positional channel names, their source, and any differing original OME labels), patching (target/achieved MPP, patch_px, read_level, read_px, resample, level0_patch, level0_step, n_patches, grid_shape, coords_convention: level0_xy), segmentation (including the resolved nuclear channel binding when used), top-level panel, and per-model models.<name> (source, embedding_dim, input_size, pooling, mean, std, transform_source_url, license, gated, weights pins, output fingerprint, and any multiplex contract). The root .zattrs carries the shared source/provenance plus a grids index (key → {mpp, patch_px, n_patches, models, grid_hash}) for discovery without opening each grid.

Read it back:

from raw2features.core.store import open_grid

# Omit the key only when the store contains exactly one grid.
g = open_grid("out/<slide_id>.embeddings.zarr", key="mpp0.5_px224")
print(g["features/uni"].shape)          # (N, 1024)
print(dict(g.attrs)["raw2features"]["patching"]["achieved_mpp"])

11. FAQ / gotchas

embed --emit-thumbnail produced no thumbnail. A missing thumbnail should be produced even when the patch arrays are complete, while an existing valid thumbnail is left unchanged. Check the command's error and output path; in a multi-grid store, non-primary overlays have the grid key in their filename. The standalone raw2features thumbnail … --overlay command is also available when only a preview is needed.

What happens when I add a model? A model with the same extraction geometry is appended to that grid. A model with a different geometry gets a new grid. Existing valid arrays are preserved; only a missing, damaged, legacy-unfingerprinted, or stale-fingerprint model output is recomputed.

verify always says incomplete. It must be given the same content-affecting flags as the embed that produced the store (same -m models, --mpp, --patch-size, etc.) so the config hash matches. Pass the same output directory as --out-dir so the receipt is also bound to the intended target store.

import torch / model load fails on the cluster. The venv was built for a different CPU architecture. Re-run uv sync on the target machine.

Gated model 401 / access error. You need both a token (--hf-token / HF_TOKEN / cached login) and access granted on the model's HuggingFace page. Open models (resnet50, dinov2) need neither. Prefer HF_TOKEN or hf auth login so the token never appears in shell history or a process listing.

Embeddings are float16. Default storage dtype (inode/byte-light). Use --features-dtype float32 if you need full precision on disk.

achieved_mpp isn't exactly my --mpp. A level within --mpp-tolerance (0.1%) of the target is read natively (no resize), so achieved_mpp is that level's native MPP. Tighten --mpp-tolerance 0 to force exact resampling, or use --snap-to-level to read a level natively on purpose.