raw2features reads a whole-slide image in OME-Zarr / OME-NGFF and writes
patch-level foundation-model embeddings to its own *.embeddings.zarr
store, with the storage backend and the model(s) independently swappable. Every
embedding records the exact slide region it came from, so it maps back to its precise location.
This guide explains each command, what actually happens when you run it, and the behaviours that are easy to get wrong (idempotent re-runs, exact MPP, multi-model fan-out, thumbnails).
- Mental model
- Install
- Commands at a glance
info- probe a slideembed- extract embeddings- What
embedactually does - Re-runs & idempotency (the "already-embedded" behaviour)
thumbnail- previews & QC overlays- Cohort runs on SLURM
- Output reference
- FAQ / gotchas
The pipeline is a chain of swappable seams; each embed run flows left to right:
OME-Zarr ──reader──▶ segmenter ──▶ patcher ──▶ embedder(s) ──▶ sink ──▶ *.embeddings.zarr
(omezarr) (otsu/none) (grid, (resnet50/dinov2/ (zarr) + receipt (optional)
exact MPP) uni/uni2_h, …)
| Seam | Default | What it does |
|---|---|---|
| reader | omezarr |
resolves NGFF metadata (MPP, pyramid levels) + does random patch reads |
| segmenter | otsu (or --no-seg) |
low-res tissue mask so only tissue patches are kept |
| patcher | grid |
tiles a regular grid at an exact target MPP |
| embedder | resnet50 (repeatable) |
patch → feature vector; one or many, run together |
| multiplex_strategy | none | optional named-channel adaptation before an RGB patch encoder |
| slide_embedder | none | patch-feature matrix → optional slide-level vector |
| sink | zarr |
writes coords/grid_index/mask/features/<model> + provenance |
List what's installed for any component: raw2features list embedders (or readers,
segmenters, patchers, sinks, slide_embedders, multiplex_strategies).
git clone https://github.com/CraigMyles/raw2features && cd raw2features
uv sync --extra zarr --extra image --extra torch --extra modelsBuild the venv on the machine you will run on - Torch wheels are
architecture-specific, so a venv built on one arch (e.g. aarch64) will not import
on another (e.g. an x86 cluster). Verify with raw2features version.
| Command | Purpose |
|---|---|
raw2features info <slide.zarr> |
print NGFF version, MPP, pyramid levels, and the exact-MPP read plan - extracts nothing |
raw2features sample <out.zarr> |
write a tiny synthetic OME-Zarr slide (quickstart / testing) |
raw2features embed <slide.zarr> <out_dir> |
extract patch embeddings for one slide |
raw2features embed-many <slide_dir> <out_dir> |
extract embeddings for a directory/cohort of slides (sharded, resumable) |
raw2features verify <slide.zarr> --receipts-dir DIR --out-dir OUT |
exit 0 iff the requested output is already complete, source-bound, and validated |
raw2features thumbnail <slide.zarr> <out_dir> |
write a thumbnail PNG (+ optional QC overlay) |
raw2features export-spatialdata <store.embeddings.zarr> |
convert an embedding store to scverse SpatialData (squidpy/napari) - see INTEROP.md |
raw2features export-h5 <store.embeddings.zarr> <out_dir> |
export features to HDF5 (one .h5 per model); needs the [h5] extra |
raw2features validate-store <store.embeddings.zarr> |
check a store conforms to the embeddings-store spec |
raw2features models |
list the model registry (name, dim, gated, source) |
raw2features list <component> |
list installed plugins for a component type (readers, segmenters, patchers, embedders, sinks, slide_embedders, multiplex_strategies) |
raw2features version |
print the version |
Every option below is also visible via raw2features <command> --help.
raw2features info slide.zarr --mpp 1.0 --patch-size 224Reads only metadata (no pixels extracted) and prints the NGFF version, level-0 MPP, every pyramid level (size, downsample, native MPP), and the read plan for your target MPP - which level it would read, at what pixel size, the resample factor, and the achieved MPP. Use it to sanity-check a slide before a run. Example:
mpp_level0: 0.2500
levels:
L0: 102917 x 43525 downsample 1.00 mpp 0.2500
...
L2: 25729 x 10881 downsample 4.00 mpp 1.0002
plan @ 1.0 um/px, 224px: read L2 (224px, resample 0.9998) -> achieved 1.0002 um/px [native, no resize]
Simplest run (open model, no token needed):
raw2features embed slide.zarr out/ -m resnet50 --mpp 1.0 --patch-size 224Multiple extractors (decode each patch once, run every model on it):
raw2features embed slide.zarr out/ -m uni -m resnet50 -m dinov2 --mpp 1.0Gated models (UNI, UNI2-h) need a HuggingFace token and access granted on the model page:
hf auth login # cached login; alternatively set HF_TOKEN
raw2features embed slide.zarr out/ -m uni--hf-token remains available for automation, but cached login or the HF_TOKEN
environment variable avoids exposing a token through shell history or process listings.
Any token passed on the command line is redacted from stored command provenance.
| Option | Default | Meaning |
|---|---|---|
-m, --model |
resnet50 |
model name; repeatable for multi-model (-f/--feature-extractor are aliases) |
--mpp |
per-model | target µm/px; default = each model's recommended MPP (0.5 for most pathology FMs, 1.0 for scale-agnostic). A value overrides the scale while retaining each model's recommended patch size; combine it with --patch-size to force one shared grid |
--source-mpp |
source metadata | level-0 source µm/px override for an uncalibrated OME-Zarr whose spatial axes declare no physical unit; normally leave unset |
--patch-size |
per-model | patch side in px; default = each model's recommended size (224 for most / 256 KRONOS2 / 448 CONCH / 512 conch_v1_5). A value overrides the size; combine it with --mpp to force one shared grid |
--step |
= patch | stride in output px; < patch gives overlap |
--no-seg |
off | tile the whole slide (skip tissue masking) |
--segmenter |
otsu |
tissue segmenter to use |
--tissue-threshold |
0.1 |
keep a grid cell if ≥ this fraction is tissue |
--features-dtype |
float16 |
stored feature dtype |
--stain-norm |
off | stain-normalize each patch before embedding (macenko|reinhard|vahadane; vahadane needs raw2features[stain]); changes the features - use separate output dirs for with/without-norm experiments |
--multiplex-strategy |
none | adapt an ordinary RGB model to a named-channel multiplex source; built-in: channelwise |
--marker |
all source channels | select a multiplex marker; repeat to set the subset and order for a native model or strategy |
--channel-names-file |
source metadata | complete positional .txt, .csv, or .tsv panel when OME channel labels are absent or incomplete |
--kronos2-additional-markers |
none | complete KRONOS2 novel-marker CSV; conditionally loads the pinned BioLinkBERT text encoder |
--config FILE |
none | YAML extraction plan (extractions: list of {model, mpp?, patch_px?}); supersedes -m/--mpp/--patch-size. The same model may repeat (one grid each - the MPP-ablation case) |
-s, --slide-encoder |
none | slide-level encoder(s) run after patch embedding (e.g. titan); reads patch features from the store, no WSI re-read. Repeatable. |
--qc |
none | per-patch QC producer(s) writing grids/<key>/qc/<tool>/ (e.g. grandqc); needs the producer's extra (raw2features[grandqc]) |
--qc-stain-norm |
off | normalize the QC input first (macenko|reinhard|vahadane; vahadane needs raw2features[stain]) - for a stain outside the QC model's domain |
--device |
auto |
auto (best of cuda→mps→cpu) | cuda | mps | cpu |
--devices |
= --device |
opt-in in-process multi-GPU, e.g. cuda:0,cuda:1; shards this slide's patches across them and gathers in coord order (output identical to one device). Patch-parallel multi-device execution is not supported with multiplex strategies or native multiplex models; see In-process multi-GPU in §6. |
--batch-size |
256 |
patches per forward pass (lower it on small GPUs) |
--amp |
auto |
precision: auto (each model's card precision) | fp32 | bf16 | fp16 |
--snap-to-level |
off | read a pyramid level natively (no resample); MPP = that level |
--mpp-tolerance |
0.001 |
read a level natively if within this fraction of target MPP |
--allow-upsample |
off | permit upsampling when the target is finer than level 0 |
--emit-geojson |
off | also write <id>.patches.geojson (QuPath patch polygons) |
--emit-thumbnail |
off | also write a thumbnail + QC overlay (see §8) |
--receipts-dir DIR |
none | enable receipts + skip-if-complete (see §7) |
--hf-token |
$HF_TOKEN |
token for gated models; cached login or the environment is preferred, and command provenance is redacted |
See raw2features models for available extractors and which are gated.
The reader receives the complete remote URI in memory so query-authenticated OME-Zarr
stores can be opened. The persisted source.uri, receipts, output identity, and
captured CLI remove userinfo and known AWS, Google, Azure, and generic authentication
parameters, while retaining semantic selectors such as object versions, generations,
series, and fragments. Credential rotation therefore does not rename the output.
Query credentials must be valid for the Zarr prefix and all child metadata/chunk objects.
A presigned S3 or GCS URL for one object usually cannot authorize the whole hierarchy;
prefer native s3:// or gs:// credential-provider configuration in that case.
For one slide, in order:
- (skip check) If
--receipts-diris set and the source-bound target is already complete for this exact config, the run returnsskippedimmediately (see §7). - Read metadata via the
omezarrreader (NGFF version, MPP, pyramid). - Build the grid at the exact target MPP (below).
- Segment the tissue (unless
--no-seg) on a low-res level and keep only the grid cells whose tissue fraction ≥--tissue-threshold. - (optional) Render the thumbnail/overlay if
--emit-thumbnail. - Embed, decode-once: for each batch, read the patches once, then run every requested model on that shared batch.
- Write the
*.embeddings.zarrstore + provenance, optional sidecars. - (optional) Write a
completereceipt if--receipts-diris set.
Foundation-model embeddings are scale-sensitive, so every patch is sampled at the exact MPP you ask for - not "close enough". The patcher:
- picks the nearest finer-or-equal pyramid level (native MPP ≤ target), so it only ever downsamples - it never invents detail by upscaling;
- reads
read_px = round(patch_px × resample)pixels at that level (resample = target_mpp / level_mpp ≥ 1); - downsamples that read to exactly
patch_px(the stored patch ispatch_pxat the target MPP, independent of any model's input size - each model then resizes the patch to its input size internally).
If a level's native MPP is within --mpp-tolerance (default 0.1%) of the target,
it is read natively with no resize (the common case for pyramids built at the
target resolution), and the recorded achieved_mpp is that level's native MPP.
Otherwise the patch is resampled and achieved_mpp == target. --snap-to-level
forces a native level read (faster, MPP = that level); --allow-upsample permits
going finer than level 0. raw2features info prints the exact plan.
OME-NGFF float arrays do not necessarily declare whether RGB values use a normalized
[0, 1] range or a byte-like [0, 255] range. The OME-Zarr reader therefore chooses
one convention when the slide is opened and reuses it for every patch and pyramid level;
it never rescales one patch differently because that patch happens to be locally dark.
The decision uses the array fill value and a deterministic 3×3 lattice of level-0
chunks (up to the first three channels): any finite observed value above 1 selects the
byte-like convention, otherwise the slide is treated as normalized. This bounded sample
avoids scanning an entire WSI at open. read_region_channels remains native-dtype and
unscaled for multiplex models.
--multiplex-strategy channelwise lets a registered RGB patch encoder process an
OME-Zarr image with a c axis and positionally matching channel names:
raw2features embed multiplex_slide.ome.zarr multiplex_out \
-m uni --multiplex-strategy channelwise \
--marker CD3 --marker CK --marker DAPI \
--multiplex-aggregation mean -s meanEach selected channel is processed independently. raw2features resolves a normalization
for that channel, repeats the normalized single-channel patch across RGB, and applies the
encoder's registered preprocessing and pooling (for example CLS for UNI or
CLS+mean-patch for Virchow2). The marker vectors are combined with mean (default) or
ordered concat; concat requires an explicit marker order. Without an explicit
selection, every physical channel must have a unique name and all channels are used in
source order. If metadata contains unnamed positions, supply a complete
--channel-names-file or explicitly select the named markers to exclude those positions.
Selected names must resolve unambiguously. No channel name has implicit inclusion or
exclusion semantics for embedding. Nuclear marker names are resolved separately when
segmentation is enabled.
--multiplex-normalization percentile is the default. It records each selected channel's
1st and 99th percentiles from the first image-pyramid level whose longest side is at most
2048 pixels. Set alternative bounds with --multiplex-percentile-low and
--multiplex-percentile-high. This is a bounded-memory, reproducible raw2features rule
rather than a platform-specific calibration. The level limit can be changed with
--multiplex-normalization-max-side-px; a larger value may choose a finer level and use
more memory. If no pyramid level fits the cap, raw2features asks for a coarser level or a
higher cap instead of silently scanning level 0. --multiplex-normalization dtype
instead maps the source dtype range to [0,1]; floating-point sources must already be
finite in [0,1]. Percentiles describe the selected pyramid level, so spatial
downsampling can smooth extrema relative to level 0; the exact level and measured values
are fingerprinted. The selected markers and physical indices, order,
normalization values and source level, RGB conversion, base-model fingerprint and
pooling, aggregation, and output dimension are stored in the grid's model/panel metadata
and output fingerprint.
The RGB vision-encoder baselines evaluated in the public KRONOS paper provide structural inspiration. This strategy adapts ordinary RGB encoders rather than implementing the marker-aware KRONOS architecture. Every marker is normalized independently, repeated to RGB, embedded independently, then aggregated with a fully recorded deterministic recipe.
Channel identity normally comes from omero.channels. If those labels are absent or
incomplete, provide a complete positional panel without rewriting the source:
raw2features embed multiplex_slide.ome.zarr multiplex_out \
-m uni --multiplex-strategy channelwise \
--channel-names-file channel_names.tsv \
--marker CD3 --marker CK --marker DAPIThe UTF-8 .txt, .csv, or .tsv file contains exactly one unique name for every
physical C-axis position, in order. A conventional one-column header is optional. Any
existing non-empty OME labels must agree; conflicts and channel-count mismatches fail
before model loading or output mutation. The file defines physical channel identity,
while repeated --marker options select and order a subset. Only the resolved ordered
panel and ordered marker request affect request identity; the file path and serialization
do not. Store provenance also records whether effective labels came from OME metadata or
a channel-names file and, when they differ, preserves the original OME labels. As with
other non-secret arguments, the recorded CLI invocation can include the file path. One
file applies to every slide in an embed-many invocation, so heterogeneous panels should
use separate runs.
Multiplex runs use nuclear-channel tissue masking when segmentation is enabled. They
require one unambiguous nuclear channel or a recognized same-stain group, such as
DNA1/DNA2 or repeated DAPI channels. A multi-channel group is averaged in float32 before
thresholding. This differs from v0.1's first-match behaviour and is bound into a new grid
identity rather than reusing the old mask.
--no-seg is fully supported and tiles the complete image, which is useful for
segmentation-free analyses or panels without a suitable nuclear channel; it can also
retain more background and require more compute.
Patch geometry remains configurable. If an extracted patch does not match the encoder's registered input size, raw2features resizes it before model preprocessing. Smaller, cell-scale fields are therefore mechanically supported, but resizing does not add image detail and the changed physical field of view should be validated for the intended task.
Strategy-derived outputs use a cohort-stable effective key under features/.
Model-agnostic slide poolers (mean, max, meanmax) accept them, while learned slide
encoders tied to another patch model are rejected. The first implementation processes one
device per slide.
The strategy is a plugin (raw2features.multiplex_strategies), with separate
panel/config preparation and slide-specific binding phases. Future versions may add
marker-to-R/G/B mappings or learned channel adapters if there is demand; either can use
the same panel-binding and fingerprint contract. Third-party strategies can accept a
JSON object through --multiplex-params; the built-in channelwise strategy uses its
explicit options and does not accept additional strategy parameters.
Native multiplex encoders consume selected marker channels together rather than converting each channel to RGB. Run KRONOS2 after installing its optional stack and accepting the gated institutional-access terms:
pip install "raw2features[all,kronos2]"
hf auth login
raw2features embed multiplex_slide.ome.zarr multiplex_out \
-m kronos2 --marker CD3 --marker CD8 --marker DAPIkronos2 is a marker-aware ViT-B/16 that returns a 768-dimensional CLS vector in fp32.
Its default patch is 256 pixels. A custom --patch-size is accepted only when it is a
multiple of 16. Marker names are matched exactly after the model's
separator-insensitive normalization; KRONOSv1 aliases and compound-name fallbacks are
not applied. --marker selects physical channels and fixes their order before
completion checks. The full source panel, selected indices, model-resolved names, and
order remain in provenance and the output fingerprint.
The released vocabulary contains DAPI and DRAQ5 but not Hoechst or DNA1/DNA2 labels.
Those common CODEX nuclear stains are not silently aliased. Exclude one from the
encoder with an explicit --marker list while still allowing nuclear segmentation to
use it, or register it through --kronos2-additional-markers with dataset-specific
statistics. When DAPI is absent and DRAQ5 is selected, raw2features follows the upstream
example by passing DRAQ5 as preferred_dapi, which applies the model's DAPI
normalization statistics; the recorded policy is not configurable in v0.2.1.
Upstream recommends batches of at least eight for reproducibility. raw2features uses
fixed eight-sample forward microbatches and pads a shorter final microbatch before
discarding the padded outputs. This runtime contract is recorded. The global
--batch-size may be left unchanged; it does not override this internal cap. Bitwise
equality across different hardware is not promised. On x86-64 Linux with CPython 3.11
or 3.12, the [kronos2] extra installs the authors' pinned xFormers reference path;
this path is validated with Python 3.12. Python 3.13 and other platforms use the
upstream PyTorch attention fallback, which is slower and may use more memory. The pinned
Torch release does not support Python 3.14. --compile is not supported for KRONOS2 in
v0.2.1; use the validated eager path.
The released KRONOS2 repository includes 288 usable marker-metadata entries, 268 marked as pretraining markers, and stores their text vectors in the checkpoint. Normal inference therefore never downloads BioLinkBERT. To register a marker outside that released metadata, supply a complete CSV:
raw2features embed multiplex_slide.ome.zarr multiplex_out \
-m kronos2 --marker NovelMarker --marker DAPI \
--kronos2-additional-markers additional_markers.csvEvery CSV row requires marker_name, marker_full_name, compartment, family,
family_desc, mean, and std. The mean and standard deviation must be finite, with a
positive standard deviation. compartment and family must exactly reuse categories
from KRONOS2's pinned marker metadata, and a novel name must not collide with that
vocabulary under the model's separator-insensitive matching.
Statistics are supplied in the scaled intensity domain consumed by KRONOS2: uint8 values divided by 255, wider unsigned integers divided by 65535, or floating values divided by 400. Signed-integer sources are rejected. raw2features does not estimate these dataset statistics. The option loads the separately pinned BioLinkBERT model, about 1.33 GB, only for novel-marker text vectors. The canonical CSV content, intensity statistics, BioLinkBERT revision, and text-encoding recipe are included in the model fingerprint.
Run native multiplex and ordinary RGB encoders in separate commands so each receives the
correct source-channel representation. An additive second command can still append its
features to an existing matching grid. Native multiplex patch execution uses one device
per slide; embed-many --devices can distribute different slides across devices.
The default otsu segmenter runs Otsu-on-saturation + morphology on a low-res
level (~8 µm/px) to produce a tissue mask, and the patcher keeps a grid cell only
if its tissue fraction ≥ --tissue-threshold. --no-seg tiles the whole slide
(and omits the mask array). Other segmenters (incl. the opt-in deep grandqc)
are pluggable via the segmenters seam - see
SEGMENTATION.md for the methods and the GrandQC QC layer.
With several -m flags (-f is an alias), each patch is read and decoded once
and reused across every model in the batch - the slow part (IO + decode) is not
repeated per model.
Each model applies its own preprocessing (mean/std/input size, sourced from
its model card), and writes into its own features/<model> array.
By default the pipeline runs on the single --device (CPU or one GPU) and is fully
portable. --devices is an opt-in runtime flag that spreads work across several
devices in one process; it never changes the embeddings (it is excluded from the
config hash), so a store built with it is identical to a single-device one and
resumes interchangeably.
Multi-device patch-parallel embed --devices execution does not currently support
multiplex strategies or native multiplex models. A multiplex run uses one device per
slide. embed-many --devices may still distribute distinct slides across devices.
embed(one slide) - patch-parallel, latency mode.embed … --devices cuda:0,cuda:1shards the slide's patches into contiguous blocks, embeds each block on its own device (one model copy + reader per device), and gathers the features back in exact coord order - so the output is byte-identical to a one-GPU run, just faster for a single large slide.embed-many(a shard of slides) - slide-parallel, throughput mode.embed-many … --devices cuda:0,cuda:1runs one warm worker per device, each pulling slides off a shared queue; every slide is embedded entirely on one device, so its output equals a one-GPU run. This is the path for a multi-GPU workstation without a scheduler (a SLURM array already parallelises one GPU per task - see §9).benchmark … --devices …measures the patch-parallel single-slide path (wall-time / patches/s stay accurate; the per-stage breakdown is single-device only).
Equivalence is exact on CPU and across distinct GPUs (each patch is processed on one
device exactly as in the single-device path). Listing the same GPU twice
(cuda:0,cuda:0) is only a single-GPU test convenience and may differ by ~1 ULP due
to shared-stream float reordering.
This section explains the already-embedded behaviour: how re-runs detect existing work and what they do with it. Worth reading before a cohort run.
embed inspects any existing <slide>.embeddings.zarr before doing work, so
re-runs are additive and idempotent - no --receipts-dir required:
- A requested model already present and output-validated against its current model fingerprint is skipped.
- A missing model is embedded and appended in place:
features/<model>is added next to the existing arrays when it shares that grid's geometry. Existing models andcoordsare left untouched. - A requested geometry not already in the store is added as another
grids/<key>/group. A difference in--mpp,--patch-size, segmentation, or another geometry field is not an error and does not wipe existing grids or mix incompatible rows in one grid. - If all requested models and auxiliary outputs are already present, no patch
features are recomputed (a model-only run returns
status: skipped).
Use --force only when you intentionally want to rebuild the whole store from
scratch.
raw2features embed slide.zarr out/ -m uni # writes features/uni
raw2features embed slide.zarr out/ -m virchow2 # ADDS features/virchow2; uni untouched
raw2features embed slide.zarr out/ -m uni -m virchow2 # both present -> nothing to do
raw2features embed slide.zarr out/ -m conch_v1_5 # ADDS its 0.5/512 grid; other grids untouchedThe shared extraction/storage grid is identified by a grid_hash, recorded in
the header. Model-specific settings such as resolved AMP live in each output
fingerprint instead, so changing them replaces that model in the same grid. "Present"
is strict: a model counts only if features/<model> has the
current expected dimension and a matching output fingerprint in both the array and
grid header, is fully finite, and has len(coords) == n_patches. For legacy arrays
without a post-write fingerprint commit marker, an all-zero final row is additionally
treated as an unwritten tail. The fingerprint covers the effective weights,
preprocessing, pooling, resolved AMP, and loader construction. A truncated, damaged,
legacy-unfingerprinted, or stale-contract array is re-embedded; unrelated model arrays
and coordinates remain untouched. A derived slide/<model> fingerprint includes its
patch input fingerprint, so stale slide vectors are recomputed too.
--receipts-dir DIR adds a cheap short-circuit on top of the store inspection
above. After confirming that any existing target store belongs to the requested
source, embed checks for a validated complete receipt for this exact
configuration (including the model set), source, and output target. If found it
returns skipped; otherwise it processes and writes a complete receipt at the
end. This is what the SLURM array uses to skip finished slides cheaply.
Completeness is validated against the actual output store, never just the
receipt file.
The receipt's config hash covers the content-affecting settings and the model set:
reader, models (order-independent), segmenter / --no-seg, --mpp, --patch-size,
--source-mpp (when set), --step, --tissue-threshold, --features-dtype,
--stain-norm, --snap-to-level, --mpp-tolerance, --allow-upsample, --amp
For multiplex requests, the hash also covers the ordered physical marker selection and
resolved positional channel names. Strategy-derived requests additionally cover their
normalization recipe, pyramid-level limit, aggregation, and other strategy parameters.
Native model parameters, including KRONOS2 additional-marker metadata, are covered when
present. The resolved physical nuclear-channel indices enter grid identity when masking
is enabled. The --channel-names-file path and serialization are not hashed; their
parsed effective names are.
Runtime-only knobs do not change the hash:
--device, --devices, --batch-size, --read-workers, --read-block,
--compile, --emit-geojson, --emit-thumbnail, --thumbnail-mpp, --max-px,
--slide-encoder, --qc, --qc-stain-norm, --qc-artifact-mpp,
--output-zarr-format, --force.
--force is the exception for control flow: it deliberately bypasses skipping and
rebuilds the target without changing the content identity.
raw2features verify <slide> --receipts-dir DIR --out-dir OUT <same content flags>
runs exactly the same completeness check and exits 0 (complete) or 1
(incomplete). It is how the SLURM array skips finished slides. A receipt is trusted
only when its slide ID, credential-free source, requested output target, and the
actual store's root/grid source provenance all agree. Pass --out-dir and the same
content-affecting flags you pass to embed, or the check is not target-aware (or
the config hash will not match).
For library callers, verify also recognizes the direct single-grid receipt written by
the public run_slide primitive. Multi-grid requests use embed_slide and its whole-run
receipt.
raw2features verify slide.zarr --receipts-dir receipts/ --out-dir embeddings/ -m uni -m resnet50 --mpp 1.0 --quietPass --force to ignore an existing store and rebuild it from scratch (all
requested models recomputed, the store overwritten). It also bypasses a valid
receipt's fast path. Deleting the output store (and any receipt) has the same effect.
An explicit --emit-thumbnail, --emit-geojson, --qc, or -s/--slide-encoder
request is checked against the matching grid even when its patch features are already
complete. Missing outputs are produced without re-embedding patches; existing valid
outputs are left alone. These runtime products do not change the patch-grid identity or
receipt content hash. The standalone thumbnail
and slide-embed commands remain convenient when you only want to add those products.
In a multi-grid invocation, a slide encoder runs on the grid containing its required
patch encoder (for example, TITAN uses the conch_v1_5 grid). Model-agnostic pooling
encoders run on each requested grid. Standalone slide-embed can infer an unambiguous
grid from the required patch model; otherwise pass --grid (and --patch-model when
needed).
Two ways to get a thumbnail; both are optional and off by default so they never slow a large feature-extraction run.
Standalone (run before or after the heavy embed run; one cheap coarse read):
raw2features thumbnail slide.zarr out/ # plain preview
raw2features thumbnail slide.zarr out/ --overlay # + tissue mask + kept-patch gridInline with embeddings (piggybacks on the already-open + segmented slide):
raw2features embed slide.zarr out/ -m uni --emit-thumbnailA thumbnail is an RGB overview read from a coarse pyramid level (never the full-resolution level 0). By default it renders at the segmentation MPP (8 µm/px), chosen by the same nearest-level rule the segmenter uses. That single choice gives two things at once:
- the thumbnail, the tissue mask, and the patch grid share a pixel grid, so the
--overlayneeds no resampling - it is a faithful QC view ("did segmentation + patching do the right thing on this slide?"); - it is also a sensible size (≈3 k px wide for a 0.25 µm/px slide).
Override the resolution with --thumbnail-mpp <µm/px> or --max-px <N> (cap the
longest side; overrides --thumbnail-mpp). Pyramid-level indices are never
exposed - they aren't portable across slides, and the whole tool derives the
level from MPP.
For the standalone --overlay, the segmenter + patcher are re-run (cheaply) so
the overlay reflects the grid params you pass (--mpp, --patch-size, --step,
--tissue-threshold, --segmenter).
<slide_id>.thumbnail.png # plain overview
<slide_id>.thumbnail.overlay.png # tissue tint + kept-patch outlines (with --overlay / --emit-thumbnail)
Inline (--emit-thumbnail) also records the thumbnail metadata in the store's
.zattrs (raw2features.thumbnail).
slurm/ ships a one-slide-per-task array, a per-batch ("shard") array, and a pre-flight
checker - see slurm/README.md for a plain-language overview and
which script to use. Each task is idempotent: it verifys the slide first and skips if
already complete, so re-submitting the same array safely resumes only the missing/failed
slides.
Before embed-many shards or loads models, it requires every resolved cohort input to
derive a unique output ID. Ordinary local IDs remain basename-based for v0.1
compatibility, so same-named slides from different directories must be renamed or run
as separate commands with different output directories. Repeated identical manifest
rows are also rejected; a per-output lock across independent invocations is not implied.
cd raw2features && uv sync --extra zarr --extra image --extra torch --extra models
# Add any model-specific extras / pinned git packages from docs/MODELS.md.
export SLIDE_DIR=/path/to/raw # dir of *.zarr stores (top-level only)
export OUT_DIR=/path/to/embeddings
export MODELS="uni resnet50" # space-separated; each becomes a -m flag
export HF_TOKEN=hf_... # only if MODELS includes a gated model
bash slurm/preflight.sh # validates venv/arch, models/HF, slide count
mkdir -p logs # SLURM opens --output here before the job runs
shopt -s nullglob
SLIDES=("$SLIDE_DIR"/*.zarr)
shopt -u nullglob
N=${#SLIDES[@]}
(( N > 0 )) || { echo "no *.zarr slides found" >&2; exit 1; }
sbatch --array=0-$((N-1))%64 --export=ALL,SLIDE_DIR,OUT_DIR,MODELS,HF_TOKEN slurm/embed_array.sbatchNotes:
- Set
--partitioninembed_array.sbatchto your cluster's GPU partition (the template ships agpuplaceholder). - Lower
BATCH_SIZEon smaller GPUs (UNI2-h is a ViT-H and is memory-hungry). - The slide list is a top-level glob (
$SLIDE_DIR/*.zarr) - never a recursivefindinto the stores - and is sorted byte-order (LC_COLLATE=C) so task i maps to the same slide on every node. - The pre-flight catches the most common cohort-killer: a venv built on the wrong architecture (it imports Torch to check).
- No scheduler? On a multi-GPU workstation,
embed-many … --devices cuda:0,cuda:1,…is the equivalent of the array: one warm worker per GPU, slides distributed across them, per-slide receipts for idempotent resume (see In-process multi-GPU in §6).
<slide_id>.embeddings.zarr/
├── .zattrs # root header: source, provenance + a grids index
└── grids/<key>/ # one per geometry (mpp, patch_px) - usually just one
├── .zattrs # this grid's full header (below)
├── coords/ # (N,2) int32 level-0 (x,y) top-left of each patch
├── grid_index/ # (N,2) int32 (row,col) on the tiling grid
├── mask/ # (n_rows,n_cols) uint8 fraction of each cell that is tissue; /255 for [0,1] (omitted with --no-seg)
├── features/<model>/ # (N,dim) float16 one array per extractor
└── slide/<output-key>/ # (1,dim) float32 optional slide-level vectors
<slide_id>.patches.geojson # optional (--emit-geojson): per-patch polygons for QuPath
<slide_id>.thumbnail.png # optional (--emit-thumbnail / thumbnail cmd)
<slide_id>.thumbnail.overlay.png # optional QC overlay
<slide_id>.<key>.patches.geojson # corresponding sidecar for each additional grid
<slide_id>.<key>.thumbnail.overlay.png # corresponding overlay for each additional grid
A store nests its patch sets under grids/<key>/ (one child per extraction geometry -
mpp0.5_px224, etc.); a one-geometry run is just a single grid. Open one grid with the
reader helper (raw2features.core.store.open_grid(store) returns the sole grid, or pass a
key), and export/info default to the sole grid (a multi-grid store wants --grid). The
first-created grid keeps the backward-compatible unsuffixed GeoJSON/overlay names; later
grids use their key as shown above. The plain thumbnail is shared by the slide.
Spatial-provenance contract (within a grid). Row i refers to the same patch across
every array of that grid: coords[i], grid_index[i], and features/<model>[i] are
aligned 1:1 in identical order. Combined with patching.level0_patch (the patch's level-0
pixel extent), any embedding is invertible to the exact slide region
[x, y, x+level0_patch, y+level0_patch] at level 0.
Each grid's .zattrs["raw2features"] records the full, self-describing header:
schema_version, provenance (version, CLI, git SHA, host, GPU), thumbnail (if emitted),
source (uri, ngff_version, reader, mpp_level0, dimensions/downsamples, source axes,
per-axis physical scale, source translation/origin, and for multiplex sources the
effective positional channel names, their source, and any differing original OME labels),
patching (target/achieved MPP, patch_px, read_level, read_px, resample, level0_patch,
level0_step, n_patches, grid_shape, coords_convention: level0_xy), segmentation
(including the resolved nuclear channel binding when used), top-level panel, and
per-model models.<name> (source, embedding_dim, input_size, pooling, mean, std,
transform_source_url, license, gated, weights pins, output fingerprint, and any
multiplex contract). The root .zattrs carries the shared source/provenance plus a
grids index (key → {mpp, patch_px, n_patches, models, grid_hash}) for discovery without
opening each grid.
Read it back:
from raw2features.core.store import open_grid
# Omit the key only when the store contains exactly one grid.
g = open_grid("out/<slide_id>.embeddings.zarr", key="mpp0.5_px224")
print(g["features/uni"].shape) # (N, 1024)
print(dict(g.attrs)["raw2features"]["patching"]["achieved_mpp"])embed --emit-thumbnail produced no thumbnail. A missing thumbnail should be
produced even when the patch arrays are complete, while an existing valid thumbnail is
left unchanged. Check the command's error and output path; in a multi-grid store,
non-primary overlays have the grid key in their filename. The standalone
raw2features thumbnail … --overlay command is also available when only a preview is
needed.
What happens when I add a model? A model with the same extraction geometry is appended to that grid. A model with a different geometry gets a new grid. Existing valid arrays are preserved; only a missing, damaged, legacy-unfingerprinted, or stale-fingerprint model output is recomputed.
verify always says incomplete. It must be given the same content-affecting
flags as the embed that produced the store (same -m models, --mpp,
--patch-size, etc.) so the config hash matches. Pass the same output directory as
--out-dir so the receipt is also bound to the intended target store.
import torch / model load fails on the cluster. The venv was built for a
different CPU architecture. Re-run uv sync on the target machine.
Gated model 401 / access error. You need both a token (--hf-token /
HF_TOKEN / cached login) and access granted on the model's HuggingFace page.
Open models (resnet50, dinov2) need neither. Prefer HF_TOKEN or hf auth login
so the token never appears in shell history or a process listing.
Embeddings are float16. Default storage dtype (inode/byte-light). Use
--features-dtype float32 if you need full precision on disk.
achieved_mpp isn't exactly my --mpp. A level within --mpp-tolerance
(0.1%) of the target is read natively (no resize), so achieved_mpp is that
level's native MPP. Tighten --mpp-tolerance 0 to force exact resampling, or use
--snap-to-level to read a level natively on purpose.