Skip to content

A vc_zarr_download_region command to download a bbox of a volume to VC3D cache. - #1707

Open
spelufo wants to merge 4 commits into
ScrollPrize:mainfrom
spelufo:vc-zarr-download-region-command
Open

spelufo wants to merge 4 commits into
ScrollPrize:mainfrom
spelufo:vc-zarr-download-region-command

Conversation

@spelufo

@spelufo spelufo commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

In one sentence:

Download a region of a volume in full to VC3D's cache. For offline viewing, for example.

Proof:

 $ ./build-macos/bin/vc_zarr_download_region --project ~/.VC3D/remote_cache/open_data/projects/PHerc1203.volpkg.json --url "https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc1203/volumes/20250820131727-9.362um-1.2m-113keV-masked.zarr" --xmin 496 --ymin 900 --zmin 11000 --xmax 6009 --ymax 6241 --zmax 15000 -l 0
Opening https://vesuvius-challenge-open-data.s3.us-east-1.amazonaws.com/PHerc1203/volumes/20250820131727-9.362um-1.2m-113keV-masked.zarr
Cache: "/Users/spelufo/.VC3D/remote_cache/open_data/volumes/PHerc1203/20250820131727-9.362um-1.2m-113keV-masked.zarr-adf63bbdf658dd8f"
Level 0 shape [z,y,x]=[18977,6844,6844] chunk=[128,128,128]
Region inclusive [z,y,x]=[11000:15000,900:6241,496:6009] covers 60984 chunks
Done. requested=60984 cached=30970 missing=30014 errors=0
  • I personally verified that the example and proof above were produced by this PR on the stated data.

Details

Since the python vesuvius module returns black without error/warning when pointed to a zarr volume inside VC's cache, it is useful to have a way to make sure a ROI in the cache has data.

Maybe in the future the vesuvius python package can share the cache wtih VC3D and download on demand, but this is useful as a standalone command anyways.

@vercel

vercel Bot commented Sep 5, 2026

Copy link
Copy Markdown

@spelufo is attempting to deploy a commit to the scroll Team on Vercel.

A member of the Team first needs to authorize it.

@spelufo
spelufo force-pushed the vc-zarr-download-region-command branch from a7cd499 to a3ea4c6 Compare September 9, 2026 05:14
@hendrikschilling hendrikschilling self-assigned this Sep 18, 2026
@hendrikschilling

Copy link
Copy Markdown
Contributor
  1. [P1] It does not compile against current main.
     The command calls the removed remoteVolumeCacheRootForEntry() helper and the old five-argument Volume::NewFromUrl(). Linux, macOS and Windows CI all fail on these calls. Adapt it to  
     the current process-wide cache-root API.
     vc_zarr_download_region.cpp:345 (https://github.com/ScrollPrize/villa/blob/3f37484dac05986c31b60952c371c95c151de14d/volume-cartographer/apps/src/vc_zarr_download_region.cpp#L345)     

  2. [P2] Bounds conversion can omit required chunks.
     scaleBounds() derives scale from array dimensions, but pyramid dimensions are rounded. I reproduced this using the PR’s PHerc1203 metadata: at level 3, base Z=15355 maps to 1919.375, 
     inside chunk 14. The command instead selects Z=1920, downloading only chunk 15. It can therefore report success while leaving the requested region incomplete. Use the cache’s existing
     levelTransform() rather than dimension ratios.
     vc_zarr_download_region.cpp:148 (https://github.com/ScrollPrize/villa/blob/3f37484dac05986c31b60952c371c95c151de14d/volume-cartographer/apps/src/vc_zarr_download_region.cpp#L148)     

  3. [P2] It ignores VC3D’s selected cache encoding.
     The command initializes the cache with the default source-mirror encoding, even when VC3D uses Delta3D. The existing cache implementation deletes an incompatible per-volume cache when
     switching formats. Consequently, running this tool can discard previously cached regions; subsequently opening VC3D can discard this download again. With VC3D already holding the     
     incompatible cache open, persistence is refused instead. Initialize the service using the same encoding setting, preferably through shared configuration code.
     vc_zarr_download_region.cpp:422 (https://github.com/ScrollPrize/villa/blob/3f37484dac05986c31b60952c371c95c151de14d/volume-cartographer/apps/src/vc_zarr_download_region.cpp#L422)     

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants