feat(snapshot): configure filesystem state limits by 0xpolarzero · Pull Request #1751 · superradcompany/microsandbox

0xpolarzero · 2026-10-02T16:00:41Z

Full snapshots fail once a sandbox's virtio-fs state outgrows its fixed limit. That happens after the guest reads a few thousand files from a host-directory mount (#1750). This PR adds one setting for that budget and carries it everywhere a snapshot is captured or read:

{ "snapshots": { "max_filesystem_state_mib": 64 } }

The default stays 4 MiB, the current backend limit. The enclosing device-state limits come from msb_krun's DeviceStateLimits / DeviceStateCodec (superradcompany/libkrun#151).

msb_krun is pinned to the merged libkrun #151 revision until that API is released. The pin must be replaced with the registry version before release.

What changed

Config: a new snapshots.max_filesystem_state_mib setting (1–4095), validated with the other config layers.
Sandbox process: gets the budget through the launch config.
- The field is sent only when it is not the default, so ordinary launches are unchanged.
- A non-default value requires the runtime's new capability. Older runtimes are refused before launch.
Capture and restore: the budget is applied to every filesystem backend and to the msb_krun device-state codec. The capture paths include live forks.
Checkpoint admission: verification, export (full and incremental), import, restore and sandbox creation from a snapshot all use the configured budget. They decode each virtio-fs state and check the filesystem state itself, so a snapshot that restore would refuse is refused at import too.
Errors: an over-budget state names snapshots.max_filesystem_state_mib.
Docs: docs/configuration.mdx and COMPATIBILITY.md.
- A running sandbox keeps the budget it started with.
- Every machine that reads a snapshot needs a large enough budget.
- Older releases ignore the key and keep 4 MiB.

How it was tested

Unit tests cover:
- configured budgets above 4 MiB, and malformed lengths staying bounded;
- config layer precedence and invalid values;
- launch compatibility, including the capability check and the older launch contract;
- an archive exported with a larger budget being refused by a receiving host at the default, including filesystem state just 1 byte over it;
- the setting name appearing in over-budget errors.
cargo test for the image, runtime, filesystem, cli and sdk crates, cargo fmt --check, and clippy as in CI.
On macOS, a sandbox reads 50,000 files from a read-only bind mount:
- At the default budget, the full snapshot fails with … filesystem state exceeds the 4 MiB budget; raise snapshots.max_filesystem_state_mib …, and the sandbox keeps running.
- The same sandbox still fails after the config is raised, because it keeps its start budget.
- A fresh sandbox at 64 MiB snapshots, restores, forks, verifies, and exports/imports into a second home at 64 MiB.
- Importing into a home at the default is refused, naming the setting, and installs nothing.
- A 500-file snapshot at the default works as before.
- Values 0 and 5000 are rejected; 1, 4 and 4095 are accepted.

Confidence Score: 5/5

No new blocking issue was identified; the PR appears safe to merge on the reviewed code.

What we checked:

Rejected DNS answers leave old addresses usable: The cache checks capacity before adding an answer. If it does not fit, the older bindings remain.

_{Reviews (13) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..."}

Closure admission capped every device state at 1 MiB, while restore already accepted 8 MiB for virtio-fs. A full checkpoint of a sandbox with enough retained host-directory inodes was therefore rejected when created, verified, exported or restored. DeviceStateRef::max_state_bytes is now the single limit.

Export and import a full checkpoint whose virtio-fs device state is 2 MiB, and assert the imported object bytes match. The resolver test now checks the size limit error for oversized states. COMPATIBILITY.md records the per-device limits and that v0.7.0 through v0.7.6 readers reject larger virtio-fs objects.

…te-limit

toksdotdev · 2026-10-02T19:26:16Z

we could actually make all of this configurable through one global setting, something like:

{
  "snapshots": {
    "max_filesystem_state_mib": 64
  }
}

this would set the backend state budget per virtio-fs device, with the enclosing device limits derived from it to allow for headers and transport state. the default could stay at the current 4 MiB backend limit.

we’d need to carry that setting through capture, verification, export/import, restore, and live forks, including the limits in msb_krun, so raising it doesn’t just hit another hard-coded 4 MiB or 8 MiB cap further down.

…te-limit

0xpolarzero · 2026-10-02T20:44:04Z

we could actually make all of this configurable through one global setting, something like:
{
  "snapshots": {
    "max_filesystem_state_mib": 64
  }
}
this would set the backend state budget per virtio-fs device, with the enclosing device limits derived from it to allow for headers and transport state. the default could stay at the current 4 MiB backend limit.

we’d need to carry that setting through capture, verification, export/import, restore, and live forks, including the limits in msb_krun, so raising it doesn’t just hit another hard-coded 4 MiB or 8 MiB cap further down.

Alright! Marking draft again and making a PR in libkrun to implement this, so I can get back to this one once it's merged and released.

0xpolarzero · 2026-10-02T21:45:59Z

@toksdotdev PR is ready right there superradcompany/libkrun#151 :)

Add snapshots.max_filesystem_state_mib (default 4, 4..=4095) and carry it through the launch contract, runner, capture, verification, export/import, restore, and the filesystem backends in place of the fixed 8 MiB admission.

…t hint on export

toksdotdev · 2026-10-03T00:39:59Z

@0xpolarzero didn't intend to push. i've undone the commits. feel free to make the global config changes. 🫡

…ackend Filesystem backends take their state budget from their own configuration instead of a process-wide setting, and device-state admission comes from the libkrun state codec. The configurable budget range is 1 through 4095 MiB.

…te-limit # Conflicts: # sdk/rust/lib/backend/local/snapshot/archive/delta.rs

… budget Closure admission compared the whole device-state object with the budget plus the full envelope allowance, so a backend state up to about 1 MiB over the budget was admitted by verification, export and import and then refused at restore. Admission and restore now decode virtio-fs states and compare the backend state with the budget, naming snapshots.max_filesystem_state_mib.

toksdotdev · 2026-10-05T18:26:54Z

need to cut a new libkrun version. after which i'll merge this 🫡

…te-limit # Conflicts: # crates/filesystem/lib/backends/passthroughfs/windows/mobility.rs

…te-limit # Conflicts: # Cargo.toml

0xpolarzero added 2 commits October 2, 2026 17:43

0xpolarzero marked this pull request as ready for review October 2, 2026 16:33

0xpolarzero requested review from appcypher and toksdotdev as code owners October 2, 2026 16:33

0xpolarzero added 3 commits October 2, 2026 18:34

Merge remote-tracking branch 'origin/main' into fix/checkpoint-fs-sta…

1a83888

…te-limit

Merge remote-tracking branch 'origin/main' into fix/checkpoint-fs-sta…

d0625b4

…te-limit

Merge remote-tracking branch 'origin/main' into fix/checkpoint-fs-sta…

12c3c64

…te-limit

Merge remote-tracking branch 'origin/main' into fix/checkpoint-fs-sta…

37480c9

…te-limit

toksdotdev added the Review In-progress label Oct 2, 2026

0xpolarzero marked this pull request as draft October 2, 2026 20:31

0xpolarzero mentioned this pull request Oct 2, 2026

feat(devices): make the virtio-fs backend state limit configurable superradcompany/libkrun#151

Merged

0xpolarzero added 3 commits October 3, 2026 01:34

feat(snapshots): make the virtio-fs state budget configurable

9d14997

Add snapshots.max_filesystem_state_mib (default 4, 4..=4095) and carry it through the launch contract, runner, capture, verification, export/import, restore, and the filesystem backends in place of the fixed 8 MiB admission.

docs(snapshots): document the virtio-fs state budget setting

5d92373

refactor(snapshots): build the capture codec once and check the budge…

6890b19

…t hint on export

toksdotdev changed the title ~~fix(checkpoint): admit virtio-fs device state up to 8 MiB in closures~~ feat(snapshot): configure filesystem state limits Oct 3, 2026

greptile-apps Bot reviewed Oct 3, 2026

View reviewed changes

Comment thread sdk/rust/lib/backend/local/snapshot/create.rs Outdated

Comment thread Cargo.toml

toksdotdev force-pushed the fix/checkpoint-fs-state-limit branch from e81c429 to 37480c9 Compare October 3, 2026 00:38

toksdotdev changed the title ~~feat(snapshot): configure filesystem state limits~~ fix(checkpoint): admit virtio-fs device state up to 8 MiB in closures Oct 3, 2026

0xpolarzero added 3 commits October 3, 2026 02:41

build: pin libkrun to the shared state codec revision

dd93338

Merge remote-tracking branch 'origin/main' into fix/checkpoint-fs-sta…

0ca1648

…te-limit # Conflicts: # sdk/rust/lib/backend/local/snapshot/archive/delta.rs

0xpolarzero changed the title ~~fix(checkpoint): admit virtio-fs device state up to 8 MiB in closures~~ feat(snapshot): configure filesystem state limits Oct 3, 2026

greptile-apps Bot reviewed Oct 3, 2026

View reviewed changes

Comment thread crates/filesystem/lib/backends/mobility.rs Outdated

fix(filesystem): stop encoding backend state once it passes the budget

f84935f

toksdotdev reviewed Oct 5, 2026

View reviewed changes

Comment thread crates/image/lib/checkpoint/resolver.rs

toksdotdev marked this pull request as ready for review October 5, 2026 18:26

0xpolarzero added 2 commits October 5, 2026 20:35

Merge remote-tracking branch 'origin/main' into fix/checkpoint-fs-sta…

74e9134

…te-limit # Conflicts: # crates/filesystem/lib/backends/passthroughfs/windows/mobility.rs

Merge remote-tracking branch 'origin/main' into fix/checkpoint-fs-sta…

3fa33ee

…te-limit # Conflicts: # Cargo.toml

Navigation Menu

feat(snapshot): configure filesystem state limits - #1751

feat(snapshot): configure filesystem state limits#1751
0xpolarzero wants to merge 16 commits into
superradcompany:mainfrom
0xpolarzero:fix/checkpoint-fs-state-limit

0xpolarzero commented Oct 2, 2026 •

edited by greptile-apps Bot

Loading

Uh oh!

toksdotdev commented Oct 2, 2026 •

edited

Loading

Uh oh!

0xpolarzero commented Oct 2, 2026

Uh oh!

0xpolarzero commented Oct 2, 2026

Uh oh!

Uh oh!

Uh oh!

toksdotdev commented Oct 3, 2026

Uh oh!

Uh oh!

Uh oh!

toksdotdev commented Oct 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

0xpolarzero commented Oct 2, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

How it was tested

Confidence Score: 5/5

Uh oh!

toksdotdev commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xpolarzero commented Oct 2, 2026

Uh oh!

0xpolarzero commented Oct 2, 2026

Uh oh!

Uh oh!

Uh oh!

toksdotdev commented Oct 3, 2026

Uh oh!

Uh oh!

Uh oh!

toksdotdev commented Oct 5, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

0xpolarzero commented Oct 2, 2026 •

edited by greptile-apps Bot

Loading

toksdotdev commented Oct 2, 2026 •

edited

Loading