feat(akita)!: externalize trusted schedules and guide recursive offloading - #1844
Merged
Conversation
Stack the current Akita profiling and lean-wire papercuts on Jolt main, pin the quotient-free Akita tip, and replace compiled schedule tables with canonical runtime artifacts. Build exact grouped catalogs per setup, serialize them with verifier preprocessing, bind their digest into the transcript, and resolve proof shapes through the same trusted scheme instance.
Record the validated catalog-digest absorption in the Fiat-Shamir inventory and keep external artifact file I/O outside the profiling setup measurement while retaining catalog admission and key construction inside it.
…ation-c02ed7928 # Conflicts: # crates/jolt-akita/src/native_batching.rs # crates/jolt-verifier/tests/fs_inventory/absorb-sites.inventory
Contributor
📏 PR diffNote Actual code changed: 3,537 lines🟢 +860 added 🔴 −2,677 removed Tests, docs, fixtures, and helper scripts excluded.
Every changed line is classified once. |
Contributor
|
Warning This PR has more than 500 changed lines of actual code (3488 added+removed; tests, docs, fixtures, and helper scripts are excluded) and does not include a spec. Large features and architectural changes benefit from a spec-driven workflow. If this PR is a bug fix, refactor, or doesn't warrant a spec, feel free to ignore this message. |
# Conflicts: # crates/jolt-akita/src/adapters.rs # crates/jolt-akita/src/native_batching.rs # crates/jolt-akita/src/shape_guard.rs # crates/jolt-verifier/tests/fs_inventory/absorb-sites.inventory
quangvdao
changed the base branch from
codex/akita-lean-opening-wire
to
main
September 4, 2026 18:36
Five identical `fn schedule_artifacts` helpers (jolt-akita's scheme tests, legacy's packed tests, the jolt-verifier akita fixtures, and both modular prover akita suites) plus eight open-coded copies of the same expression all wrapped `from_default_directory()` in an `Arc` and aborted on failure. Replace them with `AkitaScheduleArtifacts::shared_from_default_directory()`, the shared-handle form every setup call already wanted. An inherent associated fn rather than a `test-utils` module: jolt-verifier has no jolt-akita dependency and reaches the type only through legacy's `pub use`, and a re-exported type carries its inherent methods where a free function would not. jolt-akita's own tests and benches would also have needed a self dev-dependency. This needs no manifest change anywhere, and `profile.rs` is a real production caller rather than test-only scaffolding. Loader call counts per file are unchanged, so the `Arc::ptr_eq` provenance guards on advice setup (packed.rs) still compare what they did before; the doc records that constraint. Extract `AkitaScheduleArtifacts::packaged_directory()` for the second rule, `<jolt-akita>/schedules`, previously spelled out in the `from_default_directory` fallback, the shape-guard test, and the catalog regeneration guards. Those two guards deliberately keep reading the packaged directory rather than the default loader, since they validate the committed artifacts and must not be redirected by `JOLT_AKITA_SCHEDULE_DIR`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Benchmark comparison (crates)Same-runner A/B: No differences above the ±10% noise threshold. All 62 benchmarks
|
The family filter compared `ends_with(selector)` against names like `jolt-fp128-dense-bounded`, so the documented `dense` argument matched nothing and the generator silently wrote no artifacts. Match the selector as an infix and fail loudly when it selects no family. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`invalid_setup` was declared as a `&dyn Display` closure in three functions, and that signature forced a wrapping closure at every one of its fifteen uses. Make it a free function beside `invalid_batch`, `akita_error`, and `commit_failed`, taking `impl ToString` like they do, so call sites pass the fn item directly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`emit::regen` was `pub(crate)` and `MAX_PROVISIONED_ROWS` was `pub` only for tests the registry removal deleted; every remaining use is inside the defining module. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both catalog guards repeated the reason they read `packaged_directory()` instead of the default loader; that reason is the accessor's own doc. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`from_default_directory` and `DIRECTORY_ENV` were `pub` with exactly one caller each, `shared_from_default_directory`. Deployments that must handle a missing catalog use `from_directory` with a path they own, so the fallible env-aware form has no external contract to keep. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
moodlezoup
approved these changes
Sep 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
With #1842 merged, this restacks Jolt's Akita integration on current
mainand updates it to the quotient-free, external-schedule Akita stack..akscatalogs;log_T = 21, then setup-offloaded schedules fromlog_T = 21upward;No planner policy, catalog bytes, catalog path, or proof-shape description is carried in the proof.
Stack and exact revisions
mainat626d9665119c7be7dd2b0773cd6bb0beb07c3a25626d9665119c7be7dd2b0773cd6bb0beb07c3a25(former head1efb96ce64079277f83fc9f6fefd236922e99417)8cb334b1987a29e19e013d4c31a44f93dbdda5cace27c705d92b2ce5327b5c590b6dd6fe86f8479fThe ancestry-only merge
8cb334b19makes the merged #1842 commit onmaina parent without changing the PR tree: both8cb334b19and the previous green head48643c013have treecf874a6f1e187d2d3fa551557a0d33b4719b0f40. The PR base is nowmain, so the merged #1841/#1842 work is excluded from this PR's review diff.Design
Trusted schedule ownership
The host loads and validates the base K16, K256, and dense catalogs explicitly. Preprocessing admits the exact reachable advice and committed-program profiles, canonicalizes the resulting catalog, and serializes those bytes in
AkitaVerifierSetup. Deserialization validates the artifact before recreating the scheme cache.The prover and verifier therefore derive their scheme instances from the same trusted artifact. There is no process-global schedule registry and no verifier-time replanning or filesystem discovery.
Guided adaptation
For grouped roots, the selected scalar trace row remains authoritative. Guided planning freezes its:
Only the new advice/bytecode/program-image precommit profiles and suffix quantities affected by them are replanned. Missing or unsupported reachable rows fail preprocessing rather than failing later during proving.
This intentionally optimizes fast adaptation to relatively small precommitted objects; it does not claim the globally optimal schedule that a full fresh solve might find.
Decode and transcript boundary
The verifier resolves the 32-byte selection against the setup-owned catalog before deriving allocation budgets or decoding the backend proof. The shape guard and final batched verification use the same scheme instance. The validated catalog digest is absorbed into the verifier setup transcript.
Recursive cutoff: isolated four-thread measurements
Fresh measurements ran on temporary ASCII Box
bx_8ge5dtvp: large 8-vCPU/16-GB AMD EPYC-Rome VM, Ubuntu x86_64, no injected environment or secrets. The release process was pinned to CPUs 0-3, with four prover threads and an explicitly single-threaded verifier. Each size has seven retained alternating direct/offloaded pairs after one warm-up pair. Prover time is commitment plus proof generation.Values below are medians; bracketed values are percentile-bootstrap 95% intervals for the paired median.
2^202^212^222^232^24This isolated host does not support the current
log_T = 21crossover.2^23is the first size whose paired median clears the approximate 2x target, with only 1.38% median prover overhead, although its bootstrap interval still touches just below 2x.2^24is the first conservative crossover: all seven pairs exceed 2x and the interval lower bound is 2.55x.The checked-in
RECURSIVE_TRACE_LOG_T_CUTOVER = 21remains the implementation in this head; the benchmark evidence recommends moving it to 23 for the intended approximate-2x policy, or 24 for a strict lower-confidence-bound policy, followed by catalog regeneration.Raw run: 70 retained samples, 2 warm-ups, zero failures/retries; JSONL SHA-256
134a47af8f3e5b6a95a55ed18c0f2f52feec50f277a86e04cc8477c2d7e97911; benchmark binary SHA-256ee28ecc809b975fa2e6ed0401ff3a3ec7c3837d55c044a966afabc8b829b7d68.Complete Akita batched PCS proof-size cost
2^202^212^222^232^24These are complete serialized Akita batched PCS proofs (the fixed schedule selection plus backend body), not complete outer Jolt proofs.
Number of setup-offloaded levels
The production trace packs to one physical polynomial. The current checked-in catalogs contain:
2^212^21 .. 2^242^252^26 .. 2^30The catalogs extend through logical
log_T = 30. Two-polynomial adapter/tamper rows are also retained: their only topology difference is that the K256log_T = 25compatibility row already offloads two folds. Moving the cutoff to 23 would make the2^21and2^22production rows direct while leaving the offload depth at2^23and above unchanged.Compatibility
This is a protocol-epoch-breaking change. Old generated Rust catalogs, old schedule selections, serialized Akita-mode proofs, and preprocessing produced against the previous Akita revision are intentionally incompatible. There is no fallback/default catalog.
Change map
crates/jolt-akita/schedules/,src/schedules/,gen_jolt_schedules.rsscheme.rs,adapters.rs,schedule_registry.rsshape_guard.rs,native_batching.rsjolt-openings,jolt-prover,jolt-prover-legacyValidation
Completed locally at
8cb334b1987a29e19e013d4c31a44f93dbdda5caafter restacking onmain:cf874a6f1e187d2d3fa551557a0d33b4719b0f40is byte-identical to prior head48643c013;python3 scripts/check_style_invariants.py --base origin/main: clean;cargo fmt --all --check,taplo fmt --check, andgit diff --check origin/main...HEAD: clean;cargo nextest run -p jolt-akita: 47 passed;cargo nextest run -p jolt-verifier --test fs_obligations --features fs-audit --cargo-quiet: 1 passed.The push triggered fresh CI for the ancestry-only merge. The immediately preceding content-identical head
48643c013had 42/42 reported checks successful, including the complete Jolt build/test workflow, Akita fixtures, schedule drift, byte parity, FS attacks/obligations, formatting, style, Clippy, and formal-verification jobs.Earlier full local validation on the same implementation also passed:
cargo clippy -p jolt-akita --all-targets -- -D warnings;cargo clippy -p jolt-prover --all-targets -- -D warnings;typos.Review order
crates/jolt-akita/src/scheme.rsandschedule_registry.rs: catalog authority and guided admission.shape_guard.rsandnative_batching.rs: trusted selection before decode and same-instance verification.jolt-openingsplus prover call sites: explicit setup context.