Skip to content

[AUTOMATED] Bound preparation workers and borrow immutable runtime metadata - #246

Merged
sjkim1127 merged 2 commits into
mainfrom
codex/bounded-preparation-workers
Oct 15, 2026
Merged

sjkim1127 merged 2 commits into
mainfrom
codex/bounded-preparation-workers

Conversation

@sjkim1127

@sjkim1127 sjkim1127 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

[AUTOMATED]

Scope

Mechanically accelerate cold FID/no-return preparation and immutable runtime metadata lookup. No semantic/readability/score/ranking claim, CLI/schema/adapter/dependency change, skipped function, modified instruction limit or FID matching rule.

  • Borrow an indexed canonical registry descriptor for hot status/engine queries and hoist invariant endianness; preserve the public owned API, duplicate manifest order, exact original selection predicate and missing-entry errors.
  • Only seed-capturing cold FID preparation uses bounded scoped OS helpers; ordinary no-return callers remain sequential. Immutable per-function decoding cannot query CF/no-return. A local atomic queue assigns original positions once; the caller works and joins without stealing outer Rayon jobs under the CF OnceLock.
  • Existing Rayon width includes the caller; process-wide nonblocking permits bound extra helpers. Thread creation/permit failure reduces parallelism without dropping work. Restore original input order before map insertion. Preserve fixed point, raw seed/complete decode cache contracts and fallback behavior. RAII returns permits and propagates panics.

Measured evidence

Base main 3daf458fef337a2951e000a17f13fac61d4a2e24; frozen qualification revision 07dd95dbe6214b8a78fc1e4a55961ab515986489.

Large six-function native batch, three alternating baseline/candidate pairs, existing default rendering/FID workers; medians:

Baseline Candidate
Wall seconds 27.2216 15.4257
CPU seconds 122.0768 136.2194
Maximum RSS bytes 2,329,264,128 2,416,246,784

Wall -43.3%, CPU +11.6%, RSS +3.7%. This lowers latency at higher CPU cost. Small five-pair median changes are only 1.6–6.7%. No per-core efficiency or memory-cap claim. Native M4 Pro/24 GiB, release LTO; OS caches/scheduling uncontrolled. No own builds/tests/profilers during primary timing controls.

Fresh 250/250 stripped functions across 45 cells: zero changed non-time function records, including all code/layers, variables/types/PCs/maps, errors/fallbacks/evidence/counters; only seven existing durations excluded. Focused 22-function complete records also match three repetitions in each of two worker lanes and the fresh replay. Whole-population two-run determinism, guest execution/recompilation and rescoring were not measured.

Kuna 1.808 is compared with both 1 and 14 requested workers. Choosing each tool's faster observed setting separately still leaves Fission 2.83–7.24x slower; Kuna-level speed is not achieved. One requested Fission render worker does not serialize FID/preparation. We do not pick Kuna's slower 14-job setting to claim parity.

Validation

  • 17 focused nextest tests; 1,855 relevant sleigh/static/pcode/decompiler/CLI tests passed, 6 skipped.
  • Registry differential: every manifest entry, invalid/duplicate keys, borrowed identity and owned independence.
  • Queue ordering/every-input-once/skips, caller-only fallback, real OS spawn failure, unavailable permits, panic recovery, concurrent helper cap; saturated outer Rayon/OnceLock stress (24 rounds × 24 callers, explicit deadline) completes with exactly one initializer each round.
  • Serial/scoped no-return facts and all 16 captured fixture decodes match exactly. Existing raw context/bytes/delay/partial-capture coverage remains green.
  • Crate checks, release CLI, fmt, owner boundaries and scoped CI Clippy passed.
  • Separate prototype A1 symbolized samples (before the final CAS portability fix) expose remaining FPK lookup/decompression/candidate parsing; these are diagnostic snapshots, not timing or whole-run attribution. Source hashes/patch/tree, full ranges, controls and reproduction: docs/validation/preparation-workers-performance.{md,json}; proposal docs/proposals/2026-10-15-preparation-workers.md.
  • Full artifacts: benchmark/artifacts/performance/2026-10-15-preparation-workers/.

All-function hot-cache growth remains unchanged. Drafts #203/#204, DecBench #103, primary checkout and seven unrelated dirty main files are preserved. Remote CI is a separate post-submission gate and is not claimed green by local tests.

CI portability follow-up

Initial CI passed Linux tests and PE CLI smoke; lint failed because newer CI Rust deprecates fetch_update. An equivalent explicit compare_exchange_weak loop preserves the helper limit and memory order without a new API requirement or warning suppression. All timing controls, 250-function replay and both Kuna worker lanes were rerun on final A2 SHA256 5fde8204f59b90a4fba7ca2af4c32c08c41efcb066df8586a9085cf7022f65d7. The table above now reports A2; older prototype controls/samples remain explicitly separate. New-head remote CI is still required.

@sjkim1127
sjkim1127 merged commit a0539e9 into main Oct 15, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant