Repository navigation
test(EthConformance): bound actual request SSZ sizes - #409
alexanderlhicks wants to merge 30 commits into
Conversation
Follow P1/X16 at witness_state.py:127 using the public strict RLP and bounded integer providers. Reject lists, first-zero and over-width payloads while preserving complete values and the existing coarse leaf error and SC10 domain. Update local contracts, composition, citations and complete native/interpreted 635-row regressions. All 139 required checks and both fresh independent reviews pass on the exact diff; ROOT evaluation precedes this commit. Co-authored-by: Codex <noreply@openai.com>
…rguments Bring storage_decode_recipe.py in line with the style of other drivers in the repository, for example rlp_u256_differential.py: spaces after commas and around operators, two blank lines between top-level definitions, one statement per line, and the long case tables split one tuple per line so every line stays under 100 characters. The rows, their order, ids, wire bytes, results and derivation strings are unchanged; the regenerated catalogue.jsonl, input.jsonl and expected.jsonl are byte-for-byte identical to the previous output and to the committed storage_decode_inputs.jsonl and storage_decode_expected.jsonl. The output directory is now an argparse positional OUTPUT_DIR instead of a bare sys.argv[1], so --help prints usage and unknown flags are refused rather than being taken as a directory name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… evidence Wrap the lines of 100 or more characters in storage_decode_differential.py, test_storage_decode_output_parser.py and StorageDecodeGuards.lean. These are whitespace and line-break changes only; the token sequence is unchanged. The pinned decode_to(U256) observations cited in the differential checker's docstring and in EthStateCommit §3 have no committed driver or record. Label both claims with the repository's out-of-tree evidence wording. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(scripts): refresh source probes for the current EELS pin
…-witness feat(EthStateless): add the raw execution witness carrier
feat(EthFork): add Amsterdam input wire limits
…fork feat(EthStateless): add historical ProtocolFork metadata
feat(EthStateless): certify the ExecutionWitness SSZ dictionary
…tems feat(EthStateless): add raw execution-request item carriers
…-requests feat(EthStateless): add the raw ExecutionRequests aggregate
feat(EthBlock): add raw block header carriers
feat(EthBlock): add the raw Withdrawal carrier
…arriers feat(EthStateless): add raw payload and request carriers
feat(EthStateless): certify the ExecutionPayload SSZ schema
test_reference_division_output_parser.py and
test_header_carriers_output_parser.py wrote every run to a fresh
.lake/{division,header}-parser-<uuid4> directory and never removed it.
Each run leaves ~300 MB behind, and across the worktree pipeline these
accumulated to well over 100 GB.
Without --output-dir, both tests now write to a temporary directory
that is removed when the run finishes (TemporaryDirectory / addCleanup),
matching the other output-parser tests. With --output-dir the
behaviour is unchanged, so evidence capture still keeps its files.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-up. Replace mkdtemp plus addCleanup(shutil.rmtree, ignore_errors=True) with enterContext(TemporaryDirectory(...)), so a failed removal raises instead of silently recreating the leak. This also drops the shutil import and matches the division test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(scripts): clean up default parser-test output dirs
Keeps both sides of the appended CI, CONTRIBUTING and lakefile blocks (main's header-carrier regressions, then the storage-leaf regressions). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #347. At tests-zkevm@v21.0.6, decode_witness_to_mpt runs _validate_secured_trie_paths on every secured decode (incremental_mpt.py:987-1013 at 59c1473d). decodeWitnessToMpt now does the same: with secured set, a complete decoded root failing securedPathsOk against the 64-nibble key returns .malformed .securedPathLength, after every decode diagnostic and with no added query. Unsecured decodes are unchanged. - Malformed.securedPathLength (Q90), wire code 13 in every wire function. - securedTrieKeyNibbles, securedPathsOk and its five public equations; decodeWitnessToMpt_unsecured and decodeWitnessToMpt_secured. - Caller proofs for the accepted, rejected and error cases; Q90 guards comparing secured and unsecured decodes of the same database. - decoder_differential.py classifies the three assertions as code 13 and adds secured cases: 394 cases, 788 records, all equal at 59c1473d. - decoder_diagnostic_differential.py line expectations moved to 59c1473d. - EthCommit C13, section 3, 5, 9, 10 and DECISIONS Q90 updated; three listed citations resolved. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- §10 re-pin item (a) no longer says decodeWitnessToMpt skips the check; GAPS regenerated. - The guard summary lists extensions of 62-64 nibbles, as the guards do. - §4 lists the overlength EEST fixture and the secured-path adversarial rows. - §5 lists securedTrieKeyNibbles and securedPathsOk. - The IncrementalMPT row cites the wrapper at 1046-1065 (59c1473d); its listed entry is resolved and the :441/452 entry relisted. - Full citation path in TrieError; two long lines wrapped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The row cited 111-123, the class's range at e1a316a0. At 59c1473d the class runs from its decorators at 114 to its last field at 127. The :441/452 entry on the same line is relisted unchanged. The root-leaf guard comment now lists all six rejected lengths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-v2 fix(EthStateCommit): enforce strict storage-leaf admission
Resolve the citation-remap-manual.txt conflict on the two REVIEW.md entries by keeping this branch's lines (539 and 545). Neither side touches REVIEW.md, and the cited text is on those lines in the merged tree. remap_citations --check passes with and without --eels (84 entries listed for manual review), and gen_gaps --check passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(EthCommit): reject secured-trie path-length failures (Q90)
Restate the deep-witness-chain items against Q90's secured path check, with decode-level measurements at 59c1473d. DISC-001 owns the numbers, and the other documents cite it. - Chains of trie nodes: every guest decode is secured, so a chain beyond the 65-node bound is `false` in both EELS (path assert or RecursionError, caught at stateless.py:294) and the Q90 decoder. The recursion limit no longer decides the guest output. - RLP nesting inside a branch value (slot 16) is not bounded by that check. EELS raises RecursionError beyond a host-dependent depth, while the total spec decoder reads the value as empty. This is recorded as a still-open guest-level divergence candidate (DISC-001, Q6/Q20, EthStateless O12 (b)). - The new STFSpec/Conformance/Commit/deep_chain_probe.py reproduces both at decode level. It checks the pin and prints input digests. Depths are measured from the probe's own stack; the guest fails earlier. Updated: DISCREPANCIES (DISC-001), DECISIONS Q6/Q20, GAPS-CROSSCUTTING, EthCommit C17 and §10 (a), EthStateWitness §10, EthStateless §10. Fix the IncrementalMPT.lean docstring citations (:114–127 and :425) and drop the three resolved ledger entries (81 remain listed). The ledger also refreshes one unrelated line (EthStateCommit.md 217 → 219). GAPS.md is regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Probe threshold: the committed probe gives 49,994/49,995 here, and an independent run on the same versions gave 49,995/49,996. Record both and stop stating an exact upper end for the path-check range. The probe's usage line now covers both. - Spec executable: an independent review run measured the compiled decoder's memory growing with the square of chain depth (22 GB at 50,000; out of memory near 79 GB at 100,000), likely from copying the on-path HashSet at every decodeEntry level (Decoder.lean:172–183). Record this in DISC-001 and C17, and list it as a known DEBT candidate. A full DEBT entry needs a committed reproducer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The one-frame difference between the two probe runs is stable within each installed environment. The full locked EELS venv always gives 49,994/49,995, and a smaller venv with the same interpreter and package versions always gives 49,995/49,996. It does not vary from run to run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs(DISC-001): measure deep witness nesting at the pin
Prove raw request count and member-byte bounds, local supported encoding transports and the strict later witness-start consequence of actual input support. Retain empty transaction offset overhead and all parent contracts. Integrate current main's accepted storage, Q90 and tooling changes. All 148 required commands pass in four isolated lanes. Both exact-diff reviews CLEAN; nonblocking table-placement nit deferred. ROOT evaluation precedes commit. No public API or whole-guest readiness change. Co-Authored-By: Codex <noreply@openai.com>
alexanderlhicks
left a comment
There was a problem hiding this comment.
Fresh review of d7ac9fcd4e1ca4b09dc31b61784ae8817ec62fba (tree 694026cd23d4ba6a943bceddc7ed12b51a3c3a7a): CLEAN_SCOPED_WORK_ITEM for the new work described below. This is not a whole-head clean verdict.
The fifteen new private proof clients bound all eight actual request counts and the byte lengths of each transaction, extraData and the block access list by the raw request logical size R. Actual encoded-size conclusions retain their own Serializable premises. Whole-input Serializable projects support for that same request and proves the later witness start 16 + R < 2^32; it adds no final witness-end cap. Independent source-first mathematical audits checked the formula and its boundary conditions. The new proofs do not rely on the inherited documentation claim discussed below.
All 148 literal CONTRIBUTING §6.3 commands passed in each of three fresh, isolated checkouts. Each lane reports 31,509 declarations with zero findings and kernel replay of all 532 modules. Both independent final reports, original command streams, all 22 checklist producer outputs per lane and byte-compared own-cache archives were authenticated. Auxiliary setup or inspection corrections receive no candidate-validation credit; no historical process or foreign build cache was reused as a current pass.
The material documentation finding inherited unchanged from main in #408 remains open. The oversized single-node raw-decoder probes bypass the guest SSZ admission bound and do not establish the claimed guest-level divergence. This scoped verdict neither waives that finding nor downgrades it to a nit.
The independent source audits retain their original chronology and are bound again to these unchanged source bytes. All current validation completed in fresh campaigns on this exact head.
Local validation records uv 0.8.17 while CI pins 0.13.6; literal uv/Python 3.12 commands were retained, without a claim of CI-environment equivalence. No material effect from this driver difference was identified.
This review establishes no new EELS runtime agreement, guest/EEST, resource, security, cost or readiness result. Current-main integration remains separate. This review made no tracked edits, fixes, commits, pushes or merges.
The two source-map rows before the EthStateless title and the source-span formatting are nonblocking documentation polish.
Motivation
Bound the actual request counts and byte leaves used by the StatelessInput SSZ model. These local facts support later caller and root-domain composition without imposing a global validity invariant.
Change in behaviour or contract
Adds 15 private declarations to the existing request-size caller module. Every one of the eight actual array counts, every member transaction's byte length, extra data and the BAL is bounded by the raw request size R without a support premise. Encoded-size variants require only that actual request's Serializable proof.
Actual whole-input Serializable projects actual request support and proves the strict later witness start
16 + R < 2^32. The encoded version accepts any support proof for the same actual request. Empty transaction occurrences retain their four-byte offsets even when their payload-byte sum is zero. No final witness-end or total-input bound is claimed.EELS sources
Pinned
src/ethereum/forks/amsterdam/execution_engine/types.py:33–96,execution_engine/requests.py:99–116,stateless.py:144–165, andsrc/ethereum/utils/ssz.py:174–220,245–291. The additions are mathematical callers of the existing actual dictionaries and public Codec laws; they do not execute an EELS serializer or guest.Source-to-Lean rows
EthStateless §3 describes the private size clients; §7 states their complete domains and hypotheses. The controls preserve all actual counts and member occurrences, arbitrary supported requests, the strict offset boundary, and the distinction between empty payload bytes and transaction offset overhead.
Decisions and questions
No new public API, decision, validity condition, WF, Rootable, CountRecoverable or uint256 premise. Decoder and inverse composition, source/host agreement, O2/O12, generic D5/F20, fuel, backend and guest/resource/security/readiness obligations remain open.
The integration preserves current main's accepted Q90 diagnostics, strict storage-leaf admission, parser-test cleanup and DISC-001 measurements. Those inherited changes add no new disposition in this work item.
Checks and tests
All 148 current required command lines passed. Core declaration audit: 31,509 declarations, zero findings. Kernel replay: 532 modules. The changed proof file also elaborates with fatal warnings and autoImplicit/relaxedAutoImplicit disabled; independent symbolic and negative controls cover the local support and strict-offset boundaries.
Review verdicts
Both independent adversarial reviews are CLEAN. A nonblocking source-table placement nit is deferred under CONTRIBUTING §5.5. The exact source and evidence passed final evaluation before commit.
Documentation
Updates EthStateless's private caller contracts, tests and gaps, COMPOSITION and REVIEW handoffs, generated GAPS and citation records. Corrects the global SSZ source paths and regenerates the citation ledger after integrating both accepted histories. Whole-guest readiness is unchanged.
Stack
Depends on #403 at
239e3ad95c6ec02f361532cb47e658871662c029, with current main50dfb658b7eb578ab377fe080ae20d5bebb97bfbas an integration parent. Uses a new branch; existing maintainer-owned PR branches are unchanged.Generated with Codex.