Skip to content

test(EthConformance): bound actual request SSZ sizes - #409

Open
alexanderlhicks wants to merge 30 commits into
test/stateless-request-size-clientsfrom
feat/stateless-request-bounds-current
Open

alexanderlhicks wants to merge 30 commits into
test/stateless-request-size-clientsfrom
feat/stateless-request-bounds-current

Conversation

@alexanderlhicks

Copy link
Copy Markdown
Owner

Motivation

Bound the actual request counts and byte leaves used by the StatelessInput SSZ model. These local facts support later caller and root-domain composition without imposing a global validity invariant.

Change in behaviour or contract

Adds 15 private declarations to the existing request-size caller module. Every one of the eight actual array counts, every member transaction's byte length, extra data and the BAL is bounded by the raw request size R without a support premise. Encoded-size variants require only that actual request's Serializable proof.

Actual whole-input Serializable projects actual request support and proves the strict later witness start 16 + R < 2^32. The encoded version accepts any support proof for the same actual request. Empty transaction occurrences retain their four-byte offsets even when their payload-byte sum is zero. No final witness-end or total-input bound is claimed.

EELS sources

Pinned src/ethereum/forks/amsterdam/execution_engine/types.py:33–96, execution_engine/requests.py:99–116, stateless.py:144–165, and src/ethereum/utils/ssz.py:174–220,245–291. The additions are mathematical callers of the existing actual dictionaries and public Codec laws; they do not execute an EELS serializer or guest.

Source-to-Lean rows

EthStateless §3 describes the private size clients; §7 states their complete domains and hypotheses. The controls preserve all actual counts and member occurrences, arbitrary supported requests, the strict offset boundary, and the distinction between empty payload bytes and transaction offset overhead.

Decisions and questions

No new public API, decision, validity condition, WF, Rootable, CountRecoverable or uint256 premise. Decoder and inverse composition, source/host agreement, O2/O12, generic D5/F20, fuel, backend and guest/resource/security/readiness obligations remain open.

The integration preserves current main's accepted Q90 diagnostics, strict storage-leaf admission, parser-test cleanup and DISC-001 measurements. Those inherited changes add no new disposition in this work item.

Checks and tests

All 148 current required command lines passed. Core declaration audit: 31,509 declarations, zero findings. Kernel replay: 532 modules. The changed proof file also elaborates with fatal warnings and autoImplicit/relaxedAutoImplicit disabled; independent symbolic and negative controls cover the local support and strict-offset boundaries.

Review verdicts

Both independent adversarial reviews are CLEAN. A nonblocking source-table placement nit is deferred under CONTRIBUTING §5.5. The exact source and evidence passed final evaluation before commit.

Documentation

Updates EthStateless's private caller contracts, tests and gaps, COMPOSITION and REVIEW handoffs, generated GAPS and citation records. Corrects the global SSZ source paths and regenerates the citation ledger after integrating both accepted histories. Whole-guest readiness is unchanged.

Stack

Depends on #403 at 239e3ad95c6ec02f361532cb47e658871662c029, with current main 50dfb658b7eb578ab377fe080ae20d5bebb97bfb as an integration parent. Uses a new branch; existing maintainer-owned PR branches are unchanged.

Generated with Codex.

alexanderlhicks and others added 30 commits November 6, 2026 04:07
Follow P1/X16 at witness_state.py:127 using the public strict RLP and bounded
integer providers. Reject lists, first-zero and over-width payloads while
preserving complete values and the existing coarse leaf error and SC10 domain.

Update local contracts, composition, citations and complete native/interpreted
635-row regressions. All 139 required checks and both fresh independent reviews
pass on the exact diff; ROOT evaluation precedes this commit.

Co-authored-by: Codex <noreply@openai.com>
…rguments

Bring storage_decode_recipe.py in line with the style of other drivers in
the repository, for example rlp_u256_differential.py: spaces after commas
and around operators, two blank lines between top-level definitions, one
statement per line, and the long case tables split one tuple per line so
every line stays under 100 characters. The rows, their order, ids, wire
bytes, results and derivation strings are unchanged; the regenerated
catalogue.jsonl, input.jsonl and expected.jsonl are byte-for-byte identical
to the previous output and to the committed storage_decode_inputs.jsonl and
storage_decode_expected.jsonl.

The output directory is now an argparse positional OUTPUT_DIR instead of a
bare sys.argv[1], so --help prints usage and unknown flags are refused rather
than being taken as a directory name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… evidence

Wrap the lines of 100 or more characters in storage_decode_differential.py,
test_storage_decode_output_parser.py and StorageDecodeGuards.lean. These are
whitespace and line-break changes only; the token sequence is unchanged.

The pinned decode_to(U256) observations cited in the differential checker's
docstring and in EthStateCommit §3 have no committed driver or record. Label
both claims with the repository's out-of-tree evidence wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(scripts): refresh source probes for the current EELS pin
…-witness

feat(EthStateless): add the raw execution witness carrier
feat(EthFork): add Amsterdam input wire limits
…fork

feat(EthStateless): add historical ProtocolFork metadata
feat(EthStateless): certify the ExecutionWitness SSZ dictionary
…tems

feat(EthStateless): add raw execution-request item carriers
…-requests

feat(EthStateless): add the raw ExecutionRequests aggregate
feat(EthBlock): add raw block header carriers
feat(EthBlock): add the raw Withdrawal carrier
…arriers

feat(EthStateless): add raw payload and request carriers
feat(EthStateless): certify the ExecutionPayload SSZ schema
test_reference_division_output_parser.py and
test_header_carriers_output_parser.py wrote every run to a fresh
.lake/{division,header}-parser-<uuid4> directory and never removed it.
Each run leaves ~300 MB behind, and across the worktree pipeline these
accumulated to well over 100 GB.

Without --output-dir, both tests now write to a temporary directory
that is removed when the run finishes (TemporaryDirectory / addCleanup),
matching the other output-parser tests. With --output-dir the
behaviour is unchanged, so evidence capture still keeps its files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-up. Replace mkdtemp plus addCleanup(shutil.rmtree,
ignore_errors=True) with enterContext(TemporaryDirectory(...)), so a
failed removal raises instead of silently recreating the leak. This
also drops the shutil import and matches the division test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(scripts): clean up default parser-test output dirs
Keeps both sides of the appended CI, CONTRIBUTING and lakefile blocks
(main's header-carrier regressions, then the storage-leaf regressions).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #347. At tests-zkevm@v21.0.6, decode_witness_to_mpt runs
_validate_secured_trie_paths on every secured decode
(incremental_mpt.py:987-1013 at 59c1473d). decodeWitnessToMpt now does the
same: with secured set, a complete decoded root failing securedPathsOk
against the 64-nibble key returns .malformed .securedPathLength, after every
decode diagnostic and with no added query. Unsecured decodes are unchanged.

- Malformed.securedPathLength (Q90), wire code 13 in every wire function.
- securedTrieKeyNibbles, securedPathsOk and its five public equations;
  decodeWitnessToMpt_unsecured and decodeWitnessToMpt_secured.
- Caller proofs for the accepted, rejected and error cases; Q90 guards
  comparing secured and unsecured decodes of the same database.
- decoder_differential.py classifies the three assertions as code 13 and adds
  secured cases: 394 cases, 788 records, all equal at 59c1473d.
- decoder_diagnostic_differential.py line expectations moved to 59c1473d.
- EthCommit C13, section 3, 5, 9, 10 and DECISIONS Q90 updated; three listed
  citations resolved.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- §10 re-pin item (a) no longer says decodeWitnessToMpt skips the check;
  GAPS regenerated.
- The guard summary lists extensions of 62-64 nibbles, as the guards do.
- §4 lists the overlength EEST fixture and the secured-path adversarial rows.
- §5 lists securedTrieKeyNibbles and securedPathsOk.
- The IncrementalMPT row cites the wrapper at 1046-1065 (59c1473d); its
  listed entry is resolved and the :441/452 entry relisted.
- Full citation path in TrieError; two long lines wrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The row cited 111-123, the class's range at e1a316a0. At 59c1473d the class
runs from its decorators at 114 to its last field at 127. The :441/452 entry
on the same line is relisted unchanged. The root-leaf guard comment now
lists all six rejected lengths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s-v2

fix(EthStateCommit): enforce strict storage-leaf admission
Resolve the citation-remap-manual.txt conflict on the two REVIEW.md
entries by keeping this branch's lines (539 and 545). Neither side
touches REVIEW.md, and the cited text is on those lines in the merged
tree. remap_citations --check passes with and without --eels (84
entries listed for manual review), and gen_gaps --check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(EthCommit): reject secured-trie path-length failures (Q90)
Restate the deep-witness-chain items against Q90's secured path
check, with decode-level measurements at 59c1473d. DISC-001 owns the
numbers, and the other documents cite it.

- Chains of trie nodes: every guest decode is secured, so a chain
  beyond the 65-node bound is `false` in both EELS (path assert or
  RecursionError, caught at stateless.py:294) and the Q90 decoder. The
  recursion limit no longer decides the guest output.
- RLP nesting inside a branch value (slot 16) is not bounded by that
  check. EELS raises RecursionError beyond a host-dependent depth,
  while the total spec decoder reads the value as empty. This is
  recorded as a still-open guest-level divergence candidate (DISC-001,
  Q6/Q20, EthStateless O12 (b)).
- The new STFSpec/Conformance/Commit/deep_chain_probe.py reproduces
  both at decode level. It checks the pin and prints input digests.
  Depths are measured from the probe's own stack; the guest fails
  earlier.

Updated: DISCREPANCIES (DISC-001), DECISIONS Q6/Q20, GAPS-CROSSCUTTING,
EthCommit C17 and §10 (a), EthStateWitness §10, EthStateless §10.
Fix the IncrementalMPT.lean docstring citations (:114–127 and :425)
and drop the three resolved ledger entries (81 remain listed). The
ledger also refreshes one unrelated line (EthStateCommit.md 217 →
219). GAPS.md is regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Probe threshold: the committed probe gives 49,994/49,995 here, and an
  independent run on the same versions gave 49,995/49,996. Record both
  and stop stating an exact upper end for the path-check range. The
  probe's usage line now covers both.
- Spec executable: an independent review run measured the compiled
  decoder's memory growing with the square of chain depth (22 GB at
  50,000; out of memory near 79 GB at 100,000), likely from copying the
  on-path HashSet at every decodeEntry level (Decoder.lean:172–183).
  Record this in DISC-001 and C17, and list it as a known DEBT
  candidate. A full DEBT entry needs a committed reproducer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The one-frame difference between the two probe runs is stable within
each installed environment. The full locked EELS venv always gives
49,994/49,995, and a smaller venv with the same interpreter and
package versions always gives 49,995/49,996. It does not vary from
run to run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs(DISC-001): measure deep witness nesting at the pin
Prove raw request count and member-byte bounds, local supported encoding
transports and the strict later witness-start consequence of actual input
support. Retain empty transaction offset overhead and all parent contracts.
Integrate current main's accepted storage, Q90 and tooling changes.

All 148 required commands pass in four isolated lanes. Both exact-diff
reviews CLEAN; nonblocking table-placement nit deferred. ROOT evaluation
precedes commit. No public API or whole-guest readiness change.

Co-Authored-By: Codex <noreply@openai.com>

@alexanderlhicks alexanderlhicks left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fresh review of d7ac9fcd4e1ca4b09dc31b61784ae8817ec62fba (tree 694026cd23d4ba6a943bceddc7ed12b51a3c3a7a): CLEAN_SCOPED_WORK_ITEM for the new work described below. This is not a whole-head clean verdict.

The fifteen new private proof clients bound all eight actual request counts and the byte lengths of each transaction, extraData and the block access list by the raw request logical size R. Actual encoded-size conclusions retain their own Serializable premises. Whole-input Serializable projects support for that same request and proves the later witness start 16 + R < 2^32; it adds no final witness-end cap. Independent source-first mathematical audits checked the formula and its boundary conditions. The new proofs do not rely on the inherited documentation claim discussed below.

All 148 literal CONTRIBUTING §6.3 commands passed in each of three fresh, isolated checkouts. Each lane reports 31,509 declarations with zero findings and kernel replay of all 532 modules. Both independent final reports, original command streams, all 22 checklist producer outputs per lane and byte-compared own-cache archives were authenticated. Auxiliary setup or inspection corrections receive no candidate-validation credit; no historical process or foreign build cache was reused as a current pass.

The material documentation finding inherited unchanged from main in #408 remains open. The oversized single-node raw-decoder probes bypass the guest SSZ admission bound and do not establish the claimed guest-level divergence. This scoped verdict neither waives that finding nor downgrades it to a nit.

The independent source audits retain their original chronology and are bound again to these unchanged source bytes. All current validation completed in fresh campaigns on this exact head.

Local validation records uv 0.8.17 while CI pins 0.13.6; literal uv/Python 3.12 commands were retained, without a claim of CI-environment equivalence. No material effect from this driver difference was identified.

This review establishes no new EELS runtime agreement, guest/EEST, resource, security, cost or readiness result. Current-main integration remains separate. This review made no tracked edits, fixes, commits, pushes or merges.

The two source-map rows before the EthStateless title and the source-span formatting are nonblocking documentation polish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant