Skip to content

fix(storage): a silent set-mate cannot hold the mirror sync; no node source reads a clock; five route-chain rows re-verified - #1088

Closed
cryptskii wants to merge 4 commits into
mainfrom
fix/storage-mirror-liveness-and-route-chain-records
Closed

cryptskii wants to merge 4 commits into
mainfrom
fix/storage-mirror-liveness-and-route-chain-records

Conversation

@cryptskii

Copy link
Copy Markdown
Collaborator

Storage batch 2: one liveness bug fixed, the node's no-clock rule enforced by a test, and five route-chain rows re-verified against main.

Implemented and proven in this PR

A silent set-mate held the mirror sync (§6.53).

  • The node's set client had no connect or request timeout, and mirror_sync holds a node-wide one-at-a-time lock.
  • A set-mate that accepted a connection and never answered held that sync, and every later one on the node, open indefinitely.
  • The fix:
    • set_client: connect bound 5 s, request bound 10 s. A fetch that times out fails as an unreachable set-mate does (502, lock released). It stores and orders nothing, so no protocol fact reads the clock.
    • MAX_SYNC_CYCLES 1024 → 128, so one sync stays inside the SDK's 30 s request bound. Every cycle is kept as it is fetched, so the next sync continues where this one stopped.
  • Test: dsm_storage_node::bytecommit_chain::a_set_mate_that_never_answers_fails_the_sync_and_frees_it.
  • Mutation control: timeouts removed → red ("sync 0 hung on a set-mate that never answers").
  • In the SDK, route_seats::read_cell now asks every seat at once (values, cycle closes, mirror syncs), with the answers kept in route order. A silent seat costs one client timeout instead of one per seat.

No node source reads a clock: MR-STOR-0008, Partial → Met.

  • New test dsm_storage_node::no_clock_reads parses every node source with syn:
    • it reads paths and use trees whole, and macro bodies token by token;
    • it refuses Instant, SystemTime, UNIX_EPOCH, chrono and tokio::time;
    • comments and strings never count.
  • Its self-test covers grouped, renamed and glob imports, a macro body, and a comment and string that must not count. A first token-window draft missed use tokio::{sync::Mutex, time}; the self-test caught it.
  • Mutation controls, each red and then restored:
    • a grouped-import tokio::time sleep in the spool;
    • Instant::now() in db/pg.rs.
  • ci/no_clock_and_no_json.sh loses two node exemptions: the spool (which reads no clock) and a rate limiter that no longer exists. A clock read in the spool now fails the script too (checked, then restored).
  • Adds syn and proc-macro2 as node dev-dependencies. Both are already in the lock at the same versions; Cargo.lock changes only the node's dependency list.

Re-verified as already satisfied on main (records only)

These rows were Missing on notes written before route chains existed. Each cited test passes at this tree.

Row Now Evidence
MR-STOR-0131 links Met route_chain::evaluate; links carry position and prior chain
MR-STOR-0133 later-link validity Met leader chain, higher position, seat's mirror of the leader
MR-STOR-0138 Preserved ≠ Final Met every consumer treats Preserved as not final
MR-STOR-0094 ByteCommit not counted Met mirrors must agree; Core checks chain link and root itself
MR-STOR-0134 empties Partial no-response recorded and never counted; the writer never produces a taken empty
  • MR-STOR-0140 also cites the silent set-mate test.
  • VERIFICATION_MATRIX gains rows for the liveness test and the clock test, each with its mutation control.

Verification

  • dsm_storage_node bytecommit_chain 8/0 and no_clock_reads 2/0 (release, Postgres).
  • dsm_sdk route_seats 9/0 (release).
  • cargo fmt --all --check and clippy -p dsm_storage_node --all-targets -D warnings are clean. scripts/real_code_guard.py, ci/no_clock_and_no_json.sh and ci/conformance_evidence.py pass.
  • Branch is 0 behind main at d19ea36.

…ead asks every seat at once

Node side:
- The set client (set_client::pinned_set_client) had no connect or
  request timeout, and mirror_sync holds a node-wide one-at-a-time lock. A
  set-mate that accepted the connection and never answered held the sync
  and the lock open indefinitely, stalling every later mirror sync on the
  node.
- It now connects within SET_MATE_CONNECT_TIMEOUT (5 s) and answers within
  SET_MATE_REQUEST_TIMEOUT (10 s), or the fetch fails, as an unreachable
  set-mate does: BAD_GATEWAY, with the lock released. These are transport
  liveness bounds; a fetch that times out stores nothing and orders
  nothing.
- MAX_SYNC_CYCLES drops from 1024 to 128, so one sync of a member far
  behind stays well inside the SDK's 30 s per request. Every cycle is kept
  as it is fetched, so later syncs continue.

SDK side: route_seats::read_cell asked the five seats one after another, to
read values, to close cycles and to sync mirrors. A seat that does not answer
cost one client timeout per seat in each loop. The three fan-outs now run at
once (join_all), with the answers kept in route order, so the evidence Core
evaluates is unchanged.

Test: bytecommit_chain::a_set_mate_that_never_answers_fails_the_sync_and_frees_it
(a set-mate that accepts and never answers; two syncs each answer
BAD_GATEWAY within 3× the request bound). Mutation control: the set-client
timeouts removed → red ("sync 0 hung on a set-mate that never answers"),
restored.

Release, on Postgres: bytecommit_chain 8/0. route_seats, faucet flows and the
SoFi trade and pay node e2e tests 21/0. fmt, clippy -D warnings (dsm_sdk,
dsm_storage_node), the real-code guard and the clockless gate are clean.
…ws re-verified

MR-STOR-0131, 0133, 0138 and 0094 move Missing → Met and 0134 Missing →
Partial: each Missing note described a codebase before route chains. The
cited tests pass at this tree. 0134 stays Partial because the writer never
records a taken empty.

MR-STOR-0140 cites the silent set-mate test, and the matrix records its
mutation control. MR-STOR-0008 notes the set client's transport bounds.
Every node source is parsed with syn, reading paths, `use` trees and macro
bodies. Any clock or timer is refused: Instant, SystemTime, UNIX_EPOCH,
chrono, tokio::time. A comment or string never counts. The self-test covers
grouped, renamed and glob imports and a macro body; a first token-window
draft missed the grouped import.

Mutation controls, red and then restored:
- a grouped-import tokio::time sleep in the spool;
- Instant::now() in db/pg.rs.

ci/no_clock_and_no_json.sh loses two node exemptions: the spool, which
reads no clock, and a rate limiter that no longer exists. A clock read in
the spool now fails it too.

MR-STOR-0008 Partial → Met.
cryptskii added a commit that referenced this pull request Oct 1, 2026
…e SoFi PR

One PR for the three branches: the SoFi rig fixes, STORAGE's mirror
liveness and route-chain records (#1088), and CORE's ERA with two decimals
(#1087).

- MASTER: CORE's MR-SOFI-0349 beside MR-SOFI-0350–0359; §1 re-pinned to the
  merged SoFi specification (S16, S19 and S18); 928 canonical requirements.
- CONFORMANCE: MR-SOFI-0349 is Met. The SoFi routes take amounts as text in
  token units and report their display form, which was the part S18 left
  Partial. Totals regenerated.
- node tests: CORE's whole_era arithmetic, and the new SoFi tests' ERA
  balances moved to whole_era.
@cryptskii

Copy link
Copy Markdown
Collaborator Author

Included in #1089, which merged this branch's commits (and the trace fix 9dc6f84 for #1087) and repinned once at the combined head. One PR, one merge, at the owner's direction.

@cryptskii cryptskii closed this Oct 1, 2026
@cryptskii
cryptskii deleted the fix/storage-mirror-liveness-and-route-chain-records branch October 1, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant