Skip to content

test(storage): storage batch: reads, restores, arrival order, the route-chain writer, and nine rows re-verified - #1084

Merged
cryptskii merged 6 commits into
mainfrom
test/storage-batch-reads-restores-route-order
Sep 30, 2026
Merged

cryptskii merged 6 commits into
mainfrom
test/storage-batch-reads-restores-route-order

Conversation

@cryptskii

Copy link
Copy Markdown
Collaborator

One storage batch: tests and records only. No production code changes. Every row below was already satisfied by code on main. This PR either adds the tests that prove it, with mutation controls, or re-verifies the row against the specification and the current code.

On main the storage rows read Met 47, Partial 31. After this PR they read Met 60, Partial 18.

1. Newly proven: tests added in this PR

Each test was observed red under a mutation that performs what the requirement forbids, then restored.

Node side: dsm_storage_node/tests/reads_restores_and_arrival_order.rs, on the app the binary serves, each member on its own Postgres database.

Row Test Mutation control Status
MR-STOR-0117: reads cost the reader nothing a_read_asks_nothing_of_the_reader_and_moves_nothing Credit header demanded on the object read → red. A cell read recording an arrival → red. Partial → Met
MR-STOR-0120: any client restores a missing object, accepted by hash only a_client_restores_a_missing_object_accepted_by_hash_only Expected-address check skipped → red. Partial → Met
MR-STOR-0121: clients cannot reorder a cell's arrival log a_client_cannot_reorder_a_cells_arrival_log Cell read ordered by value → red. Stays Partial: handover and the loss rule are not built

Writer side: dsm_sdk::sdk::route_seats::tests, against the pinned nodes (the storage node's own code, on Postgres), with seats taken down and brought back.

Row Test Mutation control Status
MR-STOR-0148: nothing reaches a later seat before the leader answers nothing_reaches_a_later_seat_before_the_leader_answers Write going on past an unanswered leader → red. Partial → Met
MR-STOR-0151: an unanswered seat is recorded empty in its place, in route order a_seat_that_does_not_answer_is_recorded_empty_in_its_place Empty not recorded → red. Partial → Met
MR-STOR-0152: a recorded position is never written again a_recorded_position_is_never_written_again Continuing from the link count → red (position 2 rewritten). Partial → Met

The shared node-test helpers member, call and runtime move into tests/common. held_bytes_stay_held uses them in place of its own copies.

2. Re-verified as already satisfied on main: records only

These rows cited code that is gone: sofi/arith.rs (the superseded count rule), write_cell_leader_first, CellWrite, put_cell_leader_first and put_cells_leader_first. Or they described a mechanism since built as route chains. The mechanism was introduced before this PR, and this PR does not introduce it. None of these rows has a manifest row.

Row Now Rests on
MR-STOR-0046 Met dsm::route_chain::evaluate: Final is a leader link plus two links of one chain (Core's tests)
MR-STOR-0047 Met Only the first recognized value holds a leader link (Core's tests)
MR-STOR-0132 Met The leader link is the first recognized object; junk blocks nothing (Core's tests)
MR-STOR-0082 Met Each seat is durable before answering, and a write counts at three chained links (Core's route-chain evaluation)
MR-STOR-0108 Met A write goes through at three links; no payment exists (the spend gate is removed)
MR-STOR-0130 Met The writer's existing behaviour, now also shown by the section 1 writer tests
MR-STOR-0136 Met Links carried at once (section 1 writer test), counted once committed (Core's tests)
MR-STOR-0137 Met All five positions written, and a stopped chain continued (section 1 writer tests)
MR-STOR-0020 Stays Partial Live citations replace deleted ones. That Core uses nothing else from storage is not verified at every read site

Records

  • CONFORMANCE finding §6.50, §8 rows, and §7 regenerated by ci/conformance_evidence.py --write.
  • VERIFICATION_MATRIX: rows with their mutation results.
  • Pins: MR-STOR-0148, 0151 and 0152 have manifest rows whose status moves. They are repinned with --accept status from CI's map of this PR's head, in one round against the final main.

Evidence

…er holds (MR-STOR-0117, 0120, 0121)

reads_restores_and_arrival_order drives the app the binary serves, each
member on a Postgres database of its own:
- a_read_asks_nothing_of_the_reader_and_moves_nothing (MR-STOR-0117):
  seven reads (cell, index, object, spool, latest ByteCommit, cycle 1,
  commit proof) answer with nothing but their address. Three rounds answer
  the same, and the cycle closed afterwards is the one the member had.
- a_client_restores_a_missing_object_accepted_by_hash_only
  (MR-STOR-0120): a client restores A's object at B with no
  authorization, and B serves A's bytes. C refuses other bytes stated
  for that address.
- a_client_cannot_reorder_a_cells_arrival_log (MR-STOR-0121): values
  written again in another order are appended; the first entries keep
  their arrival records.

Mutation controls, each restored:
- the object read demanding a credit header: red;
- a cell read recording an arrival: red, the next cycle moves;
- the expected-address check skipped: red, other bytes are taken;
- the cell read ordered by value: red.

The shared helpers member, call and runtime move into tests/common, and
held_bytes_stay_held uses them in place of its own copies.

MR-STOR-0117 and 0120 are Met, and MR-STOR-0121 stays Partial: handover
and the loss rule are not built (§6.50, matrix rows, §7 regenerated).
None of the three has a manifest row. The node crate passes in release on
Postgres; fmt, clippy -D warnings and the real-code guard are clean.
…, never reopens a closed position (MR-STOR-0148, 0151, 0152)

Three route_seats tests against the network's pinned nodes (the storage
node's own code, on Postgres), with seats taken down and brought back:
- nothing_reaches_a_later_seat_before_the_leader_answers (MR-STOR-0148):
  with the leader down, the write stops at [NoResponse] and no later seat
  holds the value. With the leader back, every later copy carries the
  leader's record as its first link.
- a_seat_that_does_not_answer_is_recorded_empty_in_its_place
  (MR-STOR-0151): with position 2 down, the chain is links at 0, 1, 3 and 4
  and an empty at 2, and the copies at 3 and 4 carry exactly the slots
  before them.
- a_recorded_position_is_never_written_again (MR-STOR-0152): a write
  stops after recording an empty at 2, as a writer whose progress record
  fails. With that seat back it continues at 3 and 4, and position 2 is
  never written.

Mutation controls, each restored:
- the write going on past an unanswered leader: the first test red;
- the empty not recorded: the second and third tests red;
- continuing from the link count instead of the recorded position: the
  third test red.

MR-STOR-0148, 0151 and 0152 are Met (§6.50, matrix rows, §7 regenerated).
Their manifest rows move status only; they are repinned with --accept status
from CI's map when this branch opens its PR. route_seats' 9 tests pass in
release; fmt, clippy -D warnings (dsm_sdk, dsm_storage_node) and the
real-code guard are clean.
…d (MR-STOR-0020, 0046, 0047, 0108, 0130, 0132, 0136, 0137)

These rows cited code that is gone, or said no chain object exists:
sofi/arith.rs (the superseded count rule), write_cell_leader_first,
CellWrite, put_cell_leader_first and put_cells_leader_first. Each is
re-verified against the storage specification and main.
- Met on Core's route_chain::evaluate and its tests: MR-STOR-0046 (Final
  is a leader link plus two links of one chain), 0047 (only the first
  recognized value holds a leader link) and 0132 (junk first at the leader
  blocks nothing).
- Met on the writer's tests in this branch, together with Core's
  commit-before-count tests: MR-STOR-0130 (leader first, copies carry the
  chain, continued from the writer's own record), 0136 (links carried at
  once, counted once committed) and 0137 (all five positions written, a
  stopped chain continued).
- MR-STOR-0108 is Met: a write goes through at three links, and no payment
  exists.
- MR-STOR-0020 stays Partial with live citations: that Core uses nothing
  else from storage is not verified at every read site.

None has a manifest row. Core's route_chain and sofi::storage tests pass
in release (39/0); ci/conformance_evidence.py passes, and §7 is
regenerated (storage Met 59, Partial 19).
…ned links, each seat durable first

Each seat's durable write before answering was already tested
(durable_posture_tests). The clause left unbuilt, counting a write only
once its chain holds the leader's link and two further links, each
committing the one before (G2), is now Core's route-chain evaluation
(only_links_of_one_chain_count_toward_final,
a_copy_whose_carried_links_are_not_one_chain_does_not_count). No manifest
row. §6.50 extended; §7 regenerated (storage Met 60, Partial 18). The
durable-posture tests pass in release on Postgres.
…ads-restores-route-order

# Conflicts:
#	specs/requirements/CONFORMANCE_GAPS.md
This branch moves MR-STOR-0148, 0151 and 0152 from Partial to Met in §8.
Their four manifest rows read PIN_STALE (status) in CI's code map of
7981f41 (run 36789718990), whose tree fingerprint equals this tree's.
Each is repinned with --accept status: the move is this branch's
deliberate §8 change. Their manifest evidence, four route_chain tests,
passed here in release at that tree. The comparator reads 598 of 598
pins PINNED and 0 failing rows.
@cryptskii
cryptskii merged commit 62da367 into main Sep 30, 2026
24 checks passed
cryptskii added a commit that referenced this pull request Oct 1, 2026
The merge with #1084 took main's pins for MR-STOR-0146, and this
branch's changes move both rows' closures. Taken from CI's code map of
61b776a. Their 3 evidence tests passed at that commit. `make
requirement-map-intent` against that map reads 1,111 rows, 0 failing,
and 598 pins, 0 failing.
cryptskii added a commit that referenced this pull request Oct 1, 2026
…(A9), and the DSM core row sweep (#1083)

* test(core): the receiving device refuses a non-transferable token, whatever its sender checked

a_non_transferable_token_refuses_its_transfer now also drives B's own
canonical apply (apply_incoming_transfer_staged) with a transfer A
signed, of the non-transferable token B adopted, from B's pinned head
for A. It is refused by the token's operation restriction before any
acceptance is built.

A probe first showed this holds end to end: with the sender's check
removed, B's sync refused the transfer ("Token policy violation ...
Operation not permitted") and nothing was credited. The CONFORMANCE §5A
note that the recipient never checks was wrong.

Mutation: the policy check skipped only for transfers addressed to this
device turns the test red with "reached acceptance: the policy did not
refuse".

* docs(core): nine DSM core rows about SoFi behaviour re-verified now that SoFi is reachable

MR-DSM-0205, 0209, 0211, 0212, 0213 and 0214 are Met on the SoFi
end-to-end tests and the Core resolution tests. MR-DSM-0219, 0265 and
0267 stay Partial, their gaps restated. All nine had recorded SoFi as
unreachable, which has not been true since #1056 and #1064.

* test(core): a send to a device that is not a contact moves nothing

MR-DSM-0074, Amendment A3: the sender sends only over relationships it
has pre-added. wallet.sendSmart to a device A never added is refused
with "recipient must be an added contact before online send"; nothing
is debited, no position is admitted and nothing is left pending. The
check is structural (the send is built from the contact record's keys),
so it has no mutation control that leaves the send buildable.

* docs(core): MR-DSM-0039, 0074, 0079 and 0093 re-verified, all Met

- 0039: acceptance needs a claim final at the payer's next root cell.
- 0074: both ends only use pre-added relationships (new sender-side
  test).
- 0079: Core counts a link only once a ByteCommit following its parent
  commits it.
- 0093: only the named counterparty takes a step.

* fix(online): a transfer's token policy is checked before the sender's register is read

MR-DSM-0029, G13: the receiver reads the payer's register only after
every check it can decide from what it holds. The sync prevalidated each
bound pair, which walks the sender's lineage over the network, and only
the apply then checked the token's committed policy, which this device
holds. The policy check now runs first, before prevalidation. The apply
keeps its own check under the state-machine lock. Test to follow.

* test(online): a transfer its policy refuses is refused before the sender's register is read

MR-DSM-0029, G13, for f1723e3. A hostile sender signs a transfer of
its non-transferable token to B and advances its own head over it. It
then signs the step's receipt with its per-step EK, as its wallet signs
a send's receipt. Both halves reach B's boundary and bind. B's sync
refuses the pair by the token's committed policy, and no member is asked
for a cell while it does.

Mutation: remove the in-hand policy check and the test goes red. B asks
dsm-node-1 for a register cell of the sender before it refuses.

The non-transferable token request moves into a helper, which the new
test and a_non_transferable_token_refuses_its_transfer share.

* docs(core): §6.50 records the batch; MR-DSM-0029 and MR-SOFI-0311 re-verified

- §6.50 records the batch: the receiver's policy check before any read,
  the receiving device's refusal of a non-transferable token, the
  contact check, and the fifteen rows re-verified on this branch.
- MR-DSM-0029 now cites the sync's in-hand policy check and its
  mutation-controlled test. It stays Partial: adoption and the
  relationship tip are still decided after the register read.
- MR-SOFI-0311 and the §5A Transferable check row no longer say the
  recipient never checks. Both stay open: offline transfers are outside
  this round, and vault creation and SoFi legs are tested in Core only.
- VERIFICATION_MATRIX gains both gates, each with its mutation control.

* docs(spec): the relationship-key tag is DSM/smt-key (DSM Amendment A9)

Owner ruling, 2026-09-30. DSM/smt-key is the canonical domain tag for
relationship SMT keys in beta. The /v1 the explainer carried in §26's
formula and §16's example was an error. The code and its golden vector
are unchanged, and no key migrates.

- Explainer: §26's formula and §16's example corrected, with Amendment
  A9 after §26.
- MASTER: §1 re-pinned, a §7.1 entry, and MR-DSM-0115 rewritten.
- CONFORMANCE: MR-DSM-0115 and MR-DSM-0249 go Partial → Met; §6.14's
  finding is marked resolved, and §6.50 records it. DSM core totals:
  90 Met, 96 Partial, 39 Missing, 0 Violated.

* style(core): rustfmt the sender admission tests

* chore(pins): repin the 71 rows #1083 moves

- 66 code-class rows, whose closures reach storage_routes.rs,
  core_sdk.rs and the sender admission tests.
- 5 status-class rows, which this batch moved Partial → Met and accepts
  with `--accept status`: MR-DSM-0115 (two symbols), MR-DSM-0209,
  MR-DSM-0212 and MR-DSM-0249.

All taken from CI's code map of 0d1228f. Their 59 evidence tests ran
and passed at that commit. `make requirement-map-intent` against that
map reads 1,111 rows, 0 failing, and 598 pins, 0 failing.

* chore(pins): repin MR-STOR-0146's two rows after merging main

The merge with #1084 took main's pins for MR-STOR-0146, and this
branch's changes move both rows' closures. Taken from CI's code map of
61b776a. Their 3 evidence tests passed at that commit. `make
requirement-map-intent` against that map reads 1,111 rows, 0 failing,
and 598 pins, 0 failing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant