Repository navigation
feat(map): Stage 2, evidence pins and an always-run merge gate - #1057
Merged
Merged
Conversation
Stage 2 pins will rest on the map's closure hashes: a row's production closure and the closure of each test its evidence names. A closure must change whenever anything its code depends on changes, or a pin would stay fresh over changed behaviour. It did not. A workspace symbol the index has no definition of was hashed by its name alone: - prost output such as `types/proto/...` and `generated/prost_generated/...`; - a macro's items, such as `impl#[NodeId]new()`. Of the 2,181 such symbols, nearly all are prost types. A `.proto` edit, a build-script change or a macro change would leave every closure that reaches them unchanged. Now such a symbol is hashed under `DSM/code-generated/v1` with its symbol and the digest of what generates it (`DSM/code-generation-inputs/v1`): - its package's `build.rs`; - the module file its path names, or the nearest fingerprinted one enclosing it (where its `include!` or macro invocation sits); - every `.proto` file; - every macro definition. Nothing else in its package is bound, so a change elsewhere leaves it alone and does not stale every pin that touches a prost type. Every one of the 2,181 resolves to a module file. The map refuses: - one that does not; - a package whose definitions sit under two source directories. The fixture binds its own crate's files the same way. Test: generated_code_is_bound_to_what_generates_it. - The hash changes with the build script, the module file, a `.proto` file or a macro definition. - It does not change with an unrelated file or `lib.rs`. - Another package's symbol is not generated code. - A symbol no module file encloses stops the map. Mutation: with the `.proto` files dropped from the binding, the test goes red on exactly that assertion. The rebuilt map keeps every committed fact: sentinels, entry points, counts and the intent outcomes.
…epin The owner's Stage 2 rulings (2026-09-29). What a pin is. - ci/intent_pins.py keeps specs/requirements/INTENT_PINS.tsv beside the manifest. The manifest says what should be true; each pin is the evidence snapshot someone verified for one (requirement, symbol, artifact). - A pin stores its facts explicitly: - the manifest cells, and their digest (`DSM/intent-manifest-row/v1`); - the §8 status, the comparator's outcome and the map's reading; - the definition the path resolves to, and its closure; - the closure of every test its evidence names; - the commit that evidence passed at; - the digest over all of it (`DSM/code-row-evidence/v1`). - Every requirement row is pinned, known holes included. `-` rows are not. Check. `ci/intent_comparator.py --pins` reads every pin, after the outcomes, and fails a row whose: - facts moved (PIN_STALE), naming each class with the fact's old and new value; - stored digests do not match its facts (PIN_TAMPERED); - requirement row has no pin (UNPINNED); - key the manifest no longer has (ORPHAN_PIN). Pins are read only over a map of every build, because a closure is taken over every build's edges. On a host that cannot index the storage node, the comparator reports "not checked" and fails, rather than reading every pin wrongly. The gate reruns no test. A pin binds the production closure and the evidence tests' closures, so a change to either stales the row, and only a board run on the changed tree refreshes it. Commands (one key at a time): - `pin KEY` creates a pin. - `repin KEY` accepts a code refresh. Every other class that moved must be named with `--accept`: symbol, reading, outcome, status, root, reachability, evidence, citation or exception. An `--accept` for a class that did not move is refused. - `unpin KEY` removes only a pin whose key has left the manifest. - `pin-unpinned` is the one-time bootstrap: it creates pins only where none exist and never touches one that does. Before any pin is written, each command refuses unless: - the map is of this tree (its recorded fingerprint, recomputed); - this tree's code is the tested commit's (`conformance_evidence.code_at`, lifted out of its `main`); - the map resolves every evidence test to one definition (a test it cannot read is `absent:NAME` or `ambiguous:NAME`, a moved fact to the check); - every evidence test ran and passed in the given board logs. `intent_comparator.evaluate` returns a named `Evaluated` for both scripts. A refused pin check goes to stderr and fails the run. Tests: ci/test_intent_pins.py (11), over the fixture's map. - Each state is planted in copies of the fixture's manifest, requirements and pins, and asserted with its class. - Also covered: the repin and bootstrap rules, board evidence (ok, failed, crashed, not run, no such test), and test names (library, and integration `crate::stem::name` over this repository's map). - Mutations: with tamper detection removed, a hand edit reads as stale; with `status` made implicit, the repin refusal goes. Both go red. rules.tsv gains a `pin` kind. rules.rs reads PIN_STATES from Python and checks that each state's unit tests exist in ci/test_intent_pins.py. Dropping a state's row fails. The requirement_map binary gains `digest --domain manifest-row|row-evidence`, so every pin hash is a DSM domain-separated BLAKE3. Test: each_line_digests_its_cells_under_its_domain. Makefile and CI: - `requirement-map-intent` passes `--pins`. - `requirement-map-pin-tests` runs the tests in the Code map job. - The code-map artifact now carries the tree record, the source and input lists, the analyzer version and pins.tsv, so pins can be taken from CI's map. Until the bootstrap chunk commits INTENT_PINS.tsv, the intent step fails with "not checked: … does not exist". That is the truthful state.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
…s evidence The merge gate moves out of ci.yml into its own workflow, .github/workflows/code-map.yml. It runs on every pull request, every push to main, and by hand. It has no path filter: ci.yml ignores markdown-only PRs, yet a CONFORMANCE §8 edit alone moves a row's status and so its pin. Its jobs have no layer condition either. A required check skipped by a path filter would read as passed. Job "Code map": the map, its check, the intent manifest with every pin (INTENT_BUILT=android,node), the pins' tests and the mutation cases, moved unchanged from ci.yml. Job "Pin evidence": the half of the gate that runs tests. - `ci/intent_pins.py evidence --base REF` runs, on this commit with Postgres, the evidence tests of every requirement row whose pin is new, changed or missing against the base. The base is the PR's base, or the previous main. - It runs exactly those tests (`cargo test -p CRATE --lib|--test STEM|--bins -- --exact`, the board's flags). It fails on any non-zero cargo exit and on any test that did not pass. - So a pin is evidence CI proved, never a log only its author saw. - Where no pins are committed yet, it runs every row's evidence, and that log is what the one-time bootstrap pins from. - Checked end to end locally: a one-row run compiled and ran exactly its test and passed. Found and fixed on the way (gate rounds 5 to 8): - `pins_at` reads a base's pins only once the ref is a commit and the path is listed there. Every other git failure is refused with git's message, never read as "no pins". - `conformance_evidence.code_at` reads `git diff --quiet` exit 1 as a difference and refuses any other failure. - The evidence test reads real captured board logs (ci/fixtures/board-logs): a pass, a run whose filter matched nothing, a node-backed test failing without its database, and a run killed mid-test. Each is a whole `cargo test` run with the gate's flags. - The generated-code binding test and the digest test read real files and a real symbol: the fixture crate's files, the one generated symbol its index holds, the repository's `.proto`, and the fixture manifest's rows. The proto mutation still turns the binding test red on `proto/dsm_app.proto`. Tests: - ci/test_intent_pins.py: 14, including the rows the gate verifies, the exact cargo runs, and base pins read from history. - `cargo test -p requirement_map`: 99. - `make lint`: 0. - scripts/check-spdx.sh: OK.
From review (github-code-quality): `row_line` mixed an explicit return with a fall-through. It never actually returned None, because the fall-through path called `self.fail`, which raises, but the shape read as one that could. It now collects every manifest line with the key, asserts there is exactly one, and returns it. That is stricter than before: a key naming two rows now fails instead of returning the first. ci/test_intent_pins.py: 14/14.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 2 of the code map: evidence pins on the intent manifest, a strict per-row
--repin, and a merge gate that runs on every PR. This is one PR, built in gated chunks. It stays a draft until the last chunk lands.Owner rulings (2026-09-29)
specs/requirements/INTENT_PINS.tsv, keyed by (requirement, symbol, artifact). The manifest stays pure intent; the pins file is the approved evidence snapshot.tested_at, and the digest over all of them.-) carry no pin.--repinrequires the named tests to pass on the current commit.--pin-unpinned, creates pins only where none exist. It uses CI's map and full board log of one commit. After it, only--pin KEYand--repin KEYrun, one key at a time.--repinaccepts code refreshes only. Any semantic change needs--accept <class>: symbol, artifact, reachability or lifecycle, root, status, outcome, evidence, citation, or manifest row.**/*.mdPRs skip the map entirely. Branch protection comes only after owner approval.Chunk 1: closures bind generated code to what generates it
Pins rest on closure hashes. A workspace symbol the index has no definition of, such as prost output or a macro's items, was hashed by name only. So a
.proto, build-script or macro change left every closure reaching it unchanged.Now such a symbol is bound to:
build.rs;.protofile;Nothing else in its package is bound, so unrelated edits don't stale pins. All 2,181 such symbols resolve to a module file. The map refuses a symbol that does not, and a package whose definitions sit under two source directories.
generated_code_is_bound_to_what_generates_it..protofiles dropped from the binding, it goes red on that assertion.make lint0,requirement_map98 of 98.Next chunks
INTENT_PINS.tsv, pin facts, PIN_STALE and orphan or tamper checks inci/intent_comparator.py, with a Rust digest underDSM/code-row-evidence/v1.--pin KEYand--repin KEY [--accept ...], with board-log and tested-at checks.--pin-unpinnedbootstrap from CI artifacts.