Repository navigation
ci: keep the code map's indexes for pin-only and prose-only commits; cancel superseded PR runs - #1116
Conversation
… or prose; cancel superseded PR runs Code map: about seventeen of its twenty minutes (the fixture run, the three rust-analyzer indexes, the mutation cases) read code, configuration and the workflow, never a pin or prose. That half is now kept in the Actions cache under a key of every tracked file except *.md and specs/requirements/INTENT_PINS.tsv, plus the runner image. The key must match exactly; there are no restore-keys. A commit that re-pins or edits only prose restores the indexes and fixture run of the commit with the same key, where they were verified, and skips that half. Everything that reads the changed files runs on every commit: the map's tables (from the indexes and the intent manifest's root questions), its check, the intent manifest and every pin, and the pins' own tests. A map is still refused unless its fingerprint is this tree's. The cache is saved only by a run that built the indexes and passed the fixture, the map, its check and every mutation case; the intent manifest and the pins need not hold, since the commit re-pinning the rows a run left stale is the one that reuses it. Makefile: requirement-map is now requirement-map-indexes then requirement-map-tables. The recipe lines are unchanged and make requirement-map does what it did. CI, CodeQL, Code map: a newer push to the same pull request cancels the older run. Pushes to main, schedules and dispatches are their own groups and are never cancelled.
The first cached run failed in requirement-map-tables: "dsm_sdk has a build script but cargo left no output for it under ./target: the index did not run it". The map reads each shipped crate's build-script `output` (what it printed to cargo) to refuse one that sets a cfg; the indexes' cargo runs leave them under target/debug/build and target/<triple>/debug/build, and rust-cache keeps no workspace crate's. They are kept with the indexes now. Nothing else under ./target is read: the map's tables name no definition there.
|
Both paths are now validated on
On the hit, the log reads The first cached attempt (run 37295619885, attempt 2) failed in the tables step: For comparison, Code map on recent PRs took 19.9 min (#1115) and 19.7 min (#1114). |
The Code map job is the long pole: about 20 minutes on every push, including commits that only re-pin. This PR keeps its expensive half between commits whose code and configuration are the same, and cancels pull-request runs that a newer push has superseded.
Code map: keep the indexes between pin-only and prose-only commits
Where the job's time goes (run 37275560695):
The fixture, the indexes and the mutation cases read only code, configuration and this workflow. They never read a pin or prose. Checked against the sources:
*.rs/*.kt/*.ts/*.proto/…, the rust-analyzer configs,Cargo.lock/Cargo.tomland the toolchain.ci/*.tsvand the real map.INTENT_MANIFEST.tsvthrough its root queries, so the manifest stays in the key.That half is now kept in the Actions cache:
*.mdandspecs/requirements/INTENT_PINS.tsv, plus the runner image (ImageOS,ImageVersion), saltedcode-map-v1. The key must match exactly: there are norestore-keys, so a near match is a miss.intent_pins.fresh()still refuses a map whose fingerprint is not this tree's.Why the key is wider than the map's own fingerprint: the fingerprint covers sources plus a few configs, but not
Cargo.tomlfeatures, theMakefile, theci/*.pyscripts or the workflow. That never mattered while every run rebuilt. As a cache key it would let a features-only change reuse a map built under different features.The key was checked locally. Changing
INTENT_PINS.tsvorCONFORMANCE_GAPS.mdleaves it the same. ChangingINTENT_MANIFEST.tsv,Cargo.toml,dsm/src/lib.rsorcode-map.ymlchanges it.Makefile:requirement-mapis nowrequirement-map-indexesfollowed byrequirement-map-tables. Every recipe line is unchanged, andmake requirement-mapdoes what it did (dry-run checked).Cancel superseded runs
CI, CodeQL and Code map now have a
concurrencygroup per pull request withcancel-in-progressforpull_requestevents, so a newer push cancels the older run. Pushes tomain, schedules and dispatches group by commit SHA and are never cancelled or queued: each merge commit keeps its own result.Validation
This PR's first Code map run misses the cache by construction. It runs the split targets end to end and saves. Re-running that job on the same commit hits the same key and exercises the cached path. Both results will be posted here.
Not in this PR
A self-hosted runner, which would cut the cold setup (toolchain, NDK, cargo cache).