feat(bench): add scoreable persistent SWE arm

[drewstone]

What changed

• adds typed independent-2 and persistent-refine-2 SWE-bench experiment arms with exactly two completed worker sessions per task
• runs every worker through runAgentic(refine, budget=1) while isolating only the missing experiment policy: later-on-tie selection, always-on parent-patch continuation, and per-session patch receipts
• separates model generation from MODE=judge-only official scoring and requires both complete Phase-A artifacts before the first score
• binds resume and scoring to source, runtime, scorer, prompt, task, reproduction, Docker image, parent patch, and final patch fingerprints
• preserves the current SWE adapter's fail-closed report validation, timeout, and artifact-capture behavior while adding explicit scorer namespace pinning
• adds the run-capable SWE surface, immutable image resolution, execution jail, and regression coverage for continuation leak separation
• separates model TEMPERATURE from Node's TEMP filesystem root and rejects relative temp roots before any experiment dispatch
• preserves repository-relative symlink targets in cached checkout copies and rejects any copied worktree that is not git-clean before worker access

Why

This makes the breadth-versus-persistent-state comparison scoreable on SWE-bench without hidden-test feedback entering generation, unequal completed-session counts, mutable image drift, or incomplete scorer rows being reported as model outcomes.

Validation

• 38/38 focused structural, environment, and SWE adapter tests pass
• package suite passes: 25/25 TypeScript test files plus the Pier bridge
• targeted TypeScript compilation passes
• root package build passes
• judge-only startup fails closed before scoring when Phase-A inputs are absent
• git diff --check origin/main...HEAD passes
• git merge-tree --write-tree origin/main HEAD succeeds

No model calls or official SWE-bench scoring calls were made while preparing this change.

[tangletools]

✅ Auto-approved drewstone PR — aed83b19

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-17T14:42:10Z}

[tangletools]

🟡 Value Audit — sound-with-nits

Verdict	sound-with-nits
Concerns	4 (2 low, 2 weak-concern)
Heuristic	0.2s
Duplication	0.0s
Interrogation	236.8s (2 bridge agents)
Total	237.0s

💰 Value — sound-with-nits

Adds a tightly-scoped SWE-bench experiment harness (independent-2 vs persistent-refine-2 arms) with explicit provenance fingerprints, generation/judge separation, and immutable image pinning — built cleanly at the runtime's documented complete seam; two minor duplication/comment nicks do not gate

• What it does: Adds a MODE=generate / MODE=judge-only SWE-bench experiment driver (bench/src/swe-structural.mts, ~1260 lines) that runs exactly two completed worker sessions per task under two typed arms — independent-2 (two fresh attempts) and persistent-refine-2 (one attempt plus one continuation with the parent patch pre-applied). Both arms use runAgentic(refine, budget=1) per session with a later
• Goals it achieves: Make the breadth-versus-persistent-state comparison on SWE-bench scoreable without four specific threats to validity the prior setup had: (1) hidden-test feedback leaking into generation, (2) unequal completed-session counts being reported as model outcomes, (3) mutable Docker image drift between generate and score, (4) incomplete scorer rows being serialized as model outcomes. The pure policy m
• Assessment: Sound and in the grain of the codebase. The driver correctly composes per-session worker calls through the runtime's documented complete transport seam (src/runtime/strategy.ts:94, src/runtime/router-client.ts:27) rather than bypassing it — the new zaiChatRaw is a transport implementation at that seam, with a leak-guard chokepoint that refuses any outbound message carrying gold/test-patch
• Better / existing approach: No materially better architecture — the controlled-comparison shape (exact-completed-session-count invariant, paired fingerprint admission, generation/judge phase split) is genuinely absent from the built-in strategy LEGO and from composeStrategies (which only prefixes diverse system-prompt lenses per rollout; bench/src/clbench-codebase-gate.mts:212, bench/src/aec-gate.mts:24). The pure modu
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 2
• Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content

🎯 Usefulness — sound

A coherent two-arm SWE-bench experiment harness with split generation/scoring and immutable-image provenance, wired in via the established env-var driver-script pattern and layered outside (not competing with) runStrategyEvolution.

• Integration: Fully reachable: swe-structural.mts:1257 is a runnable tsx driver (MODE/EXPERIMENT_ARM/INDEPENDENT_PHASE_A env vars), matching the established swe-self-improve.mts:8 pattern. All new exports have live callers — createSweBenchEnvironment extensions consumed at swe-structural.mts:978-984, swe-bench.ts namespace plumbed via process.env.SWEBENCH_NAMESPACE set at swe-structural.mts:1228, jail/provenanc
• Fit with existing patterns: Matches the codebase grain exactly. Author justifies the out-of-runStrategyEvolution layering at swe-structural.mts:436-441 (built-ins expose only aggregate scores, no per-session patch receipts, refine may stop after shot one). swe-jail.ts consolidates shared primitives from the removed swe-repro-calibrate.mts rather than forking. No competing equivalent exists.
• Real-world viability: Fail-loud + resume-safe design: each Phase-A row validated (assertPhaseRow) and appended, reload re-verifies fingerprints (swe-structural.mts:1036-1042); concurrency clamped to [1,4] with serial per-instance work; error paths for canary/repro/image/apply/timeout/empty/infra all explicit; immutable image pinned via docker inspect ID+repoDigests and re-checked before every official score; leak guard
• Model: opencode/zai-coding-plan/glm-5.2
• Bridge attempts: 1

🔎 Heuristic Signals

🟡 Cruft: console debug added bench/src/swe-structural.mts

• console.log(═══ SWE-bench Phase A — ${preset.arm} ═══)

🟡 Cruft: todo added bench/src/swe-structural.mts

• const todo = ids.filter((id) => !done.has(id))

💰 Value Audit

🟡 Reimplemented canonicalize/canonicalJson/fingerprint vs substrate primitives [duplication] ``

bench/src/swe-structural-provenance.ts:24-38,67-69 reimplements canonicalize + canonicalJson + fingerprint (sha256-of-canonical-JSON with sha256: prefix) from scratch. The codebase already has equivalent primitives the bench package directly depends on: canonicalJson and hashContent exported from @tangle-network/agent-eval (already imported in bench/src/pier-agent.ts:19 and bench/src/corpus.ts:17), and the identical sha256: + canonical-JSON pattern at `src/candidate-executi

🟡 swe-jail.ts comment references a source file that does not exist [maintenance] ``

bench/src/swe-jail.ts:2-5 and :234 claim the code was 'extracted from swe-repro-calibrate.mts' and that 'the calibrator now imports these.' git log --all -S 'swe-repro-calibrate' returns only this PR's commit; no such file exists on main, any remote, or any branch in this checkout, and nothing in this PR imports swe-jail.ts other than swe-structural.mts and the policy test. Either the calibrator lives on an unmerged branch (forward reference) or the extraction provenance is misstated

---

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass	What it asks
Heuristic	Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication	Do added function/class names already exist elsewhere in the repo?
Value Audit	What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit	Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

_{value-audit · 20260717T144930Z}

[tangletools]

✅ Auto-approved drewstone PR — 11fb4034

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

_{tangletools · auto-approval · reason: drewstone_author · 2026-07-17T14:53:08Z}