[diillson]

What

Publishes two full LongMemEval-S runs after the recall changes (#1869, #1870, #1871): the run made on the branch before merging, and the definitive run on the merged main (6ece9b07). Results, summaries, the failure transcripts of the misses, the per-question development/validation split, and the harness as it was run. The README shows base, pre-merge and merged side by side.

Numbers (merged main)

• 473/500 = 94.6% (95% CI 92.3–96.3%), against 454/500 = 90.8% on 2026-10-20.
• Validation half, 260 questions never read: 243 = 93.5% (89.8–95.9%). The base run scored 237 on the same 260.
• Against the base run on the same 500: 27 gained, 8 lost (sign test p = 0.002). Against the pre-merge run: 9 gained, 8 lost (p = 1.0).
• Worse: abstention 27 → 25 of 30; cost per question US$0.087 → US$0.101; input tokens 71.3k → 77.6k. Tool calls per question 4.9 → 5.0.

The root READMEs (en, pt) quote the validation figure.

Checks

Results packs swept for credentials, tokens, e-mail addresses and local paths (none; paths are <scratch> and ~). Figures in the README were read from summary.json / summary-validation.json and from per-question comparisons of the three result files.

[github-actions]

Quality Gate

Result: ✅ all floors passed

Floor	Status	Result	Δ vs main	Budget
1 · Build & Static	✅	go build / vet / fmt / lint	—	—
2 · Coverage	✅	69.6% (bootstrap)	0	≥ baseline
3 · Patch coverage	⏭️	100% (req ≥ 60%)	—	≥ 60%
4 · AI smells	⏭️	diff scanned	—	—
5 · Scope budget	⚠️	12 files / 1450 LOC (code 1302 + tooling 148)	—	warn 800·25
6 · E2E	⏭️	go test -race ./e2e/...	—	≤ 15min
7 · Commit lint	✅	conventional commits	—	—
8 · Cyclo (new code)	⏭️	—	—	≤ 30
9 · Secrets scan	✅	gitleaks	—	—
10 · i18n parity	✅	missing 0, unknown 0	—	—
11 · CRD drift	✅	drifted: 0	—	—
12 · License headers	⏭️	0 missing	—	—
13 · API breaking	⏭️	0 incompatible	—	—
14 · Binary size	⏭️	—	—	100MB each
15 · Provider parity	✅	14 providers · 0 violations	—	—

Config: .github/quality-gate.yml. Workflow: .github/workflows/quality-gate.yml.