fix(daily-cache): stop a frozen under-read from outranking the sources on disk

[therickfactr]

Summary

• A day is derived into the durable cache once and then frozen behind lastComputedDate, while isPartialSurvival deliberately lets a fresh derivation shrink a day inside the settle window. Together, one parse that missed sources becomes permanent — my 7 Days Overview headline read 27% below the sum of the Daily Activity rows printed directly under it (27,346 calls vs 32,608), on five days whose transcripts were intact the whole time.
• The read path already parses the dates it reports on, so it now reconciles the two derivations per (date, provider) and keeps whichever explains more calls. Carry-forward is untouched — nothing live can outbid a day with no surviving source — and ties go to the cache, so a re-pricing still lands only through the write path.
• The write path is unchanged: what to freeze stays its decision, a still-settling day is still defined by its fresh parse, and nothing re-parses.

Fixes #1217.

Testing

• I have tested this locally against real data (not just unit tests)
• npm test passes — 1 failed | 3,500 passed, and that one failure is cli-budget > keeps overview budget lines only on unfiltered overviews, which fails identically on main (1 failed | 3,494 passed). It is the pre-noon-UTC seeding flake test(budget): seed current-month spend in the past, not at noon UTC #1216 fixes; CI here ran at 08:05 UTC and hit the same one. Zero new failures.
• npm run build succeeds

Also run: npx tsc --noEmit clean, and npm run test:locks matching main (1 failed | 35 passed, the known cache-refresh-lock heartbeat flake, same test both sides).

Against real data, on the machine that hit this: the headline now matches a fresh parse day for day, and the includes $… preserved from expired session logs footnote — which had been naming exactly the five corrupted days — drops to $0.00.

before   $6,415.71   27,346 calls    includes $4,108.78 preserved from expired session logs
after    $8,880.38   34,527 calls    (no carried footnote)

Detail

Evidence it is the cache, not the parser. daily-cache.v17.json, still on disk from before the upgrade, holds the correct figures (2026-08-26: 2,429 calls; the v29 row says 892). A single-day parse and a 6-month parse of those dates both return the full numbers today. The under-read itself was transient and I could not reproduce it — that limit is written up in #1217. This PR is about the other half: that a transient under-read becomes permanent state with no path back.

Why not fix it on the write side. I first tried re-deriving the unsettled tail on each run (gapStart = min(watermark + 1, today - SETTLE_DAYS)). That breaks six tests which deliberately guarantee the opposite — does not recompute yesterday after it has already been cached, trusts a stamped watermark over an idle tail — no re-derive treadmill, and four more. Those tests are right, so I left them alone; I flagged the change of approach on the issue before rewriting.

Cost. No new parse and no new file read. buildDurablePeriod aggregates a parse it already performed, buildDurableOverviewFromNormalizedIndex reuses normalizedDays it already computes, and when cache and parse agree the merge finds nothing to swap. A partial index can never lower a total under this rule, since a thinner slice never wins — which matters for the progressive-startup path, where the index is deliberately incomplete early.

Implementation. The reconcile is mergeDayEntries's existing partial-survival machinery under a new 'prefer-richer' mode, so slice/day totals stay reconciled by the code that already does that.

tests/durable-underread-reconcile.test.ts fails 4/6 against main and passes 6/6 here. It builds a real session fixture on a past date plus an under-reading cache row for it, then asserts the durable headline matches the live parse — along with the three cases that must not change: a genuinely expired day still carries, a richer cached slice is not dragged down to a partial live one, and an equal-call re-pricing leaves the durable value alone.

Note for anyone already affected: this stops the symptom wherever a live parse happens, but a cache that already froze an under-read stays wrong on disk. Setting "complete": false in ~/.cache/codeburn/daily-cache.v<N>.json and running once repairs it (verified: all five days healed exactly). Deleting the file instead is not equivalent — MIN_SUPPORTED_VERSION makes older generations unadoptable, so a delete drops every carried day whose sources have since expired.

[therickfactr]

Both test legs are red on tests/cli-budget.test.ts > keeps overview budget lines only on unfiltered overviews — expected 'CodeBurn September 2026\n\nNo usage …' to contain 'Monthly budget:'. That is the pre-noon-UTC seeding flake #1216 describes, not this change: CI ran at 08:05 UTC, inside the window, and it is the same single failure I get on main locally. Everything else in both legs passes (258/259 files), the new suite included.

test (22.13.0) · test (22)

Local gate for the record — npm test on main: 1 failed | 3,494 passed; on this branch: 1 failed | 3,500 passed, same test. Happy to rebase once #1216 lands so this goes fully green.

[therickfactr]

Retitled to the fix(daily-cache): convention and rewrote the body against .github/PULL_REQUEST_TEMPLATE.md. The force-push carried no code change — efa06184 and f2e82139 have the same tree, b3715ed389a99657f222c696c1ac07fb57a4e5a9, so only the commit subject moved.

That makes the two CI runs a clean A/B on identical content, and they failed differently each time:

run	cli-budget	Cache-lock suite (serial)
08:05 UTC	fail (pre-noon-UTC seeding, #1216)	pass
15:19 UTC	pass	timed out at 5m

Same tree, disjoint failures — both are flakes, neither is this change. In the second run the parallel suite itself passed whole (261 files, 2 skipped, my new suite included) and the job died in the serial lock step: The action 'Cache-lock suite (serial)' has timed out after 5 minutes. That is the fence never loses to its own heartbeat (in-process serialization) burning its 10 retries — it takes ~330s on my machine against main too, which is already over the step's 5-minute budget. Looks like the #904 / #1196 neighbourhood.

This PR touches src/daily-cache.ts, src/usage-aggregator.ts and one new test; the lock path is src/parser.ts / src/session-cache.ts, untouched. I don't have re-run rights here — a maintainer re-running test (22) should settle it.

[ozymandiashh]

The bug is real and the layer is right: a day frozen behind lastComputedDate while isPartialSurvival lets a fresh parse shrink it inside the settle window, so one transient under-read becomes permanent. Reconciling per (date, provider) against the parse the read path already ran is the correct shape, and carry-forward is untouched. I resolved the src/usage-aggregator.ts conflict locally and the merged tree is green (tsc clean, 129/129 across the 11 related suites).

One blocker before merge:

markSecondaryCarried: true marks healthy days as carried. In unionDaysForPeriod (src/usage-aggregator.ts:453) the call is mergeDayEntries(liveForCachedDates, historicalDays, true, undefined, 'prefer-richer'). With 'prefer-richer', baselineExplainsMore returns true on fresh.calls <= baseline.calls, so the tie case, which is every normal day inside retention, goes through the swap branch and sets existing.carried = true. carriedCostUSD then sweeps up the whole period and Overview / dashboard / menubar print includes $X preserved from expired session logs for money whose transcripts are on disk. Reproduced with a cache row carrying the same 4 calls the live parse finds: merge-base carriedCostUSD 0, this branch 3.16 of a 3.16 day. Same root cause hits the provider-filtered path through the !existing branch.

Fix: pass false, matching what overlayProviderDaySlices does on main for the same read-path purpose (3e48f8d0). Please add a case to tests/durable-underread-reconcile.test.ts where cache calls == live calls and carriedCostUSD === 0; the current suite only pins the strict-live-wins case, which is why 3,500 green tests miss this.

Rebase notes: the import line takes main's verbatim (main already imports mergeDayEntries); in buildDurablePeriod your liveHistoricalDays line and the sixth argument to unionDaysForPeriod need to be combined with main's overlayProviderDaySlices(allDays, freshDaysInSelection, pf) routing, not chosen over it.

Non-blocking, for the PR description or a follow-up: this is read-path only, so the corrupt row stays on disk and the 365-day history / heatmap in the menubar payload still come from raw cache days while the headline in the same payload is reconciled. A write-back of the richer day (data already in hand, no extra parse) would close that. Also worth noting the extra aggregateProjectsIntoDays + structuredClone-per-day merge runs on every buildDurablePeriod, twice per status.

[iamtoruk]

Landed as #1304 (cfa0f9f) with your commit and authorship intact, rebased onto current main, plus two follow-ups found during real-data verification: the reconcile no longer rewrites a day's carried mark (the original passed markSecondaryCarried=true, which flagged every agreeing day as preserved from expired logs), and days where no live slice can win are skipped rather than cloned. Verified on a real 137-day corpus: a frozen under-read day restores to the live figure, healthy days and the carried footnote are byte-identical to before. We could not push to your fork, hence the superseding PR. Thank you for the diagnosis and the fix.