fix(cost): make paid accounting durable#360
Conversation
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 92c5309d
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T15:03:38Z
tangletools
left a comment
There was a problem hiding this comment.
🟠 Value Audit — better-approach-exists
| Verdict | better-approach-exists |
| Concerns | 2 (1 medium-concern, 1 low) |
| Heuristic | 0.1s |
| Duplication | 0.0s |
| Interrogation | 257.2s (2 bridge agents) |
| Total | 257.3s |
💰 Value — better-approach-exists
A coherent, well-tested redesign that makes campaign cost accounting durable, crash-safe, and ceiling-enforced through a single runPaidCall admission path — but the filesystem durability adapter reinvents an existing in-repo primitive and adds an unnecessary runtime dependency.
- What it does: Replaces the old scattergun CampaignCostMeter (observe/observeTokens/observeModel) with one method — CostLedger.runPaidCall — that is the single path for every paid LLM/subprocess call. Before provider dispatch it persists a bounded 'pending' reservation; on completion it settles that reservation to a strict provider receipt (tokens + cost); on crash the pending record blocks new paid work until r
- Goals it achieves: (1) Prevent overspend: reserve the enforced maximum before dispatch, reject if it would breach the ceiling. (2) Survive crashes: persist every reservation and receipt so a killed process leaves a replayable event log, not a silent gap. (3) Prevent double-spend: callId uniqueness + provider idempotency key means a redrive after crash can't charge twice. (4) Never report an incomplete total as compl
- Assessment: The core design is strong and in the grain of the codebase. CostLedger keeps its place as the substrate primitive (src/cost-ledger.ts, no upward deps); the filesystem adapter lives in the campaign layer (src/campaign/storage.ts), respecting the documented layering rule. The pending→settled state machine, zod-validated event schema, typed PaidCallResult (succeeded/false discriminated union that cal
- Better / existing approach: The repo already has a dependency-free, tested, cross-process append+fsync+lock primitive in src/campaign/search-ledger-file.ts: appendSearchLedgerLine(path, line) does O_CREAT|O_WRONLY|O_APPEND + full writeAll (handles partial writes) + fsyncSync(fd) + fsyncSync(dir); withSearchLedgerFileLock(ledgerPath, run) does hard-link atomic CAS lock with stale-owner recovery (pid liveness via process.kill(
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 2
- Bridge warning: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content
🎯 Usefulness — sound
A single durable paid-call admission path (CostLedger.runPaidCall) replaces the loose observe/observeTokens meter across every LLM-spend surface, with file-backed reservations, ceiling enforcement, crash/resume reconciliation, and consistent helper wiring at all 15+ call sites.
- Integration: Fully reachable and already exercised. runPaidCall is wired into driver.ts:396, executor.ts:114, judges.ts:318, llm-judge.ts:187, semantic-concept-judge.ts:320, intent-match-judge.ts:193, judge-panel.ts, reviewer.ts:253, analyst/structure-findings.ts:120, every campaign proposer (gepa, skill-opt, llm-policy-edit, analysis-edit, memory, halo, trace-analyst), distillation/run-distillation.ts:180, wi
- Fit with existing patterns: Matches the codebase grain. It supersedes the prior loose CampaignCostMeter (observe/observeTokens, which had no admission control, durability, or ceiling) with one typed primitive that returns PaidCallResult callers must inspect — aligning with the repo's 'No fallbacks. Fail loud.' doctrine and the external-boundary typed-outcome convention. Durable persistence rides the existing CampaignStora
- Real-world viability: Robust beyond the happy path. The 41-test cost-ledger suite (passing locally) covers the hard paths the PR claims: concurrent reservation atomicity, crash/resume pending-record blocking, stale filesystem lock recovery (10s stale timer), partial-final-event loud failure, bounded-spend ceiling rejection before dispatch, abort-race late receipt capture, provider-maximum-violation retention, unpriced-
- Model: opencode/zai-coding-plan/glm-5.2
- Bridge attempts: 1
🔎 Heuristic Signals
🟡 Cruft: magic number added src/cost-ledger.test.ts
inputTokens: 1000,
💰 Value Audit
🟠 Filesystem adapter reinvents append+fsync+lock primitive from search-ledger-file.ts and adds proper-lockfile dep [duplication] ``
src/campaign/storage.ts:71-108 implements fsCampaignStorage().append with inline openSync('a')+writeFileSync+fsyncSync(fd)+fsyncSync(dir) and proper-lockfile.lockSync for cross-process serialization. src/campaign/search-ledger-file.ts:22-43 already exports appendSearchLedgerLine (identical durability: O_APPEND + writeAll + fsync file + fsync dir) and withSearchLedgerFileLock (identical stale-owner recovery via hard-link CAS + pid liveness). Both are schema-agnostic and reusable as-is. The new co
What this audit checks
It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.
| Pass | What it asks |
|---|---|
| Heuristic | Vague title? Whitespace-only or cruft-bearing diff? (content signals only) |
| Duplication | Do added function/class names already exist elsewhere in the repo? |
| Value Audit | What does it do? What goal does it achieve? Is it good? Better architecture or already-exists? |
| Usefulness Audit | Does it integrate and fit? Will it hold up in real use and actually get used? |
Findings are concerns, not blocks — the human reviewer decides what to do with them.
tangletools
left a comment
There was a problem hiding this comment.
✅ Auto-approved drewstone PR — 67b6101d
This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.
tangletools · auto-approval · reason: drewstone_author · 2026-07-13T16:50:16Z
Problem
Paid campaign work could exceed configured spend, lose in-flight spend across crashes, double-spend across processes, or report incomplete totals as complete.
Solution
CostLedger.runPaidCallthe single admission, reservation, receipt, resume, and summary path for campaign paid calls.This branch includes the three open #358 commits so the cost change can consume that contract directly. After #358 lands, GitHub will reduce this PR to the cost commit.
Checks
pnpm test: 3,010 passed, 2 skippedpnpm typecheck: passedpnpm lint: passed with 4 existing warnings, 0 errorsNODE_OPTIONS=--max-old-space-size=8192 pnpm build: passedpnpm verify:package: passedpnpm audit --prod: no known vulnerabilities