Skip to content

[usage] Account for what a call charged and what it cost - #184

Merged
yusiwen merged 3 commits into
masterfrom
feat/181-cost-accounting
Oct 11, 2026
Merged

yusiwen merged 3 commits into
masterfrom
feat/181-cost-accounting

Conversation

@yusiwen

@yusiwen yusiwen commented Oct 10, 2026

Copy link
Copy Markdown
Owner

Closes #181. Roadmap entry A3's cost half (tracker: #168); the price-table question stays open in #182.

What changed

  • S1 — the lanes. types.Usage gains CachedPromptTokens, CacheWriteTokens, ReasoningTokens:
    one field per billing lane, so OpenAI's nested prompt_tokens_details and DeepSeek's flat
    prompt_cache_hit_tokens land in the same place. The nested shape wins when a route sends both, so
    nothing is counted twice.
  • S2 — the charge a route reports. types.Cost, CostSource, CostEvent, CostTotals and
    ChatResponse.Cost; the OpenAI-compatible provider reads cost, falling back to
    cost_details.upstream_inference_cost for a BYOK request that billed nothing, and carries the
    amount verbatim. SetRouteInfo names the route and the unit it bills in.
  • S3 — the rates a user declares. pricing in the configuration, keyed by <route>/<model> with
    * in either position, resolved most-specific-first; PriceTable.Amount carves the detail lanes out
    of the totals and falls each detail rate back to the lane it belongs to.
  • S4 — display and docs. The status bar shows cost: <totals> plus (+N unpriced); CHANGELOG.md,
    CODEBASE.md, docs/roadmap.md, config.example.json.

Why

A3 left "what did this cost" unanswered for two independent reasons, and they need different
mechanisms:

  • A route may report the charge. OpenRouter returns cost (and a prompt_tokens_details split of
    cached and cache-written input) in its usage object, so no price list is involved at all. That
    number is a reading of the bill, so it is carried verbatim — never recomputed, never added to a
    figure derived from a table.
  • Otherwise a rate has to be applied, and a rate needs lanes. DeepSeek — the default provider —
    bills four input lanes (cache hit vs cache miss) times a peak/off-peak rule: a 50× spread on
    deepseek-flash input alone. A single input price cannot express it, which is why the usage detail
    comes first and why a naive two-number table was rejected in [usage] Decide whether to vendor a model price table (and whose data it would be) #182.

Two deliberate refusals, both pinned by tests:

  • A call whose tokens we estimated is not priced. Multiplying a declared rate by a guessed token
    count would dress a second estimate as a measurement.
  • An unpriced call is counted as unknown, never as zero. Ollama and subscription plans have no
    per-token price at all, and 0 would read as "free" in exactly the budget decisions this feeds.
    The bar shows the total and the unknowns beside it, because a total without them is a different
    claim.

Verification

  • go test ./... -count=1 → 13 packages ok, 0 failed. go test ./... -count=1 -race → same.

  • make staticcheck (v0.8.1) → exit 0; go vet ./... and gofmt -l . clean.

  • The 27 committed frame goldens pass unchanged: the cost segment appears only once a cost is known.

  • Eight mutations of the new behaviour, each failing on an assertion:

    Mutation What the test said
    nested usage detail ignored CachedPromptTokens:0 … want CachedPromptTokens:60 CacheWriteTokens:10
    upstream cost ignored cost = <nil>, want 19 USD
    lanes added on top of the totals amount = 16.1, want 11.1 (and 19, want 14 for the fallback)
    a wildcard outranking an exact key Lookup("openrouter/claude-sonnet-4").InputPerMillion = 3, want 4
    an estimated usage priced anyway events = [{… Source:declared}], want one unknown
    an unknown cost recorded as zero CostTotal = "0", want empty
    a declared rate outranking the route's figure CostTotal = "200 USD", want "7 credits"
    the UI dropping the unpriced count bar = "… cost: 0 …", want an explicit unknown
  • Money is computable by one command, which is this repository's rule for a number in a document:
    TestPriceAmountCarvesTheLanes fixes a usage and a rate and asserts the amount, lane by lane.

Honest scope

  • No live provider call was made. The reported-cost fixtures mirror OpenRouter's documented usage
    object; the lane fixtures mirror OpenAI's and DeepSeek's documented fields.
  • A route that reports a charge but no currency code is displayed without a unit rather than labelled
    with a guessed one (cost_currency names it; it is not defaulted).
  • Peak/off-peak pricing is not modelled: a declared rate is one rate. A provider that prices by the
    hour needs either a reported charge or a rate the user updates, and saying so is better than
    averaging the two.
  • Pricing a sub-agent's call is done with the same table, but its spend still is not attributed to the
    parent's totals — that is roadmap G1, as in [usage] Stop a run at a cumulative token budget (per run and per session) #179.

"A3" left two questions with no way to answer either. Some routes report what a
request cost; the rest have to be priced from rates. And rates need the lanes:
DeepSeek bills four input lanes (cache hit vs cache miss) times a peak/off-peak
rule, which one input price cannot express, and `types.Usage` could not hold a
lane.

- `types.Usage` gains `CachedPromptTokens`, `CacheWriteTokens` and
  `ReasoningTokens`: one field per *billing lane*, so OpenAI's nested
  `prompt_tokens_details` and DeepSeek's flat `prompt_cache_hit_tokens` land in
  the same place and nothing downstream has to know which route it is on. The
  nested shape wins when a route sends both, so nothing is counted twice.
- `types.Cost{Amount, Currency}`, `CostSource` and `CostEvent` carry money with
  its unit and say where the number came from. `CostTotals` accumulates per unit:
  an account's credits and the currency of a user's invoice are added apart,
  never together, because their sum would mean nothing.
- The OpenAI-compatible provider reads a reported charge (`cost`, falling back to
  `cost_details.upstream_inference_cost` for a BYOK request that billed nothing)
  and carries it verbatim. `SetRouteInfo` names the configuration route and the
  unit it bills in, so the provider knows something the agent cannot: which route
  a call went through.
- `PriceTable` holds declared rates keyed by `<route>/<model>`, with `*` allowed
  in either position, resolved most-specific-first so two entries can never both
  claim a call. `Amount` carves the lanes out of the totals (the details are
  subsets, not extra tokens), falls each detail rate back to the lane it belongs
  to, and floors every subtraction at zero.
- A call is priced from a declaration only when its tokens were *reported*:
  multiplying a declared rate by a guessed token count would dress a second
  estimate as a measurement. A call with no rate and no reported charge is
  counted as unknown — never as zero, which would read as free in exactly the
  decisions the number exists to inform.
- The status bar shows `cost: <totals>` plus `(+N unpriced)` once there is
  something to say, and is byte-identical to before when there is not.

Cost/pricing is tokens-only in a second sense worth stating: a price table of its
own is still an open decision (#182), and nothing here waits on it.

Closes #181.

Tests: `go test ./... -count=1` and `-race` green (13 packages); gofmt and
staticcheck v0.8.1 clean. Eight mutations of the new behaviour each fail on an
assertion — the nested usage shape ignored, the upstream cost ignored, the detail
lanes added on top of the totals instead of carved out, a wildcard outranking an
exact key, an estimated usage priced anyway, an unknown cost recorded as zero,
a declared rate outranking the route's own figure, and the UI dropping the
unpriced count.
The changelog entry explains the two halves a reader has to keep apart: what a
route charges (reported, carried verbatim) and what the tokens were (lanes, so a
rate can be applied). CODEBASE.md gains the new types, the price table and the
unknown-cost rule. The A3 entry now says cost was delivered in #181, and keeps
only the price-table question open in #182.
@yusiwen yusiwen added the feature A new capability rather than an improvement to existing behaviour label Oct 10, 2026
The cost segment was added whenever any call was unpriced, and a call is
unpriced whenever a route reports neither usage nor a charge. That is the
common case: the CI stubs report no usage at all, so every real run carried
"cost: unknown (2 unpriced)" — 26 columns spent telling a user who never asked
about money that we do not know what their calls cost.

It also broke a real surface: the longer bar pushed "Press Ctrl+C again to
quit" past the wrap in the permission-allow-always scenario, and the run
timed out waiting for text that had been split across two lines.

Money now appears when cost accounting is in play — a rate the user declared
in pricing, or a route that reported a charge — and shows the unknown count
beside the total from that point on. A session with neither keeps the bar it
has always had, which is not the same as reporting zero.

Verified with the real PTY runner (make test-tui-scenarios): 31 scenarios, 27
ok, 4 gated skips, including the permission-allow-always case that failed in
CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature A new capability rather than an improvement to existing behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[usage] Cost accounting: use a provider-reported cost, and give usage the cache detail it needs

1 participant