Repository navigation
[usage] Account for what a call charged and what it cost - #184
Merged
Merged
Conversation
"A3" left two questions with no way to answer either. Some routes report what a
request cost; the rest have to be priced from rates. And rates need the lanes:
DeepSeek bills four input lanes (cache hit vs cache miss) times a peak/off-peak
rule, which one input price cannot express, and `types.Usage` could not hold a
lane.
- `types.Usage` gains `CachedPromptTokens`, `CacheWriteTokens` and
`ReasoningTokens`: one field per *billing lane*, so OpenAI's nested
`prompt_tokens_details` and DeepSeek's flat `prompt_cache_hit_tokens` land in
the same place and nothing downstream has to know which route it is on. The
nested shape wins when a route sends both, so nothing is counted twice.
- `types.Cost{Amount, Currency}`, `CostSource` and `CostEvent` carry money with
its unit and say where the number came from. `CostTotals` accumulates per unit:
an account's credits and the currency of a user's invoice are added apart,
never together, because their sum would mean nothing.
- The OpenAI-compatible provider reads a reported charge (`cost`, falling back to
`cost_details.upstream_inference_cost` for a BYOK request that billed nothing)
and carries it verbatim. `SetRouteInfo` names the configuration route and the
unit it bills in, so the provider knows something the agent cannot: which route
a call went through.
- `PriceTable` holds declared rates keyed by `<route>/<model>`, with `*` allowed
in either position, resolved most-specific-first so two entries can never both
claim a call. `Amount` carves the lanes out of the totals (the details are
subsets, not extra tokens), falls each detail rate back to the lane it belongs
to, and floors every subtraction at zero.
- A call is priced from a declaration only when its tokens were *reported*:
multiplying a declared rate by a guessed token count would dress a second
estimate as a measurement. A call with no rate and no reported charge is
counted as unknown — never as zero, which would read as free in exactly the
decisions the number exists to inform.
- The status bar shows `cost: <totals>` plus `(+N unpriced)` once there is
something to say, and is byte-identical to before when there is not.
Cost/pricing is tokens-only in a second sense worth stating: a price table of its
own is still an open decision (#182), and nothing here waits on it.
Closes #181.
Tests: `go test ./... -count=1` and `-race` green (13 packages); gofmt and
staticcheck v0.8.1 clean. Eight mutations of the new behaviour each fail on an
assertion — the nested usage shape ignored, the upstream cost ignored, the detail
lanes added on top of the totals instead of carved out, a wildcard outranking an
exact key, an estimated usage priced anyway, an unknown cost recorded as zero,
a declared rate outranking the route's own figure, and the UI dropping the
unpriced count.
The changelog entry explains the two halves a reader has to keep apart: what a route charges (reported, carried verbatim) and what the tokens were (lanes, so a rate can be applied). CODEBASE.md gains the new types, the price table and the unknown-cost rule. The A3 entry now says cost was delivered in #181, and keeps only the price-table question open in #182.
The cost segment was added whenever any call was unpriced, and a call is unpriced whenever a route reports neither usage nor a charge. That is the common case: the CI stubs report no usage at all, so every real run carried "cost: unknown (2 unpriced)" — 26 columns spent telling a user who never asked about money that we do not know what their calls cost. It also broke a real surface: the longer bar pushed "Press Ctrl+C again to quit" past the wrap in the permission-allow-always scenario, and the run timed out waiting for text that had been split across two lines. Money now appears when cost accounting is in play — a rate the user declared in pricing, or a route that reported a charge — and shows the unknown count beside the total from that point on. A session with neither keeps the bar it has always had, which is not the same as reporting zero. Verified with the real PTY runner (make test-tui-scenarios): 31 scenarios, 27 ok, 4 gated skips, including the permission-allow-always case that failed in CI.
This was referenced Oct 11, 2026
[usage] Cost accounting: use a provider-reported cost, and give usage the cache detail it needs
#181
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #181. Roadmap entry A3's cost half (tracker: #168); the price-table question stays open in #182.
What changed
types.UsagegainsCachedPromptTokens,CacheWriteTokens,ReasoningTokens:one field per billing lane, so OpenAI's nested
prompt_tokens_detailsand DeepSeek's flatprompt_cache_hit_tokensland in the same place. The nested shape wins when a route sends both, sonothing is counted twice.
types.Cost,CostSource,CostEvent,CostTotalsandChatResponse.Cost; the OpenAI-compatible provider readscost, falling back tocost_details.upstream_inference_costfor a BYOK request that billed nothing, and carries theamount verbatim.
SetRouteInfonames the route and the unit it bills in.pricingin the configuration, keyed by<route>/<model>with*in either position, resolved most-specific-first;PriceTable.Amountcarves the detail lanes outof the totals and falls each detail rate back to the lane it belongs to.
cost: <totals>plus(+N unpriced);CHANGELOG.md,CODEBASE.md,docs/roadmap.md,config.example.json.Why
A3 left "what did this cost" unanswered for two independent reasons, and they need different
mechanisms:
cost(and aprompt_tokens_detailssplit ofcached and cache-written input) in its usage object, so no price list is involved at all. That
number is a reading of the bill, so it is carried verbatim — never recomputed, never added to a
figure derived from a table.
bills four input lanes (cache hit vs cache miss) times a peak/off-peak rule: a 50× spread on
deepseek-flashinput alone. A single input price cannot express it, which is why the usage detailcomes first and why a naive two-number table was rejected in [usage] Decide whether to vendor a model price table (and whose data it would be) #182.
Two deliberate refusals, both pinned by tests:
count would dress a second estimate as a measurement.
Ollamaand subscription plans have noper-token price at all, and
0would read as "free" in exactly the budget decisions this feeds.The bar shows the total and the unknowns beside it, because a total without them is a different
claim.
Verification
go test ./... -count=1→ 13 packages ok, 0 failed.go test ./... -count=1 -race→ same.make staticcheck(v0.8.1) → exit 0;go vet ./...andgofmt -l .clean.The 27 committed frame goldens pass unchanged: the cost segment appears only once a cost is known.
Eight mutations of the new behaviour, each failing on an assertion:
CachedPromptTokens:0 … want CachedPromptTokens:60 CacheWriteTokens:10cost = <nil>, want 19 USDamount = 16.1, want 11.1(and19, want 14for the fallback)Lookup("openrouter/claude-sonnet-4").InputPerMillion = 3, want 4events = [{… Source:declared}], want one unknownCostTotal = "0", want emptyCostTotal = "200 USD", want "7 credits"bar = "… cost: 0 …", want an explicit unknownMoney is computable by one command, which is this repository's rule for a number in a document:
TestPriceAmountCarvesTheLanesfixes a usage and a rate and asserts the amount, lane by lane.Honest scope
object; the lane fixtures mirror OpenAI's and DeepSeek's documented fields.
with a guessed one (
cost_currencynames it; it is not defaulted).hour needs either a reported charge or a rate the user updates, and saying so is better than
averaging the two.
parent's totals — that is roadmap G1, as in [usage] Stop a run at a cumulative token budget (per run and per session) #179.