Reports drop Grok reasoning tokens; optimize double-counts them for codex/copilot after #1078

[saulcanina]

• Codeburn: 0.9.20. Measured on main @ e12fb39 (includes fix(grok): read the CLI's own completed-turn usage instead of estimating it #1015 72aad9e); call sites re-checked on 554d32e
• Provider: Grok CLI, and any other provider outside the REASONING_INCLUDED_IN_OUTPUT set

Context

#1078 introduced billableOutputTokens(provider, outputTokens, reasoningTokens)
(src/models.ts:40-42) and a REASONING_INCLUDED_IN_OUTPUT set of
{claude, codex, copilot} (src/models.ts:35) to fix the opposite problem:
for those three providers, outputTokens already includes reasoning at the
source, so adding reasoningTokens on top double-billed it.

That fix only touches the three providers in the set. Every other provider,
Grok included, still reports outputTokens net of reasoning and
reasoningTokens separately, on the understanding that a consumer adds them
back. Three consumers do that correctly: the cost path (src/parser.ts:2651),
the models report (src/models-report.ts:129) and the audit report
(src/audit-report.ts:133), all through billableOutputTokens. The report,
sessions and overview totals do not, and optimize does it in a way that now
overcounts the #1078 set.

Bug 1: reported totals silently drop reasoning for exclusive providers

src/parser.ts:1753-1754 accumulates totalOutput and totalReasoning as
separate running sums per session, and both are carried into the session
record (totalOutputTokens, totalReasoningTokens at src/parser.ts:1820-1821).
The reported total comes from two different paths, and they fail for
different reasons.

Per-call path, the one that produces the measured number below: src/day-aggregator.ts:193
(callDay.outputTokens += call.usage.outputTokens, with the same pattern
repeated at :211 and :221 for model and slice breakdowns, and at :239),
consumed by buildPeriodDataFromDays in src/usage-aggregator.ts:477 and
surfaced by src/main.ts:496 (totalOutput = durable.data.outputTokens).
Here call.provider is available, it is already used at day-aggregator.ts:216,
so billableOutputTokens could be applied per call before accumulating.

Per-session path: src/usage-aggregator.ts:30, src/usage-aggregator.ts:751
(sessionDetailsOf), src/sessions-report.ts:58 (sessions --format json
payload), src/overview.ts:143 (overview totals). None of these four passes
the value through billableOutputTokens, and none has the session's
provider on hand at the point it sums, so the total is short by exactly the
reasoning volume for that session.

Measured evidence. Grok CLI corpus, 2026-07-20 to 2026-08-21 (query
window; data actually spans 2026-07-23 to 2026-08-20), 184 sessions, on an
isolated cache directory:

	tokens
ground truth, parsed from session files	607,588,697
reported by report / sessions	604,957,355
difference	2,631,342

The difference equals totalReasoningTokens for the same window to the exact
token. The cost path already goes through billableOutputTokens
(src/parser.ts:2651), so the divergence is display-only by construction;
cost was not re-measured. I can share the aggregated numbers above but not
the raw session files.

Bug 2: optimize.ts double-counts reasoning for the #1078 set

src/optimize.ts:3273:

// Reasoning is stored separately from ordinary output, but both are
// generated tokens for this detector. Reports already use their sum.
const outputTokens = session.totalOutputTokens + session.totalReasoningTokens

For codex and copilot sessions, totalOutputTokens is already inclusive
of reasoning: the parser accumulates whatever the provider reported, with no
per-provider adjustment (src/parser.ts:1753-1754). So this line adds the
reasoning a second time. For every other provider the repo contract is the
opposite, output exclusive of reasoning and consumers recombine through
billableOutputTokens(), as stated in the comment above
src/providers/grok.ts:417 (grok.ts:406-416; the line itself is
outputTokens: parsed.usage.output - reasoningTokens). The comment on
optimize.ts:3273 predates that helper and is now wrong in both directions:
the report/sessions/overview totals do not use the sum (Bug 1 reads
totalOutputTokens alone; models and audit do), and for the #1078 set the
sum overcounts.

This is not tied to Grok; it fires for any codex/copilot session that
reports non-zero reasoning, and gets worse the more a model thinks.

Scope

38 files under src/providers/ mention reasoningTokens, but most only ever
assign the literal 0. Counted on origin/main @ 554d32e for files that
actually populate it from parsed data: 16. Two of those, codex.ts and
copilot.ts, are in the REASONING_INCLUDED_IN_OUTPUT set and hit Bug 2,
not Bug 1. The other 14 hit Bug 1: grok.ts:168 and :184
(values[4] ?? 0), gemini.ts:135 (totalThoughts), qwen.ts:132
(usage.thoughtsTokenCount), kiro.ts:773 (estimated from reasoning
chars), plus cursor-agent.ts, hermes.ts, mux.ts, dsh.ts, zcode.ts,
droid.ts, antigravity.ts, vercel-gateway.ts, open-design.ts,
lingtai-tui.ts. (session-message.ts and sqlite-session-parser.ts also
populate it but are shared parsers parameterised by provider, so they are
not counted here).

Grok makes the gap visible because the corpus is large; on reasoning-heavy
models the share is well above the 0.43% seen here, so the under-report in
Bug 1 scales with how much a model thinks, and so does the over-count in
Bug 2 for the #1078 set.

Repro

codeburn report --provider grok --from <start> --to <end>
codeburn sessions --provider grok --format json --from <start> --to <end>

Compare either total against turn_completed.usage.reasoningTokens summed
across the same window. For Bug 2, run optimize on a codex or copilot
session with non-zero reasoningTokens and compare its generated-tokens
denominator against totalOutputTokens alone.

Fix shape

(a) Route all report/aggregation sites and optimize.ts:3273 through
billableOutputTokens(provider, outputTokens, reasoningTokens) instead of
reading or summing the raw fields. This is the same shape #1078 and #1079
already established for cost, so it keeps one source of truth instead of
two. The natural point for the per-call path is day-aggregator.ts:193 and
its three siblings, where the provider is already in scope. The four
per-session sites need to derive the provider per session; the cost path
does not help here, parser.ts:2651 runs per call and reads
call.provider, since SessionSummary has no provider field today.

(b) Normalize at parse time instead: make every provider's outputTokens
inclusive of reasoning at the source, drop the exclusive/inclusive split
entirely, and let cost sites that need the breakdown read
reasoningTokens on the side without adding it. Smaller diff at the
aggregation sites, but touches all 16 provider files above and removes the
contract #1078 just formalized.

(a) is consistent with the direction #1078/#1079 already took.

Closing

Happy to provide anonymized aggregates for the corpus above if useful.