fix(copilot): include copilot legacy chat sessions - #1232
Conversation
…jayly/copilot-legacy-json-sessions
ozymandiashh
left a comment
There was a problem hiding this comment.
Thanks for taking on the legacy format. I can't merge this as it stands: the tests pass, but the defects are all inside untested branches, and several change numbers for users on the current format too. Everything below was measured on the branch merged with current main (clean merge, full suite green apart from the known #1216 flake).
-
Reasoning tokens are billed as web-search requests.
src/providers/copilot.ts:702-709passesreasoningTokensas the sixth argument tocalculateCost, which iswebSearchRequests. A session with an 8,000-char thinking block on claude-sonnet-4-6 comes out at $20.04 where the true cost is about $0.03, withcostIsEstimated: false. Every other call site in the file passes0there. -
The rewrite regresses the current
.jsonlpath. The old parser readmetadata.promptTokensandmetadata.outputTokens || completionTokensbecause the two fields do not live in the same object (seechatSessionSampleRequestin the tests). The new code requires both in one object, so the repo's own real-data shape (metadata.promptTokens: 32543+ rootcompletionTokens: 490) falls through to char estimation: 32,543 input tokens become 2. Rows with no token fields, previously skipped, are now emitted as estimates. That's not a behavior-preserving refactor. -
Model ids get mangled instead of resolved.
normaliseLegacyModelIdturns every dotted version into dashes and the repo has no dash-to-dot normalisation, so anything not in the aliases you added loses or changes pricing:gemini-2.0-flash-001$0.15/Mtok to $0,gpt-4.5-preview$75 to $30 (matches a different row),gpt-5.3-codex-sparkfolded into the non-Spark SKU. Resolve through the alias mechanism; don't rewrite ids at the provider. -
Unresolvable model now falls back to
claude-sonnet-4-5. Main returnedunknownand $0 plus the unknown-model warning. The branch invents premium spend ($3.00 on a 1M-token request with no model info) and suppresses the diagnostic. -
Double-count with transcripts.
hasChatSessionFiles()(line 3358) still matches.jsonlonly, so a workspace with only legacy.jsonsessions no longer suppressesGitHub.copilot-chat/transcripts/. Both sources are discovered, dedup keys are in different namespaces, anddocs/providers/copilot.md:366documents this exact invariant. Extend the gate to.json, and key legacy sessions onsession.sessionId || basenameso the two formats of one session dedup against each other. -
Scope.
models.tsdisplay-name changes forcopilot-*-auto(affects dash, menubar, JSON),deriveGeminiShortNamenow overriding the short-name table, eight JetBrains model tokens, theinferJetBrainsProjectregex rewrite (which dropsC:/repoon Windows because the downstream filter still requires a leading slash), and awin32early-return added to an unrelated existing test. Please move all of that out; this PR should be the legacy chat sessions and nothing else.
There is no test coverage for the new cost path (grep -c 'renderedUserMessage|toolCallRounds|CacheBreakpoint' tests/ is 0), which is why CI is green. Fixtures exercising 1-5 are required for a re-review.
Summary
Adding support for VS Code's GitHub Copilot chat sessions prior to February 2026 to copilot's parser.
How it works:
The parser scan for
chatSessions/*.jsonand parses the legacy chat sessions if there is arequestsarray in root JSON object. The schema of those JSON files and its definitions likeIToolCallRoundandThinkingDatacan be found in https://github.com/microsoft/vscode-copilot-chat.This old format reads all of input, output, reasoning messages and tool calling outputs to determine number of tokens used by a model, which is also stored in the request object. It reads exact token counts (
promptTokens,completionTokens,usage) when available, otherwise it fallbacks to character-based estimation.This parser also maps the model used in auto model selection to the actual model used.
Testing
npm testpassesnpm run buildsucceedsnpm run dev -- todayshows correct costs and session counts for this providernpm run dev -- models --provider copilotshows correct model names and pricingTerminal output of running
npm run dev -- models --provider copilot -p lifetimeBefore:
After: