Repository navigation
[Bug]: SSE reasoning stream deltas (delta.reasoning / thought) accumulate duplicate state objects in SQLite causing context explosion across turns #44581
Description
Description
During multi-turn testing with reasoning models (such as ox-alpha and models outputting streaming thought/reasoning tokens), we observed severe prompt token inflation across consecutive turns. After investigating the local session storage, we found that every streaming reasoning SSE chunk (delta.reasoning / delta.thought) was being persisted as an individual { type: "reasoning", state: ... } object inside session_message.content[] instead of being concatenated / folded into a single reasoning content block.
(Attribution: Discovered during multi-turn testing with ox-alpha/reasoning models; investigated and code-isolated together with my AI pairing assistant).
Empirical Evidence & Token Metrics
1. SQLite Inspection (session_message)
Inspecting the SQLite database for a single streaming assistant message containing reasoning tokens revealed over 375 discrete reasoning objects within content:
// session_message.content (excerpt)
[
{ "type": "reasoning", "state": { "text": "Let" } },
{ "type": "reasoning", "state": { "text": " me" } },
{ "type": "reasoning", "state": { "text": " check" } },
... // 375+ entries (~1.09 MB of redundant JSON structure per single response)
]2. Context Inflation Across Turns
When subsequent turns re-serialize the full conversation history to upstream providers, the repeated JSON envelope overhead and duplicated objects inflate prompt context rapidly:
| Turn Count | Prompt Tokens (Without Folding) | Prompt Tokens (With In-Place Folding) | Context Bloat Factor |
|---|---|---|---|
| Turn 1 | ~2.5k | ~2.5k | 1.0x |
| Turn 5 | ~118k | ~14k | ~8.4x |
| Turn 10 | 300k+ (Context Explosion / OOM) | ~28k | ~10.7x |
Root Cause Analysis
In the streaming reducer / message processor, incoming SSE delta chunks containing delta.reasoning or delta.thought push a new object onto the message content[] array on each chunk arrival, rather than identifying the existing reasoning block and appending chunk text in-place:
// Problematic behavior:
// Every chunk creates a new content array element
content.push({ type: "reasoning", state: { text: chunk.delta.reasoning } });Proposed Fix
1. In-Place Stream Accumulator / Reducer
When processing incoming delta chunks, fold into the active reasoning block or append to the existing block:
function reduceReasoningDelta(content: ContentPart[], deltaText: string): ContentPart[] {
const lastPart = content[content.length - 1];
if (lastPart && lastPart.type === "reasoning") {
// Fold in-place
lastPart.state = {
...lastPart.state,
text: (lastPart.state?.text ?? "") + deltaText,
};
} else {
// Initialize single reasoning part
content.push({
type: "reasoning",
state: { text: deltaText },
});
}
return content;
}2. Request Scrubbing Before Dispatch
When serializing conversation history for upstream LLM completion requests, ensure consecutive/fragmented reasoning blocks are compacted into single text blocks or filtered according to provider capabilities:
function compactMessageContent(content: ContentPart[]): ContentPart[] {
return content.reduce<ContentPart[]>((acc, current) => {
const prev = acc[acc.length - 1];
if (prev && prev.type === "reasoning" && current.type === "reasoning") {
prev.state = {
...prev.state,
text: (prev.state?.text ?? "") + (current.state?.text ?? ""),
};
return acc;
}
acc.push(current);
return acc;
}, []);
}Additional Context
- OpenCode Version: 1.18.x / latest
- OS: Linux / WSL2 / macOS
- Willingness to contribute: Happy to open a clean PR with unit tests for the stream reducer and message normalizer if maintainers would like!
Activity
JavaGT commentedon Aug 28, 2026
Confirmed independently with detailed evidence — see #45791 (same root cause). Key findings: a live reproduction on v0.0.0-beta-18414 stored 792 identical full-text reasoning objects (32,251 chars each = 25.5 MB) for a single turn; per-request provider-reported input in a real session grew 14,972 → 156,718 → 171,779 → 300,968 tokens (555k input + 640k cache-read total for a 12-record session, /bin/zsh.054, only 2,134 output tokens). The duplication lives in
state.reasoningDetails[0].text(top-leveltextis empty on all but the first object), and the objects are created throughout the stream sharing one finalized metadata blob. Suggested fix (in #45791): coalesce reasoning events into one logical part, attach final metadata once without shared references, and defensively deduplicate outgoingreasoning_detailsfor existing corrupted sessions.terryzwt commentedon Sep 20, 2026
Adding a confirmed reproduction on the current release, plus a downstream symptom that is more severe than context inflation.
Still reproduces on 2.0.11 (latest) with
kimi-code-plan-cn/kimi-for-coding(Moonshot Kimi,api.kimi.com, OpenAI-compatible), even though #45864 merged — so this accumulation path is not fully covered. Same accumulation root cause described here, but applied to structuredreasoning_details: one entry per streamed token, never merged byindex, persisted with the streamingindex.The accumulated array is not only bloat — on replay the provider hard-rejects the request, making the model unusable past the first turn:
and after stripping
indexfrom every entry, the next rejection is:(
position 2is the first assistant message.) Reproduces consistently, every second turn, including turns after tool calls.Minimal repro:
Replayed
reasoning_detailson the assistant message:[ { "index": 0, "type": "summary", "summary": "Preparing" }, { "index": 0, "summary": " to" }, { "index": 0, "summary": " reply" }, ... { "index": 1, "type": "encrypted", "encrypted": "..." } ]Most entries have no
typebecause each chunk was appended instead of folded byindex.Two extra observations:
interleaved: { "field": "reasoning_content" }for these models, soreasoning_detailsarguably shouldn't be replayed (or should be normalized) at all.reasoning_detailsmust not carry the streamingindex— so the fix likely needs to stripindex(and/or not persist it) rather than only fixing the lifecycle.Workaround that unblocks us in the meantime: a small local plugin on
ctx.session.hook("http.request")that deletesmessage.reasoning_detailsfrom outgoing requests for the Kimi providers. Verified for both plain multi-turn and tool-call turns. Happy to share it if useful.rekram1-node commentedon Oct 8, 2026
was fixed by:
#45864
for some OpenAI-compatible providers (we saw it with MiniMax and GLM through OpenRouter), the chat stream parser could close and reopen the reasoning block mid-response. Each reopen created a new reasoning part. Depending on the provider, that left you with one part per token or a long run of empty parts after the text.