Skip to content

Commit 0fead1c

Browse files
committed
fix(agent-runtime): release an unclosed think-tag open as text at flush
An explicit <think> open committed the rest of the step to the thinking box unconditionally, so when a model wrote the tag as prose (docs, quoted templates, a broken chat template) the visible answer landed in the thinking box and the user saw an empty or short reply. Give the explicit open the same treatment the implicit head already has: hold until a close settles the block as reasoning, and release the hold as text when the step ends without one — an answer is delayed, never swallowed. The budget stays implicit-head-only: a genuine think block can legitimately run long, and only its close (or flush) settles it. Refs #1155
1 parent 28663a3 commit 0fead1c

2 files changed

Lines changed: 101 additions & 34 deletions

File tree

‎packages/agent-runtime/src/util/__tests__/think-tag-stream.test.ts‎

Lines changed: 43 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -59,10 +59,50 @@ describe('ThinkTagStream — paired tags', () => {
5959
expect(joined(run(['ends with <']), 'text')).toBe('ends with <')
6060
})
6161

62-
it('treats an unclosed open tag as reasoning through end of stream', () => {
62+
it('releases an unclosed open tag as text at flush', () => {
63+
// The open may be a truncated thought or the tag quoted as prose — both
64+
// look identical until the step ends, so the hold resolves as text either
65+
// way: an answer is delayed, never swallowed (issue #1155, bug 2).
6366
const out = run(['<think>truncated thou', 'ght'])
64-
expect(joined(out, 'reasoning')).toBe('truncated thought')
65-
expect(joined(out, 'text')).toBe('')
67+
expect(joined(out, 'reasoning')).toBe('')
68+
expect(joined(out, 'text')).toBe('truncated thought')
69+
})
70+
71+
it('keeps an answer quoted around a prose open tag as text', () => {
72+
// Docs quoting the tag: the answer before it already streamed as text,
73+
// the text after it must not vanish into the thinking box.
74+
const out = run([
75+
'Write <think> ',
76+
'like this in your docs. The answer continues here.',
77+
])
78+
expect(joined(out, 'reasoning')).toBe('')
79+
expect(joined(out, 'text')).toBe(
80+
'Write like this in your docs. The answer continues here.',
81+
)
82+
})
83+
84+
it('holds an explicit open only until its close arrives', () => {
85+
const stream = new ThinkTagStream()
86+
expect(stream.push('Answer part one. <think>plan it')).toEqual([
87+
{ type: 'text', text: 'Answer part one. ' },
88+
])
89+
expect(stream.push('</think>Here is the answer.')).toEqual([
90+
{ type: 'reasoning', text: 'plan it' },
91+
{ type: 'text', text: 'Here is the answer.' },
92+
])
93+
})
94+
95+
it('releases an explicit open on a native reasoning chunk as text', () => {
96+
const stream = new ThinkTagStream()
97+
expect(stream.push('A <think> quoted open')).toEqual([
98+
{ type: 'text', text: 'A ' },
99+
])
100+
expect(stream.disarmImplicitOpen()).toEqual([
101+
{ type: 'text', text: ' quoted open' },
102+
])
103+
expect(stream.push(' and the answer continues')).toEqual([
104+
{ type: 'text', text: ' and the answer continues' },
105+
])
66106
})
67107
})
68108

‎packages/agent-runtime/src/util/think-tag-stream.ts‎

Lines changed: 58 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,15 @@
2222
*
2323
* 1. `<think>…</think>` — paired tags. Content between them is reasoning.
2424
* Unambiguous, free, always on.
25-
* 2. A bare `<think>` that never closes (a truncated thought). Everything
26-
* after it is reasoning.
25+
* 2. A bare `<think>` that never closes. Two different events produce this:
26+
* a thought truncated mid-stream, or the tag written as PROSE — docs that
27+
* mention the tag, a quoted template, a lane with a broken chat template.
28+
* The two are indistinguishable while the deltas arrive, so everything
29+
* after the open is HELD rather than emitted (same mechanism as rule 3).
30+
* A close settles the hold as reasoning; a step that ends without one
31+
* releases it as text, so an answer is delayed but never swallowed.
32+
* (History still stores the raw text either way — this is a
33+
* display/reclassification concern, not context loss.)
2734
* 3. An orphan `</think>` with no open tag — the DeepSeek shape above, where
2835
* the open tag was consumed by the chat template's prefill. The text
2936
* BEFORE it is reasoning, but by the time the marker arrives that text has
@@ -153,27 +160,34 @@ export function historyLeaksThinkTags(
153160
export class ThinkTagStream {
154161
/** Trailing bytes withheld because they may be the start of a tag. */
155162
private partial = ''
156-
/** Leading content withheld while `implicitOpen` is still undecided. */
163+
/** Content withheld while the classification of an open block is still
164+
* undecided: the implicit head, or everything since an explicit open. */
157165
private held = ''
158166
private implicitOpen: boolean
159167
private inThinkBlock: boolean
168+
/** True while `held` is waiting for the close that settles its
169+
* classification. Armed by construction (implicitOpen) and by every open
170+
* tag; cleared by the close ({@link confirmOpenHold}) or by giving up
171+
* ({@link abandonOpenHold}) — budget, a native reasoning chunk, or flush. */
172+
private holdingForOpen: boolean
160173

161174
constructor(options: ThinkTagStreamOptions = {}) {
162175
this.implicitOpen = options.implicitOpen ?? false
163176
this.inThinkBlock = this.implicitOpen
177+
this.holdingForOpen = this.implicitOpen
164178
}
165179

166180
/**
167-
* Give up on `implicitOpen` and release anything held as text.
181+
* Give up on the hold and release anything held as text.
168182
*
169-
* Called when the step turns out not to be leaking after all. The strongest
170-
* such signal is a native reasoning chunk: a lane that populates
171-
* `reasoning_content` is by definition not putting the thought in `content`,
172-
* so whatever is in `content` is the answer.
183+
* Called when the step turns out not to be thinking in `content` after all.
184+
* The strongest such signal is a native reasoning chunk: a lane that
185+
* populates `reasoning_content` is by definition not putting the thought in
186+
* `content`, so whatever is in `content` is the answer.
173187
*/
174188
disarmImplicitOpen(): ThinkStreamSegment[] {
175-
if (!this.implicitOpen) return []
176-
return this.abandonImplicitOpen()
189+
if (!this.holdingForOpen) return []
190+
return this.abandonOpenHold()
177191
}
178192

179193
push(chunk: string): ThinkStreamSegment[] {
@@ -189,10 +203,11 @@ export class ThinkTagStream {
189203
this.addReasoning(segments, buffer.slice(0, closeIdx))
190204
buffer = buffer.slice(closeIdx + CLOSE_TAG.length)
191205
this.inThinkBlock = false
192-
// The close the implicit block was waiting for: everything held is
193-
// confirmed reasoning. It can only happen once — a later orphan close
194-
// is an ordinary stray marker and is stripped below.
195-
this.confirmImplicitOpen(segments)
206+
// The close the hold was waiting for: everything held is confirmed
207+
// reasoning — whether the block was opened implicitly (a leaked chain
208+
// of thought) or explicitly. A later orphan close is an ordinary
209+
// stray marker and is stripped below.
210+
this.confirmOpenHold(segments)
196211
continue
197212
}
198213

@@ -203,6 +218,11 @@ export class ThinkTagStream {
203218
this.addText(segments, buffer.slice(0, openIdx))
204219
buffer = buffer.slice(openIdx + OPEN_TAG.length)
205220
this.inThinkBlock = true
221+
// An open tag could be a block the model is thinking in, or the tag
222+
// quoted as prose. Both look identical until a close (or the end of
223+
// the step) settles it, so hold from here rather than committing the
224+
// rest of the step to the thinking box.
225+
this.holdingForOpen = true
206226
continue
207227
}
208228
// Orphan close with nothing to close: drop the marker so it cannot reach
@@ -220,31 +240,32 @@ export class ThinkTagStream {
220240
}
221241

222242
/** Emit everything withheld. A partial tag that never completed was always
223-
* just text, and content held for an orphan close that never came is the
224-
* answer — releasing both here is what makes the speculation lossless. */
243+
* just text, and content held for a close that never came is the answer —
244+
* releasing both here is what makes the hold lossless. An unclosed open is
245+
* treated exactly like an orphan close: the marker is scaffolding, the
246+
* text around it is the answer. */
225247
flush(): ThinkStreamSegment[] {
226248
const segments: ThinkStreamSegment[] = []
249+
if (this.holdingForOpen) segments.push(...this.abandonOpenHold())
227250
const trailing = this.partial
228251
this.partial = ''
229-
if (trailing) {
230-
if (this.inThinkBlock) this.addReasoning(segments, trailing)
231-
else this.addText(segments, trailing)
232-
}
233-
if (this.implicitOpen) segments.push(...this.abandonImplicitOpen())
252+
if (trailing) this.addText(segments, trailing)
234253
return segments
235254
}
236255

237-
/** The orphan close arrived: what was held was reasoning after all. */
238-
private confirmImplicitOpen(segments: ThinkStreamSegment[]): void {
239-
if (!this.implicitOpen) return
256+
/** The close arrived: what was held was reasoning after all. */
257+
private confirmOpenHold(segments: ThinkStreamSegment[]): void {
258+
if (!this.holdingForOpen) return
259+
this.holdingForOpen = false
240260
this.implicitOpen = false
241261
const held = this.held
242262
this.held = ''
243263
if (held) push(segments, 'reasoning', held)
244264
}
245265

246266
/** No close is coming: what was held was the answer. */
247-
private abandonImplicitOpen(): ThinkStreamSegment[] {
267+
private abandonOpenHold(): ThinkStreamSegment[] {
268+
this.holdingForOpen = false
248269
this.implicitOpen = false
249270
this.inThinkBlock = false
250271
const held = this.held
@@ -259,16 +280,22 @@ export class ThinkTagStream {
259280
// A nested/duplicated open tag inside a block is scaffolding, never thought.
260281
const cleaned = text.split(OPEN_TAG).join('')
261282
if (!cleaned) return
262-
if (!this.implicitOpen) {
283+
if (!this.holdingForOpen) {
263284
push(segments, 'reasoning', cleaned)
264285
return
265286
}
266-
// Still undecided: this is reasoning only if an orphan close confirms it,
267-
// so hold rather than send. Past the budget the step is answering, not
268-
// thinking, and the hold is released as text.
287+
// Undecided: reasoning only if a close confirms it, so hold rather than
288+
// send. The budget bounds only the implicit-head speculation — a held
289+
// chain of thought from a leaking lane runs well past it, so past the
290+
// budget the step is answering, not thinking, and the hold is released as
291+
// text. An explicit open gets no budget: a genuine think block can
292+
// legitimately run long, and only its close (or flush) settles it.
269293
this.held += cleaned
270-
if (this.held.length >= IMPLICIT_OPEN_BUDGET_CHARS) {
271-
segments.push(...this.abandonImplicitOpen())
294+
if (
295+
this.implicitOpen &&
296+
this.held.length >= IMPLICIT_OPEN_BUDGET_CHARS
297+
) {
298+
segments.push(...this.abandonOpenHold())
272299
}
273300
}
274301

0 commit comments

Comments
 (0)