Repository navigation
[agent] Bound the OpenAI-compatible stream after finish_reason - #178
Merged
Merged
Conversation
Reading to `data: [DONE]` (issue #174) is what makes the usage chunk reachable. It also removed the early return at `finish_reason` without putting anything in its place: an endpoint that sends a complete answer and then holds the connection open left the step waiting for the whole-request client timeout, and `chatStream` returned the text it had with a nil error, so the loop could not tell a cut stream from a finished one. Measured against an endpoint that completes its answer and then goes quiet: 2 ms when the stream ends with `data: [DONE]`, 1.502 s at a 1.5 s caller deadline without it — and neither entry point passes a deadline of its own, so in production the wait is the full 120 s. - `openAIIdleTimeout` (2 min), reset by every line the scanner delivers, plus `openAIRequestTimeout` (120 s) for the batch path: the same two-bounds shape, and the same cancel-cause idiom, as `provider_ollama.go` (issue #1). Both are package variables so a test can shrink them. - A failed read is now an error (`openai stream: …`) instead of the partial answer with a nil error, which is also what the Ollama provider does. - `openAIRequestError` names the bound that fired, and keeps `errors.Is(err, context.Canceled)` for a caller cancellation — that is how the TUI renders an interrupted run, and it is pinned by a direct unit test rather than left to the transport: an HTTP/1.1 read error happens to wrap context.Canceled already, an HTTP/2 stream close does not. A stream that *closes* without `[DONE]` is still accepted as complete. OpenAI-compatible servers vary here, and failing their working answers would cost more than the truncation this catches. `[DONE]` remains the marker that ends the read early. Closes #176. Tests: `go test ./... -count=1` and `-race` green (13 packages); gofmt and staticcheck v0.8.1 clean. Four mutations of the new behaviour fail on an assertion — or, with the bound disabled, on the test timeout, which is itself the demonstration that the read blocks without it.
This was referenced Oct 10, 2026
[agent] Bound the SSE read after finish_reason so a stream without [DONE] cannot stall the call
#176
Closed
yusiwen
pushed a commit
that referenced
this pull request
Oct 11, 2026
Pin that a non-streaming `Chat` against an endpoint that accepts the connection and never answers fails within `openAIRequestTimeout`, mirroring `TestOllamaBatchTimesOutOnSilentEndpoint`. Test only: the code half of #176 already landed as `49964e2` (#178) — the same idle-timeout work this PR set out to do — so the author's force-push (`7222e40`, whose base already carried that fix) left the branch with this test alone. Authored by AdamMagued in #177. Verified: the test fails against a silent `httptest` endpoint at the shrunk bound (0.15 s locally, Go 1.27.1 darwin/arm64); `go test ./agent -count=1 -race` green; CI on `7222e40` 9/9 jobs pass with annotations at the per-job baseline (1, or 2 for browser and tui-visual). Refs #185, #186.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #176. Regression introduced by #175, found while delivering #174.
What changed
agent/provider_openai.gonow carries the same pair of bounds the Ollama provider does:openAIRequestTimeout(120 s) for one non-streaming request, andopenAIIdleTimeout(2 min) forthe silence between lines of a stream, reset by every line the scanner delivers. Both are package
variables so a test can shrink them, and the cancel cause names the bound (
errOpenAISilent), thesame idiom as
provider_ollama.go(issue agent: the Ollama provider has no request timeout, so a stalled endpoint hangs the run #1).openai stream: …— instead of the partial answer with a nilerror, so a cut stream cannot be mistaken for a finished one. That is also what the Ollama provider
has done since issue agent: the Ollama provider has no request timeout, so a stalled endpoint hangs the run #1.
openAIRequestErrornames the bound that fired and keepserrors.Is(err, context.Canceled)for acaller cancellation, which is how a TUI run renders as interrupted.
chatStreamtakes the watchdog as an argument; the parser-level tests passnil.CHANGELOG.md,CODEBASE.md.Why
#175 had to read past
finish_reasonto reach the usage chunk, and removed the early return therewithout replacing it. Measured against an endpoint that completes its answer and then holds the
connection open:
data: [DONE]"Hi",err=nil"Hi",err=nil"Hi",err=nilresp=nil,err=openai stream: context deadline exceededThe 1.5 s there is the caller's deadline in the repro; neither entry point passes one, so the real
wait was
openAIRequestTimeout— the whole 120 s. The worse half is the second column: a truncatedstream was reported as a successful answer.
Rejected alternative: stopping at
finish_reasononce usage has been seen. An endpoint that neverreports usage would then return at the wrong point and cut the flush that follows the marker, which
is the defect #175 fixed.
Verification
go test ./... -count=1→ 13 packages ok, 0 failed.go test ./... -count=1 -race→ same.make staticcheck(v0.8.1) → exit 0.gofmt -l .→ no output.The new gate was mutation-checked four ways:
a stream producing a chunk every 100ms failed: openai stream: stream went silent for 400msa stream that went quiet after finish_reason returned success with "Hi"on a cancelled context, error = http2: stream closed, want errors.Is(err, context.Canceled)The last row is why
openAIRequestErrorhas a direct unit test: on HTTP/1.1 the read error alreadywraps
context.Canceled, so removing the helper's classification does not change the streamingsubtest — the mutation survived until the helper itself was pinned with an HTTP/2-shaped error.
Honest scope
httptestservers.full idle bound (2 min) before failing. Distinguishing "the answer is complete but the sentinel
never came" from "the model is still thinking" would need a shorter post-
finish_reasonbound,which this change deliberately does not add.
[DONE]is still accepted as complete, deliberately:OpenAI-compatible servers vary, and failing their working answers would cost more than the
truncation it would catch.
[DONE]still ends the read early.than as partial content that looked like a finished answer. An interrupted run still reads as
interrupted (pinned above), but the transcript difference is real.
/tmp/dsh-repro-175(scratch module,replace→ this repository).