Skip to content

fix(agent): ask again when a coder response leaks function-call markup instead of ending the task - #1875

Merged
diillson merged 2 commits into
mainfrom
fix/agent-unparsed-tool-call-recovery
Oct 23, 2026
Merged

diillson merged 2 commits into
mainfrom
fix/agent-unparsed-tool-call-recovery

Conversation

@diillson

Copy link
Copy Markdown
Owner

What

In coder mode, a response whose tool call came out as <invoke name=...>/<parameter name=...> markup (the function-call format some models fall back to), or as a run of stray </invoke>, carried no call chatcli recognizes. The loop took the text for the final answer and printed "Task completed" with the action never run.

Now the response is checked for that markup outside code fences and inline code. If it is there, the model is told that nothing was executed, shown the <tool_call> format, and asked to resend, at most three times in a row (the counter resets when a turn acts).

The markup is deliberately not converted into a call. Text that only looks like a call (quoted from a file or a tool result) must never run because it appeared in a response; the correction costs one turn and leaves every policy gate untouched.

Why

Found by reading the failures of a LongMemEval-V2 run: 33 of 484 coder attempts (7%) ended without the file the agent had been asked to write, and 29 of those 33 ended with exactly this markup as the final message. Among the 451 attempts that finished, 1 did.

Evidence

  • Offline, over the 484 saved final messages: the detector flags 29 of the 33 failures (the other 4 have no markup at all, a different cause) and 1 of the 451 successes. That one had already written its file, so it would cost one extra correction turn.
  • End to end with the real binary against a scripted fake model that leaks the markup first: the previous binary prints "Task completed" and the file does not exist; this one corrects, the model resends, the command runs and the file contains the expected text.
  • Unit tests cover the four shapes seen, the cases that must not fire (markup inside a fence or inline code, a valid tool call, prose, a single stray closer) and that the correction never echoes the markup. golangci-lint v2: 0 issues; go test ./cli/... passes.

…p instead of ending the task

A coder-mode response whose tool call was written as invoke and parameter
tags, or as a run of stray closers, carried no call chatcli recognizes. The
loop took it for the final answer and printed task completed with the action
never run, so a file the agent meant to write never existed.

The response is now checked for that markup outside code fences and inline
code, and when it is there the model is told that nothing was executed and
shown the tool_call format, up to three times in a row. The markup is not
converted into a call: text that only looks like one, quoted from a file or a
tool result, must never run because it appeared in a response.
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Quality Gate

Result: ✅ all floors passed

Floor Status Result Δ vs main Budget
1 · Build & Static ✅ go build / vet / fmt / lint — —
2 · Coverage ✅ 69.6% (bootstrap) 0 ≥ baseline
3 · Patch coverage ⚠️ 67.9% (req ≥ 60%) — ≥ 60%
4 · AI smells ✅ diff scanned — —
5 · Scope budget ✅ 3 files / 162 LOC (code 162 + tooling 0) — warn 800·25
6 · E2E ✅ go test -race ./e2e/... — ≤ 15min
7 · Commit lint ✅ conventional commits — —
8 · Cyclo (new code) ✅ 1 file(s) under threshold — ≤ 30
9 · Secrets scan ✅ gitleaks — —
10 · i18n parity ✅ missing 0, unknown 0 — —
11 · CRD drift ✅ drifted: 0 — —
12 · License headers ✅ 0 missing — —
13 · API breaking ✅ 0 incompatible — —
14 · Binary size ✅ chatcli 100.4MB · operator 55.4MB — 100MB each
15 · Provider parity ✅ 14 providers · 0 violations — —

Config: .github/quality-gate.yml. Workflow: .github/workflows/quality-gate.yml.

@diillson
diillson merged commit 5b8d280 into main Oct 23, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant