Skip to content

fix(task-board): duplicate check survives a reasoning model's answer - #7379

Merged
viktormarinho merged 1 commit into
mainfrom
fix/task-board-dedup-reasoning-models
Sep 18, 2026
Merged

viktormarinho merged 1 commit into
mainfrom
fix/task-board-dedup-reasoning-models

Conversation

@viktormarinho

@viktormarinho viktormarinho commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Second production follow-up to #7365 (#7376 made the outcome visible; this fixes what it revealed).

With duplicateCheck in the output, the throwaway-org test now showed skipped: unparseable answer from qwen/qwen3.5-flash-02-23 on two of three calls. The org's fast tier resolves to a reasoning model: it thinks before it answers, so the JSON arrives after a <think> block, after narration that may itself contain braces, or not at all within the 1200-token budget.

Changes

  • parseModelJson strips <think>…</think>, ignores code fences anywhere, and tries every top-level balanced {…} from the last one backwards, so narration before the answer never wins.
  • maxOutputTokens 1200 → 8000, room for the model to reason before the JSON.
  • The skip reason now carries finishReason and the answer length, so a truncation reads as finish=length instead of a generic "unparseable". The warn log gets the tail of the answer rather than the head.
  • Two parser tests: a <think> block with braces inside, and narration with an earlier JSON object before the real one.

Testing

28 tests in duplicate-check.test.ts, bun run check, bun run lint, bun run knip, bun run fmt. Will re-run the production scenario on the same org once deployed and report on this PR.


Summary by cubic

Fixes task-board duplicate check for reasoning models so their JSON answer is used instead of being skipped as unparseable.

Bug Fixes

  • parseModelJson strips thinking blocks, ignores code fences, and tries top-level JSON objects from last to first.
  • Raises the fast tier maxOutputTokens from 1200 to 8000, which may increase cost/latency when the extra budget is used.
  • Skip reasons now include the finish reason and answer length so truncation is distinguishable from malformed output.
  • Adds parser tests for a reasoning block with braces and narration containing an earlier JSON object.

Written for commit b5e621d. Summary will update on new commits.

Review in cubic

In production the fast tier resolves to a reasoning model that thinks before
it answers: the JSON arrived after a <think> block, after narration with
braces of its own, or not at all inside 1200 output tokens. Strip the think
block, take the last well-formed object, give it 8000 tokens, and put the
finish reason and length in the skip reason so the next miss explains itself.
@viktormarinho

Copy link
Copy Markdown
Contributor Author

Production data behind this PR, from the throwaway org:

  • With the org's default fast pick (a qwen flash reasoning model): 3/3 skipped: unparseable answer once the board had a dozen cards. Longer boards mean longer thinking, so the JSON never fits in 1200 tokens or arrives after a <think> block.
  • With the org's fast tier pinned to anthropic/claude-haiku-4.5 via ORGANIZATION_SETTINGS_UPDATE, on the code already deployed (fix(task-board): duplicate check answers via generateText and reports its outcome #7376, before this PR): 3/3 matched on paraphrases (EN↔PT, both directions), 2/2 no_match on related-but-distinct tasks (same area, different change).

So the judgement and the confidence gate are right; the remaining failure is purely the answer format of reasoning models on the default pick, which is what this PR addresses. Orgs that pin a non-reasoning fast model already work without it.

@viktormarinho
viktormarinho merged commit fe582a7 into main Sep 18, 2026
33 checks passed
@viktormarinho
viktormarinho deleted the fix/task-board-dedup-reasoning-models branch September 18, 2026 19:47
decocms Bot pushed a commit that referenced this pull request Sep 18, 2026
PR: #7379 fix(task-board): duplicate check survives a reasoning model's answer
Bump type: patch

- decocms (apps/api/package.json): 4.382.0 -> 4.382.1
- @decocms/native (apps/native/package.json): 4.382.0 -> 4.382.1

Deploy-Scope: server
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant