Skip to content

server: service restart mid-turn resumes the in-flight turn twice (duplicated assistant messages + concurrent runs) #57582

Closed as not planned

Description

@TomiJerez

Summary

When the background service restarts while an assistant turn is in flight, the new server resumes the turn twice: two identical assistant messages (same text, same tool calls, different tool-call IDs) get persisted, and two concurrent runs execute the same tool calls in parallel. The corrupted-looking history later makes strict providers reject the rebuilt request with HTTP 400, and the session can end up with two different assistant answers to a single user message.

Environment

  • opencode version: v2.1.1 (opencode --version)
  • OS: Windows_NT 10.0.26300 (win32 x64)
  • Terminal: TERM=xterm-256color
  • Shell: cmd.exe (%COMSPEC%), PowerShell 7.6.7 in use
  • Install/channel: latest, local=false
  • Active plugins: one local auto-discovered plugin (~/.config/opencode/plugins/model-fallback.ts, a session retry hook that switches models on provider errors). It is not believed to be the cause: the duplication happened before any model error; the plugin only handled the resulting 400 afterwards.

Reproduction

Intermittent (observed once, sequence fully captured):

  1. Have a session with an assistant turn in flight (assistant message streaming with tool calls being executed).
  2. Restart the background service while the turn is running (opencode service start after the previous server process was killed; also observed right after a Session failed ... Failed query: select "hash", "value" from "instruction_blob" ... on the dying old server).
  3. The new server picks the session back up.

Expected Behavior

The in-flight turn should be resumed at most once: one run continues, no duplicated assistant messages are persisted.

Actual Behavior

  • Two near-identical assistant messages were persisted in the same second (same reasoning/text and the same two shell tool calls with identical arguments, but different tool-call IDs, one with re-ordered JSON keys — looks like a re-execution of the same turn).
  • The server log shows two different run IDs spawning the exact same tool commands concurrently (two runs for one session).
  • The user's next message then triggered a model request that was rejected by a strict provider: HTTP 400 provider.invalid-request: messages[38].tool_calls[0].function.arguments must be a valid JSON object string (all tool-call inputs persisted in the DB are valid JSON objects, so the malformed payload appears during request rebuilding of the duplicated history).
  • Meanwhile the parallel run's request with the same history succeeded, so the session ended up with two assistant answers to one user message (one finish=stop from the original model, one with tool calls from a fallback model), streaming in parallel for ~45 seconds. The user had to switch models manually to continue.

Additional Context

  • After the restart, PRAGMA integrity_check / quick_check on the SQLite DB are ok and the previously failing instruction_blob query works, so no database corruption — the "Failed query" error was the old server's connection dying mid-read, not a broken DB.
  • Timeline (UTC, from DB and opencode.log):
    • 00:40:55 old turn runs service start (restart triggered)
    • 00:41:00 new server up; duplicated assistant messages created (seq N and N+2, same content)
    • 00:41:53 user sends next message
    • 00:41:54 run A's request → HTTP 400 (malformed tool_calls in rebuilt request); run B's request succeeds at the same second
    • 00:41:55–00:44:09 both answers stream concurrently and both persist
  • Session/model details available on request; session IDs, prompts, and tool contents redacted here.

Activity

  1. opencode-agent commented on Oct 21, 2026

    @opencode-agent
    Contributor

    Thanks for the detailed timeline, @TomiJerez. I tried to reproduce this on v2.1.1 (Linux, background service, a mock model), but couldn't:

    • Killing the service with kill -9 during a long shell tool call and then running opencode service start: the new service added one "The server restarted while you were working…" notice and resumed the turn once.
    • Having the turn's own tool call kill the service and then run opencode service start, as in your sequence: again one resume, one assistant message and no concurrent runs, even when the resumed turn made new tool calls.

    Two runs on one session usually means two server processes were driving it at the same moment. Your log shows the old server still running a database query (Failed query … instruction_blob) right before the new one came up, so the old process may still have been alive when the new service resumed the session. That would be the same problem as #51213. Some details would help us tell:

    • The exact commands the turn ran around 00:40:55: how the old server was stopped (e.g. taskkill, opencode service stop, or a crash) and whether service start came from the agent's shell tool.
    • Whether anything else had a server on the same data directory at that time, e.g. the desktop app, an opencode serve, another TUI or opencode run --standalone.
    • The opencode.log lines from roughly 00:40:50 to 00:41:05 (redacted is fine), especially lines that show which process each run ID came from and any resume, restart or drain messages from the two servers.

    Related issues, ordered by likelihood:

  2. opencode-agent commented on Oct 24, 2026

    @opencode-agent
    Contributor

    Closing this for now, @TomiJerez, since we haven't heard back. On v2.1.1 the background service resumed an interrupted turn only once in every restart we tried, including when the turn itself killed the service and ran opencode service start. So we still need the details asked for above to find what caused two runs in your case: how the old server was stopped, whether another server or client shared the same data directory, and the opencode.log lines from around 00:40:50–00:41:05.

    If it happens again, please reply here with those details and we can reopen it. If it turns out a second server was still running, #51213 covers that case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions