Repository navigation
fix(server): a hung Codex app-server no longer blocks every later Codex run - #26555
MahirAbbas wants to merge 1 commit into
Conversation
…ex run All Codex threads on one instance share one app-server session. If that app-server hangs before it answers `initialize`, the request waits forever while it holds the session's initialize permit. Every later Codex run reuses the same live session and waits behind it, so runs stay in "starting" until T3 restarts. Changing the Codex binary path does not help, because the live session is reused. `initialize` now times out after 30 seconds. On timeout the adapter fails the session's event stream, which makes the session manager retire the session and stop its app-server. The next run spawns a fresh app-server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Initialization timeout and failure handling apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.ts, apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.test.ts |
The adapter applies a 30-second timeout to initialization. If it expires, the adapter fails the event stream and initialization request with a session-scoped error. The regression test checks both failures. |
Priority:
Estimated code review effort: 2 (Simple) | ~10 minutes
Change: Bug fix
Possibly related issues
- pingdotgg/t3code#22422: Both changes fail the event stream when a Codex app-server does not respond, addressing session lifecycle handling.
Suggested labels: bugfix, size:S
Suggested reviewers: juliusmarminge, macroscopeapp
Merge Risk: 
Minimal · up to 31398
-
Hung sessions can now be retired and replaced. Runs already queued on the old session may still time out sequentially. No merge-blocking issue was established.
Pre-merge checks: 
4
Finishing Touches: 💡 1
-
🛠️ Fix failing CI checks: 💡
-
- Commit to this branch
- Create a new PR
🧪 Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
Problem
All Codex threads on one instance share one
codex app-serversession. If that app-server hangs before it answersinitialize, the request waits forever while it holds the session's initialize permit (CodexAdapterV2.ts,ensureInitialized). Every later Codex run reuses the same live session (ProviderSessionManager.openreturns the existing entry) and waits behind the permit. The runs stay in "starting" withstartedAt: nulluntil T3 restarts. A new Codex binary path does not help, because the live session is reused. Status probes use fresh processes, so Codex looks healthy the whole time.This is a small, focused fix for an obvious hang: a request with no timeout that holds a shared lock.
How I established it
This happened on T3 Code Nightly 0.1.2-nightly.20261021 on macOS. A hung
syspolicydmade every new exec ofcodexstop at_dyld_start. The server traces show:CodexAdapterV2.opensucceeded). Its firstCodexAppServerProtocol.requestnever got an answer.CodexAdapterV2.openran again until the restart at 22:16. Three delegated Codex runs and two other runs each stopped at the firstCodexAppServerProtocol.requestinProviderTurnStartService.loadFromProvider. They waited 64 s to 414 s, and each one ended only when it was cancelled.cbea0d64sent its request 3 ms after the previous waiter was cancelled.Claude runs kept working the whole time, so the effect worker was not blocked.
Fix
initializenow times out after 30 seconds. It normally answers in under a second. On timeout, the adapter fails the session's event stream, the same signal #26188 uses when the app-server exits. The session manager then retires the session and closes its scope, which stops the hung app-server. The waiting run fails instead of staying in "starting", and the next run spawns a fresh app-server with the current settings.Open PR #26253 (one app-server per thread) would limit this hang to one thread. It does not add a timeout, so that thread would still hang.
Verification
retires the shared session when initialize never answersinCodexAdapterV2.test.ts. A replay app-server never answersinitialize. The test checks thatensureThreadfails after 30 s (TestClock) and that the session event stream fails.vp test run src/orchestration-v2/Adapters/CodexAdapterV2.test.ts: 109 passed.tsc --noEmitforapps/server, plusvp lintandvp fmt --checkon the two changed files: clean.Not checked: I did not reproduce the hang end to end in a running T3 instance.
Model: Claude Opus 5.5 in Claude Code, running in T3 Code.
🤖 Generated with Claude Code