fix(process): Stop actually kills the dev server (#90) - #136
Merged
Merged
Conversation
…rting success (#90) Stop signalled only the tracked pid, which under shell:true is the sh -c wrapper, not the real dev server. POSIX sh doesn't forward SIGTERM to its child, so the real process (e.g. next-server) survived and kept the port bound while stopProject returned { success: true } unconditionally. The ss/kill -9 fallback only ran on a thrown error, which never happened, so it was dead code. - startProject now spawns with detached: true so the child is its own process-group leader; stopProject signals the whole group via process.kill(-pid, ...) instead of just the wrapper. Windows has no negative-pid group signalling, so it degrades to single-process kill there. ESRCH (group already gone) is treated as already-stopped, not an error. - stopProject is now async: SIGTERM -> poll checkPort -> escalate to SIGKILL -> poll again -> only then decide. It returns success: false with an error if the port is still bound after both escalations, instead of lying. - The ss/kill -9 port-based fallback is now a real path that always runs when the tracked route didn't verifiably free the port, including when there's no tracked entry at all (an orphan from before hexops started, or from an earlier crash -- today's incident). - Fixed a latent stoppingProjects leak: it was being marked unconditionally even when there's no tracked ChildProcess to ever fire the 'close' event that clears it, which would permanently disable crash notifications for that projectId after an orphan stop. Now only marked right before signalling a live tracked entry. - Updated the one production caller (stop route) and DevServerGuardDeps.stop to await the new async signature; runWithDevServerGuard was already async/await throughout so this was a one-line change. Extended process-manager.test.ts with 9 new tests (mocked spawn/execFileSync/ checkPort, fake timers for the escalation windows -- no real process spawned or port bound). Verified separately against a real process: full report in stop-fix-report.md, including a real-process transcript showing the tracked sh+node pair on a scratch port is fully gone and the port released after stopProject, with hexops' own dev server on port 3000 left untouched. 382 -> 391 tests, 46 files, all passing. tsc --noEmit clean.
…C2/I1-3/M1-4) Independent review of 4e8c550 confirmed the core group-kill + port-verified success shape is correct, but found two critical issues and three important ones, plus four minors. All addressed: - C1: one test in process-manager.test.ts had no mock on process.kill, so after the previous test's afterEach restored all mocks, its process.kill(-pid, ...) calls hit the real syscall against a fabricated pid. Fixed structurally: process.kill is now spied in the stopProject describe block's own beforeEach with a safe no-op default, so no test in the file can reach the real syscall even if it forgets to stub it itself. - C2: a real regression in the previous commit. stoppingProjects was only marked when the tracked entry was "live" (exitCode === null), but with shell: true a grandchild holding the stdio pipes (and the real port) can keep 'close' pending well after the direct child looks exited -- exactly the #90 shape. That case fell through unmarked, producing a false "crashed" notification and, with restartOnCrash on, relaunching a server the user had just stopped. Now marked whenever a tracked ChildProcess exists at all, not only when it's live. - I1: stopByPort now refuses to kill process.pid or process.ppid (logs and skips instead), since it's newly reachable from a tracked stop whose group died but whose port is held by something else, not just the pre-existing fully-untracked-orphan case. - I2: runWithDevServerGuard now aborts (blocked: true) before running the operation if the stop failed, instead of silently running an install against a server that may still be live -- unreachable before #90 since stop effectively always "succeeded", real now. - I3: detached: true (the #90 fix) severs tracked children from hexops's own process group and session, so Ctrl-C/terminal-close no longer reaps them the way they did when children stayed attached. Added shutdownTrackedProcesses(), wired to real SIGINT/SIGTERM outside test runs (guarded so importing this module under vitest never installs a handler that could process.exit() a test worker). - M1: close handler now guards activeProcesses.delete on process identity, so a late close from a superseded child can't untrack its replacement. - M2: reverted checkPort's timeout back to its 1000ms default -- a shortened per-attempt timeout was a false-success path in the function meant to eliminate them (checkPort resolves false on its own connect timeout too). - M3: the tracked-success path now also confirms the process itself exited (short grace window + a defensive final SIGKILL if not), not just the port, since we're already holding a live ChildProcess handle. - M4: exported signalProcessGroup for direct testing and added ESRCH-vs-EPERM unit tests plus an integration-level EPERM test -- the prior ESRCH test only proved the overall stop succeeded via mocked checkPort, which passed identically whether ESRCH was actually swallowed or just caught upstream. 391 -> 399 tests, 46 files, all passing. tsc --noEmit clean. Re-verified against a real process (same scratch-port harness as the original commit): stopProject succeeds, port released, tracked pid and all real child pids confirmed dead, hexops' own port-3000 server untouched throughout. Full writeup appended to stop-fix-report.md.
alamb-hex
added a commit
that referenced
this pull request
Aug 11, 2026
A 24KB implementation working-note was committed alongside the #90 fix. It is scratch material — review correspondence and verification transcripts — not a repo artifact, and docs/ is where anything durable belongs. Removing rather than relocating: the content it preserves (review findings and their reproductions) is captured in PR #136's description and in the commit messages themselves.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #90 — "Stop returns success while next-server keeps running."
The bug, in four layers
startProjectspawned with{ shell: true, detached: false }, sochild.pidwas thesh -cwrapper, not the realnext-server.stopProjectsentSIGTERMto that wrapper. POSIXshdoesn't forward signals, so the server survived and kept its port.{ success: true }having verified nothing.ss/kill -9fallback only ran if the tracked kill threw, which it never did — dead code in the common case.Observed live yesterday: two orphaned
sh -c node/node server.jspairs held port 3000, a plain SIGTERM couldn't reach them, and a fresh start failed withEADDRINUSE.The fix
detached: trueso the child is a process-group leader.process.kill(-pid, …)reaches the whole tree. Windows degrades to a single-process kill.checkPortafter signalling, escalate SIGTERM → SIGKILL, and returnsuccess: falsewith a clear error if the port is still bound. Reporting success while a server lives was the entire bug.stopProjectis now async; its caller andrunWithDevServerGuardupdated.Also fixed along the way:
stoppingProjectsleaked permanently for untracked ids, suppressing the next genuine crash notification for that project.Review found five blockers, all fixed
Two were serious enough to call out:
process.kill(-6666, 'SIGKILL')for real. It asserted onlysuccess === false, which held whether the signal hit nothing or destroyed an unrelated process tree — so it could never fail because of it. On a box running ~35 dev servers that's a live hazard.process.killis now mocked structurally in a sharedbeforeEach; verified with a tripwire config that traps the real syscall — 31 tests, zero hits.stoppingProjectswas gated on "is the child live" rather than "does a tracked child exist". Withshell: truethe wrapper can exit while a grandchild holds the pipes and the port — the P1: Stop returns success while next-server keeps running — orphans accumulate #90 shape itself — so'close'fired withintentional: false, and withrestartOnCrashenabled it relaunched the server the user just stopped. Verified by reverting the guard in a temp copy and watching the new test fail with exactly that notification.Plus:
stopByPortnow refusesprocess.pid/process.ppid(a project misconfigured to :3000 would have made HexOps kill its own server);runWithDevServerGuardaborts rather than runningpnpm installagainst a still-live server (newly reachable, since a failed stop is now a real outcome); and aSIGINT/SIGTERMhandler reaps tracked children, becausesetsid()severed the foreground group that used to do it.Verification
399 tests / 46 files,
tsc --noEmitclean,pnpm buildexit 0.Real processes, not mocks — SIGKILL escalation against a SIGTERM-ignoring server on a scratch port:
stopProjectreturned success after 4.2s, port free, all 8 group pids dead.Known follow-ups (issues to file)
shell: trueis theshwrapper — killed instantly by the group SIGTERM, so the guard always passes. A server that closes its listener then hangs returnssuccess: truewith the port free and the process still alive. Only affects servers that trap SIGTERM (a stock Next dev server doesn't), but it's a narrower instance of P1: Stop returns success while next-server keeps running — orphans accumulate #90.override-removedoesn't checkblocked, so a failed guard-stop surfaces as success there.finally.shell: falseprojects are force-killed 500ms after releasing their port.