Skip to content

[Fix] ACA Sandboxes wake-loop fix and improved suspend state management - #1018

Merged
mrubens merged 5 commits into
RooCodeInc:developfrom
tebieshi:fix/aca-sandboxes-sleep-wake-remediation
Aug 3, 2026
Merged

[Fix] ACA Sandboxes wake-loop fix and improved suspend state management#1018
mrubens merged 5 commits into
RooCodeInc:developfrom
tebieshi:fix/aca-sandboxes-sleep-wake-remediation

Conversation

@tebieshi

@tebieshi tebieshi commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Related issue

N/A

Why this PR exists

  • A maintainer explicitly invited this PR in the linked issue or discussion
  • I am a maintainer / this is internal Roomote work

What changed

Fixes an Azure Container Apps Sandboxes sleep/wake death loop: after a task was suspended (standby), every wake attempt died on "waking up" and the sandbox never stayed suspended.

Four interlocking defects, verified live against sandbox + DB event timelines:

  1. Zombie workers killed every wake. A suspended sandbox resumes with its worker process frozen mid-run, but the run is finalized (Completed) at suspend time, so the worker's run token is dead. The zombie kept running, held sandbox-server port 4200, and the next wake's worker died ~1 minute into boot — then the 2-minute stale-worker sweep suspended the sandbox mid-boot, and the cycle repeated on every wake.
    → Workers now terminate themselves when the server reports their run as finalized (heartbeat rejects with the terminal run-guard message), and resume launches reap leftover workers (pkill) before booting the new one.
  2. Retained-but-Running sandboxes were invisible. Sleep-check candidacy requires snapshotId IS NULL, so a retained standby sandbox left Running (failed wake, portal stop/start, legacy port wake) was never re-suspended.
    → Standby-retention (5-min cadence) now re-suspends azure retained handles whose instance reports running, guarded so it never touches a sandbox an active run owns.
  3. Ports woke sandboxes out-of-band. OnDemand activation resumes a suspended sandbox on any inbound traffic, but standby runs are always Completed at suspend time, so traffic-driven wakes could only orphan.
    → All Roomote-added sandbox ports now use Manual activation; wake is deliberate-only (wake prompt / message send).
  4. Deleted sandboxes retried forever. A 404 from getInstanceStatus (sandbox deleted out-of-band) went down the generic retry path and re-failed every minute indefinitely.
    → Definitive not-found now finalizes the run (idle → complete; otherwise fail/cancel).

How it was tested

  • New unit tests: sleep-check not-found finalization (404 → finalize, idle-complete, transient-error retry preserved), standby-retention orphan re-suspend (5 tests incl. active-session and genuine-snapshot guards), worker heartbeat terminal-run exit; azure contract test updated to Manual port activation.
  • Suites: compute-providers 210/210, bullmq 146/146, worker heartbeat 7/7; pnpm lint and pnpm check-types pass on all four touched packages.
  • Live validation on a production deployment after deploying this branch: wake button works (clean boot on the previously looping sandbox), task suspended to Stopped at the keepalive deadline, and the standby handle was recorded for the next wake.

Checklist

  • The PR title follows the repo convention: [Fix], [Feat], [Improve], [Refactor], [Docs], or [Chore] followed by a user-facing description
  • This PR is small and scoped to one change
  • pnpm lint and pnpm check-types pass locally
  • I added tests or included a clear manual validation note above
  • I removed secrets, tokens, private keys, and customer data from code, logs, and screenshots
  • If this change should appear in the changelog, I ran pnpm changeset

tebieshi and others added 3 commits August 3, 2026 01:03
- Finalize runs when Azure instance is missing instead of retrying
  forever
- Re-suspend orphaned Azure standby instances during retention job
- Kill stale worker processes before launching new Azure worker
- Terminate worker when heartbeat reports run finalized
- Use Manual port activation to prevent unintended sandbox wakes
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
standby retention logic, and remove redundant ESLint disable comments
for explicit any usage.
@roomote-community

roomote-community Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

No new code issues found. See task

  • packages/compute-providers/src/adapters/azure.ts:925: Manual activation is only sent while adding a new port. Existing ports return 409 and are accepted unchanged, while ports persist across stop/resume, so retained sandboxes created before this deployment keep OnDemand and can still be woken by ordinary inbound traffic. Migrate or recreate existing ports before treating the conflict as success.

Reviewed d760153

On a 409 from ports/add, remove and re-add the port with Manual
activation instead of silently accepting the conflict. Ports persist
across stop/resume, so legacy OnDemand ports keep waking suspended
sandboxes on any inbound traffic. Ports already on Manual are left
untouched.
@tebieshi
tebieshi marked this pull request as ready for review August 3, 2026 16:58
Copilot AI review requested due to automatic review settings August 3, 2026 16:58

@mrubens mrubens left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mrubens
mrubens merged commit 4e1c966 into RooCodeInc:develop Aug 3, 2026
14 checks passed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses an Azure Container Apps sandbox suspend/wake “death loop” by preventing out-of-band wakes, cleaning up zombie worker processes on resume, re-suspending orphaned retained standbys, and finalizing runs when the underlying Azure instance has been deleted.

Changes:

  • Switched Azure sandbox port exposure to Manual activation and added a conflict-time migration to recreate legacy OnDemand ports.
  • Made workers self-terminate when the control plane reports their run is finalized; added a pre-launch reaper for leftover workers on resume/restore.
  • Enhanced BullMQ jobs to re-suspend retained standbys found running without an owning run and to finalize runs on definitive Azure 404 instance-not-found errors (with tests).

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/compute-providers/src/adapters/azure.ts Change port activation to Manual and migrate legacy ports on conflict.
packages/compute-providers/src/tests/azure.contract.test.ts Update contract expectations and add tests for port-migration behavior.
apps/worker/src/run-task/polling/worker-heartbeat.ts Exit worker on terminal run-token guard heartbeat failure.
apps/worker/src/run-task/polling/worker-heartbeat.test.ts Add unit test ensuring terminal-run heartbeat triggers worker exit.
apps/controller/src/compute-providers/spawn-azure-worker.ts Reap leftover worker processes before launching a new worker on resume/restore.
apps/bullmq/src/scheduled-jobs/standby-retention.ts Re-suspend orphaned retained standbys that are running, with active-run guards.
apps/bullmq/src/scheduled-jobs/sleep-check.ts Finalize runs when Azure definitively reports instance-not-found (404).
apps/bullmq/src/scheduled-jobs/tests/standby-retention.test.ts New tests for orphan re-suspend behavior and guard conditions.
apps/bullmq/src/scheduled-jobs/tests/sleep-check.test.ts Tests for 404 finalization + transient error retry behavior.
apps/bullmq/src/jobs/snapshot.test.ts Update eslint-disable comment to match tooling behavior.
.changeset/aca-sandboxes-sleep-wake-remediation.md Changeset describing the end-user visible remediation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +74 to +77
if (
heartbeatError instanceof Error &&
heartbeatError.message.includes(RUN_TERMINAL_HEARTBEAT_MESSAGE)
) {
Comment on lines +412 to +417
.runCommand({
instanceId: machine.machineId,
cmd: 'pkill',
args: ['-f', '/sandbox/worker/dist/worker.js'],
signal: AbortSignal.timeout(15_000),
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants