[Bug]: watch_pull_request can miss late required gates that pass between polls

[ryuudotgg]

Before submitting

• I searched existing issues and did not find a duplicate.
• I included enough detail to reproduce or investigate the problem.

Area

apps/server

Steps to reproduce

The following sequence reproduces the problem directly in evaluatePullRequestWatch, without starting a live watcher:

1. Evaluate a watch with passed: false against a PR containing:
   • Tests: required, successful.
   • Smoke Tests: not required, pending.
2. The evaluator reports checks-passed and returns a watch with passed: true.
3. Evaluate the returned watch again, keeping the same head SHA and supplying no new comments or conflicts. This time:
   • Tests: required, successful.
   • Smoke Tests: not required, successful.
   • Smoke Tests Gate: a newly appearing required check, already successful.
4. Observe that the evaluator returns an empty changes array.

This models a dependent required job being created and completing between polling passes. The watcher never observes that job pending.

Expected behavior

When a new required check appears, its successful completion should not be suppressed by the earlier notification for a smaller set of required checks.

The agent should receive an updated required-checks-passed notification even if the new check completed before the next poll.

This report does not request notifications for every optional check or ask the server to determine workflow readiness.

Actual behavior

No new notification is generated.

In apps/server/src/orchestration-v2/pullRequestWatch.ts, the successful-check notification depends on passedNow && !passed. Both values remain true when an additional required check first appears already successful.

I executed the actual evaluator and reproduced this result. As a control, observing the new gate pending before observing it successful does produce another notification.

The reported live incident involved ryuudotgg/forge#605, head 43fe2891e7ce6525818fc04b99677e63e3b74741:

• Install Smoke completed successfully at 2026-10-03T21:33:17Z.
• The required Install Smoke Gate was created at 2026-10-03T21:33:17Z.
• That gate completed successfully at 2026-10-03T21:33:22Z.
• The agent had already received a required-checks-passed notification and said it would resume once the remaining checks finished. No follow-up was observed before manually resuming it.

The five-second gate lifetime can fall between polling passes. Its timestamps and required status were verified, but the watcher's exact polling history for that incident was not captured.

Impact

Major degradation or frequent failure

Version or commit

0.0.46-nightly.20261003.2632

Environment

No response

Logs or stack traces

{
  "initialRequiredPass": [
    {
      "kind": "checks-passed",
      "count": 1,
      "required": true
    }
  ],
  "newRequiredGateAlreadySuccess": [],
  "requiredGateObservedPendingThenSuccess": [
    {
      "kind": "checks-passed",
      "count": 2,
      "required": true
    }
  ]
}

Screenshots, recordings, or supporting files

No response

Workaround

Manually resume the agent after the remaining checks finish.

[juliusmarminge]

Note

Grok responding on behalf of Julius.

Triage

Thanks for the write-up and for running the evaluator directly, @ryuudotgg! This is a real bug on current main (5e35272fda), and I didn't find a duplicate or a fix PR. The PR watch added in merged #15057 reports "required checks passed" from a single true/false flag. Because of that, a required check that first shows up already green gets dropped.

What happens

evaluatePullRequestWatch in apps/server/src/orchestration-v2/pullRequestWatch.ts (around line 63) treats the required set as passed when every required check is settled and none failed. It emits checks-passed only when that's true and watch.passed is false, then stores passed: true. That flag resets only when the head SHA changes, and it doesn't remember which required checks that report covered.

That matches your three evaluations exactly:

1. Only Tests is required, so the evaluator emits checks-passed with count: 1 and sets passed.
2. Smoke Tests Gate appears required and already successful, but passed is already true, so changes stays empty.
3. When the new gate is seen pending first, passed clears, and the next successful read emits count: 2.

PullRequestWatchReactor sweeps once a minute, so a gate job that GitHub creates after its dependencies and that finishes in a few seconds fits between passes. Your Install Smoke Gate window (created 21:33:17Z, successful 21:33:22Z UTC) is that case. The earlier wake isn't revised, and the agent was told to end its turn, so nothing resumes it. As you said, this isn't about optional checks or server-side readiness, which #15057 left to the agent.

Likely fix area

• One option is to remember the required check names covered by the last pass, like failedChecks remembers names. checks-passed would fire again when the settled required set includes a new name, even one that's already successful. ThreadPullRequestWatch.passed in packages/contracts/src/threadPullRequest.ts is only a boolean, so this would mean a new contract field.
• Existing passed: true watches without a name list could adopt the current required names on upgrade without a wake, so healthy watches don't all fire once.
• A test next to "reports passed once the required checks pass" in pullRequestWatch.test.ts could lock in both sequences you ran.

A required check with the same name that reruns and passes entirely between two polls would be a separate gap. Until a fix lands, manually resuming is the right workaround. A maintainer will decide on the fix direction.