Skip to content

fix(grillme): pin each turn to the triggering PR - #10

Merged
johnmcollier merged 1 commit into
mainfrom
fix/grillme-pin-trigger-pr
Aug 21, 2026
Merged

johnmcollier merged 1 commit into
mainfrom
fix/grillme-pin-trigger-pr

Conversation

@johnmcollier

Copy link
Copy Markdown
Contributor

Summary

Turn 2 on #7 did three related wrong things, all from one cause:

  1. PR_NUMBER was unset inside the sandbox Bash tool (pre-script sets it on the runner; that does not enter the sandbox).
  2. The agent ran gh pr list and grilled fix(grillme): require a recommended answer on inline questions #9 (most recently updated) instead of the triggering PR.
  3. It queried fix(grillme): require a recommended answer on inline questions #9's review threads, saw none, called itself Turn 1, and emitted inline comments on .fullsend/custom/agents/grillme.md — files that are not in docs: OpenSpec for task status workflow #7's diff. GitHub returned 422; the post-script fell back to this issue comment, including chain-of-thought.

The unanswered thread on #7 was never seen because the agent was not looking at #7.

Fixes

  • Prompt: resolve the PR from GITHUB_ISSUE_URL first. Never gh pr list to choose a target. Existing <!-- grillme --> threads mean this is not turn 1.
  • Post-script: strip preamble before the Grillme heading; if inline comments 422, say so instead of claiming they were pinned.
  • RUN_URL falls back to STATUS_RUN_URL so the footer is not empty.

May conflict with #9 (both edit grillme.md). Merge #9 first if both are still open.

Test plan

Made with Cursor

Turn 2 on #7 listed open PRs after PR_NUMBER was unset in the sandbox
Bash tool, then grilled #9. Inline comments 422'd (wrong files) and
landed as an issue comment that claimed Turn 1.

- Resolve the target from GITHUB_ISSUE_URL; never gh pr list to pick
- Count existing <!-- grillme --> threads before calling it turn 1
- Drop chain-of-thought before the Grillme heading
- Surface a warning when GitHub rejects inline comment paths

Co-authored-by: Cursor <cursoragent@cursor.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 21, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 7:39 PM UTC · Completed 7:55 PM UTC

Commit: 4a3eb43 · View workflow run →

@johnmcollier
johnmcollier merged commit 1825ea7 into main Aug 21, 2026
10 checks passed
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 21, 2026 •

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 7:41 PM UTC · Completed 7:52 PM UTC

Commit: 4a3eb43 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #10 — fix(grillme): pin each turn to the triggering PR

What happened

PR #10 is a human-authored fix (co-authored with Cursor) for a bug in the repo's custom grillme agent. During the third /fs-grillme invocation on PR #7, the agent grilled the wrong PR (#9 instead of #7) because PR_NUMBER was not available inside the sandbox.

Root cause: The grillme harness config declares GITHUB_ISSUE_URL, GH_TOKEN, and REPO_FULL_NAME in env.sandbox, but not PR_NUMBER or ISSUE_NUMBER. The pre-script sets those via $GITHUB_ENV (runner scope), which does not propagate into the sandbox. Without a PR number, the agent fell back to gh pr list, picked the most recently updated PR (#9), posted inline comments on files not in #7's diff, got 422 errors, and the post-script fell back to an issue comment that leaked chain-of-thought.

The fix is comprehensive — five defensive layers:

  1. Prompt pins target PR to GITHUB_ISSUE_URL (available in sandbox)
  2. Explicit guardrail: "Do not gh pr list to pick a target"
  3. Turn detection via existing <!-- grillme --> threads
  4. Post-script strips chain-of-thought preamble before the Grillme heading
  5. Post-script warns when inline comments 422 (paths not in diff)

Workflow assessment

This was a well-executed incident response. The human diagnosed the root cause correctly, implemented a multi-layered fix, and merged quickly. The review agent (run 32519516024) started but the PR was self-merged within ~80 seconds with no completed reviews.

Existing issue coverage

The underlying patterns are already tracked in fullsend-ai/fullsend:

  • Env var propagation to sandbox: Multiple open issues cover sandbox env var access (#2936, #3373, #5162). This incident provides concrete evidence of a custom agent failing because pre-script-set vars don't enter the sandbox.
  • Review timing vs. merge: Issues #4509, #4375, #6040 cover scenarios where the review agent doesn't complete before a PR is merged. This PR was merged ~80 seconds after opening.
  • Wrong PR targeting: Issue #1326 covers dispatch passing wrong PR numbers. This incident is a variant where the agent itself (not dispatch) selected the wrong PR.

No new proposals are warranted — the fix is already merged, the root cause is architectural (sandbox env var boundary), and the prompt-level workaround using GITHUB_ISSUE_URL is the correct approach given the harness config's env propagation model.

@fullsend-ai-review

Copy link
Copy Markdown

Review skipped — this PR is already merged.

The /fs-review command only reviews open pull requests.

Posted by fullsend post-review check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant