Skip to content

fix(grillme): require a recommended answer on inline questions - #9

Merged
johnmcollier merged 1 commit into
mainfrom
fix/grillme-inline-recommendations
Aug 21, 2026
Merged

johnmcollier merged 1 commit into
mainfrom
fix/grillme-inline-recommendations

Conversation

@johnmcollier

Copy link
Copy Markdown
Contributor

Summary

Grillme's inline threads on #7 asked good questions but never took a stance. The original one-question model always included Recommended answer; that was lost in the thread rewrite.

This restores it:

  • Each new inline comment is question + **Recommended:** <answer and why>
  • Follow-up thread replies that still ask a decision question do the same
  • Omit the recommendation only when there is genuinely no technical default (a values/priority call)

Prompt + grilling skill both updated. Example comment bodies in the agent output contract now include the recommendation line.

After merge, a new /fs-grillme session on a PR will pick this up (workflow reads main). Existing threads on #7 will not be rewritten; start a fresh session or continue and look for recommendations on new/follow-up comments.

Test plan

Made with Cursor

The original grilling skill always paired each question with a
recommended answer. The inline-thread rewrite dropped that, so turn-1
comments on #7 were probes with no stance.

Restore **Recommended:** on new inline comments and on follow-up
replies that still ask a decision question. Omit it only when the
agent truly has no technical default.

Co-authored-by: Cursor <cursoragent@cursor.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 21, 2026 •

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 7:22 PM UTC · Completed 7:35 PM UTC

Commit: 88b3ce5 · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Low

  • [internal consistency] .fullsend/custom/skills/grilling/SKILL.md:30 — SKILL.md step 2 ("Reply in-thread when an answer needs a follow-up") is not updated to mention including **Recommended:** on follow-up replies, while grillme.md explicitly adds this rule at lines 225–226: "When the follow-up is itself a decision question, include **Recommended:** the same way as new comments." The two documents now diverge on follow-up behavior. In practice the agent definition (grillme.md) is the primary runtime document and does contain the instruction, so the risk is low.
    Remediation: Add a clause to SKILL.md step 2 such as: "When a follow-up is itself a decision question, include **Recommended:** <answer and why> the same way as new comments."

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: The following inline comments could not be posted on the diff (GitHub returned 422) and are included here instead:

  • .fullsend/custom/skills/grilling/SKILL.md (file-level): Line 30 · [low] internal consistency

SKILL.md step 2 ('Reply in-thread when an answer needs a follow-up') is not updated to mention including Recommended: on follow-up replies, while grillme.md explicitly adds this rule at lines 225-226. The two documents now diverge on follow-up behavior. In practice the agent definition (grillme.md) is the primary runtime document and does contain the instruction, so the risk is low.

Suggested fix: Add a clause to SKILL.md step 2 such as: 'When a follow-up is itself a decision question, include Recommended: the same way as new comments.'

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Aug 21, 2026
@johnmcollier
johnmcollier merged commit aa08686 into main Aug 21, 2026
30 checks passed
@fullsend-ai-retro

fullsend-ai-retro Bot commented Aug 21, 2026 •

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 7:41 PM UTC · Completed 7:48 PM UTC

Commit: 88b3ce5 · View workflow run →

@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #9 — fix(grillme): require a recommended answer on inline questions

What happened

PR #9 was a human-authored fix (johnmcollier, with Cursor) to the grillme agent configuration. It restored the **Recommended:** answer requirement on inline questions after real-world testing on PR #7 showed the grillme agent asked good probing questions but never took a stance — the author even replied "I'm not sure, what do you think?" expecting the agent to have an opinion.

The review agent (run 32518019927, fullsend v0.36.0, agents from fullsend-ai/agents@6bdcab69) ran four sub-agents (correctness, style-conventions, intent-coherence, docs-currency) plus an adversarial challenger. It approved the PR with one low-severity finding: SKILL.md step 2 was not updated to mention **Recommended:** on follow-up replies, creating a consistency gap with grillme.md. The PR was merged ~19 minutes after creation without addressing this finding.

Workflow quality

  • Review quality: Good. The review agent caught a real consistency gap, rated it appropriately as low-severity, and produced zero false positives.
  • Rework rate: Zero. Single commit, single review iteration, merged. The unaddressed finding creates a small latent consistency issue.
  • Token cost: Slightly elevated. The sonnet model was unavailable on GCP Vertex, forcing 3 of 4 sub-agents to re-dispatch on opus. This added ~3 minutes and higher per-token cost. The agent handled the fallback gracefully.
  • Time to resolution: 19 minutes from PR creation to merge, with ~14 minutes spent on the review agent.

Observations on existing tracked issues

What went well

The overall workflow was healthy: real-world testing (grillme on PR #7) surfaced a UX gap, a human authored a targeted fix, and the review agent provided useful feedback. The review agent correctly identified the one remaining consistency issue in a 2-file, 24-line change to agent configuration files — solid performance on a meta-review task (reviewing agent definitions).

Proposals filed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-merge All reviewers approved — ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant