chore: add .claude/REVIEW.md with CI drift check#3561
Merged
Conversation
|
Contributor
WalkthroughThis PR adds a repository-specific review policy file (.claude/REVIEW.md) that defines "Important finding" criteria, lists items to always check and items to skip, and provides legacy/V1 non-blocking guidance and confidence-calibration notes. It also adds a pull-request-triggered GitHub Actions workflow (check-review-md.yml) that runs anthropics/claude-code-action on PR events, checks out the repo, reads the policy file, inspects changed files via git diff, and posts up to three targeted suggestions as a sticky comment. Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
.github/workflows/check-review-md.yml (1)
33-35: ⚡ Quick winCapture script exit code explicitly from the pipeline.
Line 35 currently relies on shell option behavior. Using
PIPESTATUS[0]makes the check robust and unambiguous.Suggested fix
- name: Run check id: check run: | set +e bash .claude/scripts/check-review-md.sh | tee check-output.txt - echo "exit_code=$?" >> "$GITHUB_OUTPUT" + echo "exit_code=${PIPESTATUS[0]}" >> "$GITHUB_OUTPUT"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/check-review-md.yml around lines 33 - 35, The current workflow captures the pipeline exit code via "$?" which can reflect the last command (tee) instead of the script; change the capture to use bash's PIPESTATUS to get the exit code of the first command (the check script). Specifically, after running "bash .claude/scripts/check-review-md.sh | tee check-output.txt" read the script's exit with PIPESTATUS[0] (or assign PIPESTATUS to a variable) and then write that value to GITHUB_OUTPUT so the workflow records the actual exit status of check-review-md.sh rather than tee.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.claude/scripts/check-review-md.sh:
- Line 10: The script currently has "set -uo pipefail" but not "set -e", and the
"cd \"$REPO_ROOT\"" call can fail silently; update the script so a failed cd
causes immediate exit — either enable "set -e" (e.g., change to "set -euo
pipefail") or make the cd call defensive like "cd \"$REPO_ROOT\" || { echo
'failed to cd to REPO_ROOT' >&2; exit 1; }" so subsequent commands never run
from the wrong directory.
- Around line 76-79: In resolve_glob_prefix(), the case that matches both '*'
and '?' uses a parameter expansion that only strips at '*' (variable s with the
expansion that currently targets '*' segments); update that expansion so it
strips from the first path segment containing either a '*' or a '?' instead of
only '*', e.g. use a shell parameter expansion that looks for a character class
containing both wildcard characters so paths like foo/ba?.ts correctly reduce to
foo.
---
Nitpick comments:
In @.github/workflows/check-review-md.yml:
- Around line 33-35: The current workflow captures the pipeline exit code via
"$?" which can reflect the last command (tee) instead of the script; change the
capture to use bash's PIPESTATUS to get the exit code of the first command (the
check script). Specifically, after running "bash
.claude/scripts/check-review-md.sh | tee check-output.txt" read the script's
exit with PIPESTATUS[0] (or assign PIPESTATUS to a variable) and then write that
value to GITHUB_OUTPUT so the workflow records the actual exit status of
check-review-md.sh rather than tee.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 0c109001-a3e0-45c2-aa90-6a3e4194e4b4
📒 Files selected for processing (3)
.claude/REVIEW.md.claude/scripts/check-review-md.sh.github/workflows/check-review-md.yml
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (29)
- GitHub Check: Analyze (javascript-typescript)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: units / e2e-webapp / 🧪 E2E Tests: Webapp
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: sdk-compat / Bun Runtime
- GitHub Check: sdk-compat / Node.js 22.12 (ubuntu-latest)
- GitHub Check: sdk-compat / Node.js 20.20 (ubuntu-latest)
- GitHub Check: sdk-compat / Cloudflare Workers
- GitHub Check: sdk-compat / Deno Runtime
🧰 Additional context used
🪛 LanguageTool
.claude/REVIEW.md
[grammar] ~9-~9: Ensure spelling is correct
Context: ...breakage.** Old and new versions of the webapp/supervisor run side-by-side during depl...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🪛 Shellcheck (0.11.0)
.claude/scripts/check-review-md.sh
[warning] 10-10: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
(SC2164)
🔇 Additional comments (1)
.claude/REVIEW.md (1)
1-50: LGTM!
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/check-review-md.yml:
- Around line 3-10: The workflow triggers on all PRs (except ignored paths) but
should only run when the REVIEW.md or the check script change; update the
pull_request trigger by adding an explicit paths: filter that lists
".claude/REVIEW.md" and ".claude/scripts/check-review-md.sh" so the job only
runs for PRs that modify those files (replace or add a paths: array next to the
existing types: block under on: pull_request).
- Around line 22-26: Remove the unnecessary permission entry "issues: write"
from the permissions block so only the required keys remain (e.g., keep
"contents: read", "pull-requests: write", and "id-token: write"); locate the
permissions mapping (the permissions: { ... } block) and delete the "issues:
write" line to follow least-privilege practice while leaving "id-token: write"
intact for OIDC use (the workflow uses use_sticky_comment: true which relies on
pull-requests: write).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: e0b42613-0aac-44df-9b33-645f68a48a4e
📒 Files selected for processing (1)
.github/workflows/check-review-md.yml
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (31)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: units / e2e-webapp / 🧪 E2E Tests: Webapp
- GitHub Check: sdk-compat / Cloudflare Workers
- GitHub Check: sdk-compat / Node.js 22.12 (ubuntu-latest)
- GitHub Check: sdk-compat / Bun Runtime
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: sdk-compat / Node.js 20.20 (ubuntu-latest)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: sdk-compat / Deno Runtime
- GitHub Check: typecheck / typecheck
- GitHub Check: audit
- GitHub Check: Zizmor
- GitHub Check: Analyze (python)
- GitHub Check: Analyze (javascript-typescript)
- GitHub Check: Analyze (actions)
🔇 Additional comments (2)
.github/workflows/check-review-md.yml (2)
28-44: LGTM!
46-84: LGTM!
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
.github/workflows/check-review-md.yml (1)
55-55: ⚡ Quick winParameterize the base branch reference.
The git diff command hardcodes
origin/mainas the base branch. While this works for the current repository, it reduces portability and would break if the PR targets a different base branch (e.g.,develop,v2, or a release branch).Use
${{ github.event.pull_request.base.ref }}to dynamically reference the actual base branch of the PR.♻️ Proposed fix to parameterize base branch
prompt: | You are auditing this PR for drift against `.claude/REVIEW.md`. ## Context `.claude/REVIEW.md` is the repo's source of truth for what AI / agent code reviewers should treat as critical findings (rolling-deploy safety, hot-table indexes, recovery-path queries, testcontainers usage, Lua versioning, etc.). It is consumed by review agents to calibrate severity. If REVIEW.md goes stale, every future agent review degrades. ## Your task 1. Read `.claude/REVIEW.md` in full. - 2. Run `git diff origin/main...HEAD --name-only` to see which files changed in this PR. + 2. Run `git diff origin/${{ github.event.pull_request.base.ref }}...HEAD --name-only` to see which files changed in this PR. 3. Sample the diff itself for any of these four signals:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/check-review-md.yml at line 55, Update the hardcoded base branch in the workflow step that runs the command 'git diff origin/main...HEAD --name-only' to dynamically use the PR's actual base branch; replace the literal 'origin/main' reference with the GitHub Actions expression for the PR base (using github.event.pull_request.base.ref) so the command becomes parameterized and works for PRs targeting develop/release/v2 branches.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/check-review-md.yml:
- Line 55: Update the hardcoded base branch in the workflow step that runs the
command 'git diff origin/main...HEAD --name-only' to dynamically use the PR's
actual base branch; replace the literal 'origin/main' reference with the GitHub
Actions expression for the PR base (using github.event.pull_request.base.ref) so
the command becomes parameterized and works for PRs targeting develop/release/v2
branches.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 92980e96-a62d-4c5e-b4f4-fa64d9eeef27
📒 Files selected for processing (1)
.github/workflows/check-review-md.yml
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: audit
- GitHub Check: Zizmor
- GitHub Check: audit
- GitHub Check: Analyze (python)
- GitHub Check: Analyze (actions)
- GitHub Check: Analyze (javascript-typescript)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2026-05-12T14:34:38.795Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 3561
File: .github/workflows/check-review-md.yml:3-10
Timestamp: 2026-05-12T14:34:38.795Z
Learning: In this repo’s `.github/workflows/check-review-md.yml`, the workflow is intentionally configured to run on *all* `pull_request` events (e.g., `opened`, `ready_for_review`, `synchronize`) and not only when `.claude/REVIEW.md` changes. The Claude Code audit compares each PR’s diff against `REVIEW.md` to detect contradictions and new undocumented patterns, so restricting the trigger to paths limited to `.claude/REVIEW.md` would undermine that coverage. Do not suggest narrowing the `pull_request` trigger to only REVIEW.md-related path changes.
Applied to files:
.github/workflows/check-review-md.yml
d-cs
approved these changes
May 12, 2026
2 tasks
matt-aitken
pushed a commit
that referenced
this pull request
May 12, 2026
…mpling (#3567) ## Summary Follow-up to #3561. The drift-audit workflow timed out on PR #3542 (92 files, +5962 lines) by hitting `--max-turns 15` before reaching a verdict, leaving a red ❌ on that PR with no sticky comment. ## Changes - `--max-turns` bumped from 15 to 30. - Prompt now opens with an explicit "Strategy" section: read REVIEW.md once, scan the file-list only, open at most 5 files (3-5 on PRs >50 files), and bias toward finishing over exploring. - Final rule: *"when in doubt between one more file read and finish now — finish now."* The audit is allowed to miss things. It is not allowed to time out and leave a red X. ## Test plan - [ ] Verify this PR's audit posts `✅ REVIEW.md looks current for this PR.` (small diff) - [ ] After merge, retry the audit on #3542 or a similarly large PR and confirm it completes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
.claude/REVIEW.md— a repo-specific source of truth for what AI / agent code reviewers should treat as critical in this codebase (rolling-deploy safety, hot-table indexes, recovery-path queries, testcontainers usage, etc.). Pairs with a Claude-based PR audit that flags drift between REVIEW.md and the code as it evolves.How the audit works
Mirrors the existing
.github/workflows/claude-md-audit.ymlpattern. On non-draft, non-fork PRs that touch code,anthropics/claude-code-actionreads REVIEW.md, samples the PR diff, and posts a sticky comment with up to 3 of:[stale]— rule cites a path / function / table that's been removed or renamed[contradiction]— code in the PR violates a current rule[missing]— PR introduces a new pattern future reviewers should know about[obsolete]— rule asserts a constraint the repo has moved pastIf nothing's off, posts
✅ REVIEW.md looks current for this PR.Test plan
head.repo.full_name == github.repository)