Skip to content

commit-msg guard: a quoted skip-ci marker suppresses CI with no artifact, and an empty check list reads as green — refuse the marker mechanically (#194) - #213

Merged
martin-conur merged 2 commits into
mainfrom
task/skip-ci-guard
Sep 15, 2026
Merged

martin-conur merged 2 commits into
mainfrom
task/skip-ci-guard

Conversation

@martin-conur

Copy link
Copy Markdown
Owner

Closes #194.

GitHub scans the ENTIRE head-commit message for its CI-skip markers, not just
the subject line. An agent that merely quotes one while describing another
commit suppresses its own workflow run. It happened twice in one hour to two
different workers; the second was correctly explaining the first, in the
commit message that killed its own CI. (This commit message is written with the
token broken for exactly that reason — and the guard it adds checked it.)

The failure leaves no artifact. The marker-bearing commit 06afcf1 was the PR
head for nine minutes and never appears in gh run list at all: not queued,
not skipped-with-a-record, not cancelled. Absence is indistinguishable from
"not looked at yet" when scanning a PR, which is why both workers reported
"CI green" in good faith off a check list that was empty rather than passing.
It also defeats the natural mitigation — "check that CI passed" cannot catch
this, because there is nothing to check. Only "does a run exist for this exact
SHA" can.

So the primary fix is mechanical, per the PM note on the ticket: a prompt
warning is demonstrably insufficient when the agent understands the mechanism
and is writing about it.

  • New canonical bin/ci-guard, symlinked by all seven installers: scans a
    message (scan), a commit or range (check), or runs as a git hook
    (commit-msg). Comment lines and the scissors tail are skipped — git strips
    both before storing the message, and commit --verbose routinely puts the
    marker there when the PR is about the marker.
  • All seven task-work copies install it as the repo's commit-msg hook, via
    a new drift-guarded ci-guard-hook region plus lib/ci-guard.sh. Git shares
    one hooks dir across a repo's worktrees, so it covers every commit in the
    repo, not just the worktree it was installed from — 4c3e11f, the sync commit
    that triggered both occurrences by being described, was made on main.
  • Never clobbers: a pre-existing commit-msg hook is preserved as
    commit-msg.local and chained; both names taken means skip-with-warning
    rather than guess. Idempotent. No-ops when ci-guard is off PATH, so
    uninstalling task-force cannot lock you out of committing. Any write failure
    degrades to a warning — task-work is set -e and must not abort over this.
  • The refusal names its own escape hatches, because writing about a marker is
    the legitimate case every affected ticket requires: break the token, or
    --no-verify, or TASK_FORCE_NO_CI_GUARD=1. A guard with no documented way to
    write the token gets bypassed or reverted.

Defence in depth, not the mechanism:

  • All 8 worker prompts (5 commands/worker.md + 3 kiro agents/worker.json) gain
    a "CI markers" item and a strengthened "Green" item: an empty check list is
    not a pass, and gh run list -c "$(git rev-parse HEAD)" returning 0 means
    green is not a claim you can make. Wording is byte-identical across loadouts
    and stays repo-generic per worker.md: pre-PR checklist (CHANGELOG, doc-beat, reuse, spec-comments, green gate) (#176) #177.
  • The 6 reviewer prompts get the same empty-vs-passing distinction against the
    PR's headRefOid — a reviewer cross-checking "CI green" off an empty list
    repeats the bug at one remove.

Tests: new tests/ci_guard.bats (33) covers all five markers case-insensitively,
comment/scissors stripping, the documented escape hatch passing, refusal
messaging, hook install (idempotent / foreign-hook chaining / core.hooksPath /
linked worktree / unwritable dir), and end-to-end refuse + bypass. worker_checklist
gains the 8-prompt and 6-reviewer assertions and widens the drift window so the
new fenced proof-of-run command is compared too. Full suite 1095 ok / 0 not ok,
drift check and shellcheck -x green.

Upgrading: re-run ./install.sh <loadout> for the new ci-guard symlink, then
task-init <loadout> for the refreshed prompts and steering docs.

Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com

martin-conur and others added 2 commits September 15, 2026 09:55
…act, and an empty check list reads as green — refuse the marker mechanically (#194)

Closes #194.

GitHub scans the ENTIRE head-commit message for its CI-skip markers, not just
the subject line. An agent that merely *quotes* one while describing another
commit suppresses its own workflow run. It happened twice in one hour to two
different workers; the second was correctly *explaining* the first, in the
commit message that killed its own CI. (This commit message is written with the
token broken for exactly that reason — and the guard it adds checked it.)

The failure leaves no artifact. The marker-bearing commit 06afcf1 was the PR
head for nine minutes and never appears in `gh run list` at all: not queued,
not skipped-with-a-record, not cancelled. Absence is indistinguishable from
"not looked at yet" when scanning a PR, which is why both workers reported
"CI green" in good faith off a check list that was empty rather than passing.
It also defeats the natural mitigation — "check that CI passed" cannot catch
this, because there is nothing to check. Only "does a run exist for this exact
SHA" can.

So the primary fix is mechanical, per the PM note on the ticket: a prompt
warning is demonstrably insufficient when the agent understands the mechanism
and is writing about it.

- New canonical `bin/ci-guard`, symlinked by all seven installers: scans a
  message (`scan`), a commit or range (`check`), or runs as a git hook
  (`commit-msg`). Comment lines and the scissors tail are skipped — git strips
  both before storing the message, and `commit --verbose` routinely puts the
  marker there when the PR is about the marker.
- All seven `task-work` copies install it as the repo's `commit-msg` hook, via
  a new drift-guarded `ci-guard-hook` region plus `lib/ci-guard.sh`. Git shares
  one hooks dir across a repo's worktrees, so it covers every commit in the
  repo, not just the worktree it was installed from — 4c3e11f, the sync commit
  that triggered both occurrences by being described, was made on main.
- Never clobbers: a pre-existing commit-msg hook is preserved as
  `commit-msg.local` and chained; both names taken means skip-with-warning
  rather than guess. Idempotent. No-ops when ci-guard is off PATH, so
  uninstalling task-force cannot lock you out of committing. Any write failure
  degrades to a warning — task-work is `set -e` and must not abort over this.
- The refusal names its own escape hatches, because *writing about* a marker is
  the legitimate case every affected ticket requires: break the token, or
  `--no-verify`, or TASK_FORCE_NO_CI_GUARD=1. A guard with no documented way to
  write the token gets bypassed or reverted.

Defence in depth, not the mechanism:

- All 8 worker prompts (5 commands/worker.md + 3 kiro agents/worker.json) gain
  a "CI markers" item and a strengthened "Green" item: an empty check list is
  not a pass, and `gh run list -c "$(git rev-parse HEAD)"` returning 0 means
  green is not a claim you can make. Wording is byte-identical across loadouts
  and stays repo-generic per #177.
- The 6 reviewer prompts get the same empty-vs-passing distinction against the
  PR's headRefOid — a reviewer cross-checking "CI green" off an empty list
  repeats the bug at one remove.

Tests: new tests/ci_guard.bats (33) covers all five markers case-insensitively,
comment/scissors stripping, the documented escape hatch passing, refusal
messaging, hook install (idempotent / foreign-hook chaining / core.hooksPath /
linked worktree / unwritable dir), and end-to-end refuse + bypass. worker_checklist
gains the 8-prompt and 6-reviewer assertions and widens the drift window so the
new fenced proof-of-run command is compared too. Full suite 1095 ok / 0 not ok,
drift check and shellcheck -x green.

Upgrading: re-run `./install.sh <loadout>` for the new ci-guard symlink, then
`task-init <loadout>` for the refreshed prompts and steering docs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…region marker, and pin the ci-guard section's placement

#212 landed `task-init:managed:*` markers in the workflow docs and moved this
repo's "Pre-PR checklist (this repo)" out of the managed region, below the end
marker — the section 4c3e11f deleted three times, now structurally safe. My
CI-markers and Green bullets were written against the in-region copy, so the
rebase resolution puts them on the surviving one.

The ci-guard section itself stays *inside* the managed region: it documents a
hook task-init ships, so it must be refreshed on re-run, not frozen as
repo-specific content.

Three tests pin what the rebase could silently have broken:

- the ci-guard section is byte-identical across all 8 workflow docs (same shape
  as radio_runbook's runbook_block, which is why it works unchanged on the
  dogfood copy)
- the section sits above the managed-region end marker — below it, task-init
  would never refresh it and it would go stale on every upgrade
- this repo's pre-PR checklist still carries the CI-markers gate, since nothing
  else guards the region below the end marker

Also cross-references #183/#212 in the CHANGELOG entry: 4c3e11f is the
task-init clobber that broke main, and the marker it carried is why no run
caught it. One bug did the damage, the other hid it.

Re-confirmed after rebase: radio_runbook 3 and 4 pass (the 8 workflow-doc
blocks are still byte-identical within each loadout group), check-drift clean
at 23 groups with both the ci-guard-hook and managed-region-source entries,
managed_region 22/22, full suite 1125 ok / 0 not ok, shellcheck -x green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@martin-conur
martin-conur merged commit 419b8b0 into main Sep 15, 2026
4 checks passed
@martin-conur
martin-conur deleted the task/skip-ci-guard branch September 15, 2026 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

worker prompts: a quoted [skip ci] in a commit message silently suppresses CI — warn, and treat an empty check list as not-green

1 participant