diff --git a/patterns/ci-sweeper.md b/patterns/ci-sweeper.md index f29294f..2bc2258 100644 --- a/patterns/ci-sweeper.md +++ b/patterns/ci-sweeper.md @@ -15,6 +15,7 @@ Slower overnight cadence (30–60m) is fine when no one is watching. - `ci-triage` — Parse CI logs, identify failing job/step, classify failure type (flake, regression, env, config) - `minimal-fix` — Smallest change that addresses the specific failure +- `loop-guard` — Circuit breaker: log each attempt to `loop-ledger.json` and run `loop-context --check` before retrying; escalate instead of looping on the same failure - Project test/lint skill — Build and test commands for your stack ## State @@ -48,7 +49,7 @@ Track: commit SHA, failing job, attempt count, worktree/PR link, outcome. - If actionable: open worktree → implementer drafts fix. 3. Verifier sub-agent checks: fix addresses failure, no unrelated changes, tests pass locally. 4. Open PR or comment on existing PR with proposed fix. -5. If attempts exceed max (e.g. 3): escalate to human with full context. +5. Before each retry, `loop-guard` runs the circuit breaker (`loop-context --check`). If the same failure recurs N× or attempts exceed max (e.g. 3), it trips: escalate to human with a pruned context summary instead of looping. 6. Prune resolved failures from active list. ## Verification Strategy diff --git a/templates/SKILL.md.loop-guard b/templates/SKILL.md.loop-guard new file mode 100644 index 0000000..e46a0c6 --- /dev/null +++ b/templates/SKILL.md.loop-guard @@ -0,0 +1,71 @@ +--- +name: loop-guard +description: > + Circuit breaker for fix-capable loops. Before each iteration, append the last + attempt to loop-ledger.json and run loop-context --check; if it escalates, + stop and hand the human a clean summary instead of looping in vain. +user_invocable: true +--- + +# Loop Guard (Circuit Breaker) + +You keep a fix loop from burning tokens on a problem it cannot solve. You wrap +every iteration of an action skill (`minimal-fix`, `ci-triage`, `dependency-triage`, …) +with a deterministic circuit-breaker check powered by +[`loop-context`](https://github.com/cobusgreyling/loop-engineering/tree/main/tools/loop-context). + +The breaker needs no LLM call, so it is cheap enough to run on every iteration. + +## The ledger + +`loop-ledger.json` records the loop's goal and one entry per attempt: + +```json +{ "goal": "Get failing CI green", "attempts": [] } +``` + +After every iteration, append what you just tried: + +```json +{ "iteration": 3, "action": "patch flaky auth test", "outcome": "failure", + "error": "AssertionError: expected 200 got 500", "tokensUsed": 1800 } +``` + +`outcome` is `success | failure | noop`. Always include `error` on failures — +that is how the breaker detects a repeated (stagnant) failure. + +## Before each iteration + +1. Append the previous attempt to `loop-ledger.json`. +2. Run the breaker: + ```bash + npx @cobusgreyling/loop-context --check --ledger loop-ledger.json + ``` +3. Act on the exit code: + - **0** → continue. Optionally trim the next prompt first: + `npx @cobusgreyling/loop-context --inject --ledger loop-ledger.json` + - **2** → **STOP.** The breaker tripped — same error N× in a row, too many + consecutive failures, the token budget, or the iteration cap. Do not retry. + +## On escalate (exit 2) + +1. Capture a clean, pruned summary for the human: + ```bash + npx @cobusgreyling/loop-context --inject --ledger loop-ledger.json > escalation.md + ``` +2. Write the escalation into STATE.md High Priority (or open an issue). +3. Exit the loop. A human decides the next step. + +## Rules + +- Never widen thresholds just to keep looping — escalation is a feature, not a failure. +- Never edit the ledger to hide a repeated error; the breaker exists to catch it. +- Defaults: 3× same error, 5 consecutive failures, 10 iterations. Tune with + `--stagnation`, `--no-progress`, `--max-iterations`, `--token-budget`. + +## Interaction with other skills + +- `minimal-fix` / `ci-triage` — record each attempt's outcome + error in the ledger. +- `loop-verifier` — a verifier rejection is a `failure`; log it so repeats trip the breaker. +- `loop-constraints` — honors "escalate after N attempts"; this skill makes it mechanical. +- `loop-budget` — `--token-budget` mirrors the daily cap in loop-budget.md. diff --git a/templates/loop-constraints.md b/templates/loop-constraints.md index b08f058..6157dc0 100644 --- a/templates/loop-constraints.md +++ b/templates/loop-constraints.md @@ -18,6 +18,7 @@ - Never disable tests to make CI green - Never refactor unrelated code — one fix per run - Max 3 fix attempts per item; escalate after +- Enforce the attempt limit mechanically: log each try to `loop-ledger.json` and run `loop-context --check` before retrying (see the `loop-guard` skill) ## Communication - Always tell me what you're about to do before doing it diff --git a/tools/loop-init/README.md b/tools/loop-init/README.md index beadd5a..edc9471 100644 --- a/tools/loop-init/README.md +++ b/tools/loop-init/README.md @@ -31,6 +31,13 @@ After scaffolding, always run `npx @cobusgreyling/loop-audit . --suggest` and ac L2 patterns (`ci-sweeper`, `dependency-sweeper`) also copy `minimal-fix` and `loop-verifier` templates when missing from the starter. +Fix-capable patterns (`pr-babysitter`, `ci-sweeper`, `dependency-sweeper`, `post-merge-cleanup`) also get a **circuit breaker**: + +- `loop-guard` skill — logs each attempt to `loop-ledger.json` and runs [`loop-context`](../loop-context) `--check` before retrying +- `loop-ledger.json` — seeded with the pattern's goal and an empty `attempts` array + +The breaker escalates (same error N× in a row, too many consecutive failures, token budget, or iteration cap) instead of looping in vain. Report-only patterns skip it. + Every scaffold also creates: - `loop-budget.md` — pattern-specific daily caps and kill switch diff --git a/tools/loop-init/dist/cli.js b/tools/loop-init/dist/cli.js index 7bbfe52..0883f2d 100644 --- a/tools/loop-init/dist/cli.js +++ b/tools/loop-init/dist/cli.js @@ -142,6 +142,37 @@ async function copyL2Templates(pattern, tool, targetDir, templatesRoot, dryRun) await copyTemplateVerifier(templatesRoot, targetDir, tool, dryRun); } } +/** Per-pattern goal seeded into loop-ledger.json for the circuit breaker. */ +const LEDGER_GOAL = { + 'daily-triage': 'Keep the repo healthy and STATE.md current', + 'pr-babysitter': 'Get the watched PR review-ready and green', + 'ci-sweeper': 'Get failing CI back to green', + 'dependency-sweeper': 'Land safe dependency updates', + 'post-merge-cleanup': 'Clean up regressions from recent merges', + 'changelog-drafter': 'Draft accurate release notes', + 'issue-triage': 'Triage the open issue queue', +}; +/** + * Fix-capable loops retry actions, so they need a circuit breaker: scaffold the + * loop-guard skill plus a seeded loop-ledger.json wired to `loop-context`. + * Report-only patterns (daily-triage, issue-triage, changelog-drafter) don't + * retry fixes, so they skip this to keep the scaffold minimal. + */ +async function scaffoldCircuitBreaker(pattern, tool, targetDir, templatesRoot, dryRun) { + if (!PATTERNS_NEEDING_FIX.has(pattern)) + return; + await copyTemplateSkill(templatesRoot, 'SKILL.md.loop-guard', targetDir, tool, 'loop-guard', dryRun); + const ledgerPath = path.join(targetDir, 'loop-ledger.json'); + if (await exists(ledgerPath)) + return; + const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], attempts: [] }, null, 2)}\n`; + if (dryRun) { + console.log(` would write: ${ledgerPath}`); + return; + } + await writeFile(ledgerPath, seed); + console.log(' created: loop-ledger.json (circuit breaker)'); +} function formatTokenCap(n) { if (n >= 1_000_000) return `${n / 1_000_000}M`; @@ -442,6 +473,7 @@ Examples: await copyFile(loopMd, path.join(targetDir, 'LOOP.md'), dryRun); } await copyL2Templates(pattern, tool, targetDir, templatesRoot, dryRun); + await scaffoldCircuitBreaker(pattern, tool, targetDir, templatesRoot, dryRun); await scaffoldObservability(pattern, tool, targetDir, templatesRoot, dryRun); await scaffoldConstraints(targetDir, templatesRoot, tool, dryRun); if (tool !== 'opencode' && !dryRun && !(await exists(path.join(targetDir, 'AGENTS.md')))) { @@ -475,6 +507,11 @@ npm run lint console.log(` npx @cobusgreyling/loop-audit ${auditArg} --suggest`); } } + if (PATTERNS_NEEDING_FIX.has(pattern)) { + console.log(''); + console.log('Circuit breaker wired (loop-guard skill + loop-ledger.json):'); + console.log(' npx @cobusgreyling/loop-context --check --ledger loop-ledger.json'); + } console.log(''); console.log(`First loop (${tool}):`); console.log(` ${firstLoopCommand(pattern, tool)}`); diff --git a/tools/loop-init/src/cli.ts b/tools/loop-init/src/cli.ts index d0c6cc1..f01413f 100644 --- a/tools/loop-init/src/cli.ts +++ b/tools/loop-init/src/cli.ts @@ -184,6 +184,45 @@ async function copyL2Templates( } } +/** Per-pattern goal seeded into loop-ledger.json for the circuit breaker. */ +const LEDGER_GOAL: Record = { + 'daily-triage': 'Keep the repo healthy and STATE.md current', + 'pr-babysitter': 'Get the watched PR review-ready and green', + 'ci-sweeper': 'Get failing CI back to green', + 'dependency-sweeper': 'Land safe dependency updates', + 'post-merge-cleanup': 'Clean up regressions from recent merges', + 'changelog-drafter': 'Draft accurate release notes', + 'issue-triage': 'Triage the open issue queue', +}; + +/** + * Fix-capable loops retry actions, so they need a circuit breaker: scaffold the + * loop-guard skill plus a seeded loop-ledger.json wired to `loop-context`. + * Report-only patterns (daily-triage, issue-triage, changelog-drafter) don't + * retry fixes, so they skip this to keep the scaffold minimal. + */ +async function scaffoldCircuitBreaker( + pattern: Pattern, + tool: Tool, + targetDir: string, + templatesRoot: string, + dryRun: boolean, +) { + if (!PATTERNS_NEEDING_FIX.has(pattern)) return; + + await copyTemplateSkill(templatesRoot, 'SKILL.md.loop-guard', targetDir, tool, 'loop-guard', dryRun); + + const ledgerPath = path.join(targetDir, 'loop-ledger.json'); + if (await exists(ledgerPath)) return; + const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], attempts: [] }, null, 2)}\n`; + if (dryRun) { + console.log(` would write: ${ledgerPath}`); + return; + } + await writeFile(ledgerPath, seed); + console.log(' created: loop-ledger.json (circuit breaker)'); +} + function formatTokenCap(n: number): string { if (n >= 1_000_000) return `${n / 1_000_000}M`; if (n >= 1_000) return `${n / 1_000}k`; @@ -524,6 +563,7 @@ Examples: } await copyL2Templates(pattern, tool, targetDir, templatesRoot, dryRun); + await scaffoldCircuitBreaker(pattern, tool, targetDir, templatesRoot, dryRun); await scaffoldObservability(pattern, tool, targetDir, templatesRoot, dryRun); await scaffoldConstraints(targetDir, templatesRoot, tool, dryRun); @@ -560,6 +600,12 @@ npm run lint } } + if (PATTERNS_NEEDING_FIX.has(pattern)) { + console.log(''); + console.log('Circuit breaker wired (loop-guard skill + loop-ledger.json):'); + console.log(' npx @cobusgreyling/loop-context --check --ledger loop-ledger.json'); + } + console.log(''); console.log(`First loop (${tool}):`); console.log(` ${firstLoopCommand(pattern, tool)}`); diff --git a/tools/loop-init/test/cli.test.mjs b/tools/loop-init/test/cli.test.mjs index 914ed6a..807776d 100644 --- a/tools/loop-init/test/cli.test.mjs +++ b/tools/loop-init/test/cli.test.mjs @@ -1,6 +1,6 @@ import { test } from 'node:test'; import assert from 'node:assert/strict'; -import { mkdtemp, rm, access } from 'node:fs/promises'; +import { mkdtemp, rm, access, readFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import path from 'node:path'; import { execFile } from 'node:child_process'; @@ -122,3 +122,39 @@ test('loop-init scaffolds ci-sweeper with bundled assets', async () => { await rm(dir, { recursive: true, force: true }); } }); + +test('loop-init scaffolds circuit breaker (loop-guard + ledger) for fix patterns', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'loop-init-cb-')); + try { + await exec('node', [CLI, dir, '--pattern', 'ci-sweeper', '--tool', 'grok']); + await access(path.join(dir, '.grok', 'skills', 'loop-guard', 'SKILL.md')); + const ledger = JSON.parse(await readFile(path.join(dir, 'loop-ledger.json'), 'utf8')); + assert.equal(typeof ledger.goal, 'string'); + assert.ok(ledger.goal.length > 0); + assert.deepEqual(ledger.attempts, []); + } finally { + await rm(dir, { recursive: true, force: true }); + } +}); + +test('loop-init scaffolds circuit breaker for pr-babysitter (opencode paths)', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'loop-init-cb-oc-')); + try { + await exec('node', [CLI, dir, '--pattern', 'pr-babysitter', '--tool', 'opencode']); + await access(path.join(dir, 'skills', 'loop-guard', 'SKILL.md')); + await access(path.join(dir, 'loop-ledger.json')); + } finally { + await rm(dir, { recursive: true, force: true }); + } +}); + +test('loop-init does NOT scaffold circuit breaker for report-only daily-triage', async () => { + const dir = await mkdtemp(path.join(tmpdir(), 'loop-init-nocb-')); + try { + await exec('node', [CLI, dir, '--pattern', 'daily-triage', '--tool', 'grok']); + await assert.rejects(() => access(path.join(dir, 'loop-ledger.json'))); + await assert.rejects(() => access(path.join(dir, '.grok', 'skills', 'loop-guard', 'SKILL.md'))); + } finally { + await rm(dir, { recursive: true, force: true }); + } +});