Repository navigation
feat(engine): read a passing check's CI log to settle a claim on the reviewer's press (#174) - #184
Merged
Merged
Conversation
…r's press (#174) An unverifiable claim whose `settles` is `ci-log` now offers Read the CI log (`ciLogFetch`) when checks passed at the pull request's head. Nothing is fetched with the review: only the reviewer's press, as the engine's new `readCiLog` request names the claim with the reviewer's token, lists the check runs at the reviewed head commit again, downloads the passing GitHub Actions jobs' logs with the same trimming, size limits and handling as a failed job's, and has the agent judge the claim again with them. Each line the new verdict cites is labelled as a passing check's (`passingCheck`), and the verdict names the logs it read (`ciLogs`); the saved review follows it. A claim carried over from an earlier commit's review has its unpressed offer made again from the CI at the new head, or withdrawn, and a carried verdict judged with the earlier commit's passing logs never stands, so it is judged again. The verdicts prompt goes to version 7: when the logs shown are passing checks', it names them so. The result version goes to 25, and the extension's validator checks the new fields. The extension offers Read the CI log on step 5's row and as a link on the finding's thread, says so on the overview, and labels a passing check's cited line on every surface. The privacy, reviewing and agent-safety pages and the README say passing checks' logs are fetched only on the press. Closes #174
…one (#174) A comment drafted from a finding settled by a passing check's log now cites that line as 'CI log of <check> (a passing check), line N', as the overview, the finding's thread and step 5 label it; the draft's checks accept the labelled location.
#174) Master's #185 lists an unverifiable claim only a CI log could settle quietly on step 5, with no thread and as no finding, so the thread's Read the CI log link could no longer be reached. The press moves to that claim's quiet step 5 row and to its overview row, a button each, and the thread keeps only the note of the passing checks' logs a re-judged finding was judged with; step 5 shows that note too. The result version stays above master's, at 25.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Closes #174.
Today only failed checks' logs are fetched and offered to the verdicts pass as evidence, so a claim such as "
scripts/check.shpasses: ruff, pyright and 39 tests" stays unverifiable even when the CI check that runs that script passed at the pull request's head. This change lets the reviewer settle such a claim with a passing check's log.An unverifiable claim whose
settlesisci-logoffers Read the CI log when the pull request has passing checks at its head. Pressing it fetches those checks' logs, with the same size limits and handling as failed checks' logs, and judges the claim again with them as evidence. The new verdict cites the log lines as it would a failed log's, labelled as from a passing check. Nothing is fetched until the reviewer presses, as for library source (ADR 0003).It is done when nothing is fetched before the press, which a test shows; when a claim judged again with a passing check's log cites its lines, labelled as from a passing check; when
docs/privacy.mdanddocs/reviewing.mdsay passing checks' logs are fetched only on the reviewer's press; and when the extension tests that need VS Code run in CI only. Validation launches, downloads and installs neither VS Code nor any other tool: it goes through the engine protocol, the unit tests and rendered output, and the extension tests that need VS Code run only in CI.What Changed
offerCiLogReads), and a newreadCiLogRPC method (ci-log-read.ts) fetches those passing GitHub Actions jobs' logs — trimmed and capped as failed jobs' logs are, nothing fetched before the press — and re-judges the claim with them as evidence (review result schema v25, verdicts prompt v7), citing the log lines labelled as from a passing check.second-look.readCiLogcommand and a Read the CI log button on the claim's quiet step 5 row and in the overview, shows which passing checks' logs the re-judged verdict used, and labels citations of a passing check's log in the overview, side bar, finding bodies and draft comments.docs/privacy.md,docs/reviewing.md,docs/agent-safety.md,docs/getting-started.mdand the README state that passing checks' logs are fetched only on the reviewer's press (ADR 0003).Risk Assessment
✅ Low: The change is well-bounded, mirrors the established library-fetch pattern at every layer (gated press, metered reads, redacted errors, versioned protocol/prompt, carried-over re-judging), satisfies each acceptance criterion with behavioral tests, and review found no substantiated defect.
Testing
The criteria that could be driven against the real running product were validated live: the fetch log from the live engine run proves the passing job's log is fetched only after the reviewer's press (jobs/9002/logs 0x before, 1x after, while the failed job's is read with the review); the re-judged verdict cites the passing check's log line with ciLog+passingCheck, and the rendered overview/side-bar surfaces label it 'CI log of check / lint (a passing check), line 11' and say the claim was judged again with the passing checks' logs; refused presses fail plainly and fetch nothing. The remaining criteria were covered by targeted tests and file reading only, not live: the library-fetch re-offer and no-carry-over rules pass as engine tests (server.test.ts, carry-over.test.ts, ci.test.ts, review.test.ts), the docs' press-only wording was read in docs/privacy.md and docs/reviewing.md, and the CI-only integration configuration was read from vitest.config.ts and .github/workflows/ci.yml. All 425 targeted engine and extension tests pass, and the round-1 recorded fix (offerLibraryFetches on the CI-log re-judged verdict) is implemented with its test. No failures, no unfixable setup issues.
Evidence: GitHub request log of the live engine run (review marker → failed job's log only; readCiLog press marker → passing job's log)
Evidence: Full JSON-RPC transcript with the real engine serve process (initialize, review, readCiLog, refusals)
~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-review-result-offered.json)~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-review-result-after-press.json)~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-saved-review-result.json)~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-overview-offered.html)~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-overview-read.html)~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-side-bar-offered.html)~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-side-bar-read.html)Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
npx vitest run packages/engine/test/ci.test.ts packages/engine/test/server.test.ts packages/engine/test/verdicts.test.ts — 118 passed, including 'runRpcServer reading a passing check's CI log' (offer with the review, no passing log fetched before the press, re-judged citation with passingCheck, library fetch re-offered, refusals) and verdicts prompt/citation labellingnpx vitest run packages/engine/test/carry-over.test.ts packages/engine/test/draft-comment.test.ts packages/engine/test/review.test.ts — 57 passed, including a CI-log-judged verdict never carrying over and the drafted citation labelled '(a passing check)'npx vitest run packages/extension/test/overview.test.ts packages/extension/test/side-bar.test.ts packages/extension/test/findings.test.ts packages/extension/test/engine-client.test.ts packages/extension/test/review-result.test.ts — 250 passed, including the overview button and labelled citation, the step-5 quiet row's Read the CI log button and command wiring, no thread for such claims, the readCiLog round-trip, and isReviewResult v25 accepting/rejecting the new shapesnpm run build, then a live drive: node .tmp-live/drive.mjs — spawned the real second-look-engine serve process, spoke its JSON-RPC protocol over stdio (initialize → review → readCiLog), with a fixture-backed GitHub transport that logs every request and a disposable agent executable passed through the product's own agent path setting; pressed Read the CI log on claim 0 and adversarially on claim 1, index 99, and without a tokenRendered the reviewer's surfaces from the live drive's own results through the extension's real rendering code (temporary vitest run): overview and side-bar HTML before and after the press, both live results also passed through the extension's isReviewResult guardRead docs/privacy.md ('Passing checks' logs, only when you press) and docs/reviewing.md ('Reading a passing check's CI log': 'Nothing is fetched until you press'), and confirmed vitest.config.ts excludes test/integration from every default run while .github/workflows/ci.yml runs it under xvfb (CI only) — these were static file readings, not live drives✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.