Skip to content

feat(engine): read a passing check's CI log to settle a claim on the reviewer's press (#174) - #184

Merged
lbildzinkas merged 5 commits into
masterfrom
fm/sl-ci-log-route
Oct 12, 2026
Merged

lbildzinkas merged 5 commits into
masterfrom
fm/sl-ci-log-route

Conversation

@lbildzinkas

@lbildzinkas lbildzinkas commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Intent

Closes #174.

Today only failed checks' logs are fetched and offered to the verdicts pass as evidence, so a claim such as "scripts/check.sh passes: ruff, pyright and 39 tests" stays unverifiable even when the CI check that runs that script passed at the pull request's head. This change lets the reviewer settle such a claim with a passing check's log.

An unverifiable claim whose settles is ci-log offers Read the CI log when the pull request has passing checks at its head. Pressing it fetches those checks' logs, with the same size limits and handling as failed checks' logs, and judges the claim again with them as evidence. The new verdict cites the log lines as it would a failed log's, labelled as from a passing check. Nothing is fetched until the reviewer presses, as for library source (ADR 0003).

It is done when nothing is fetched before the press, which a test shows; when a claim judged again with a passing check's log cites its lines, labelled as from a passing check; when docs/privacy.md and docs/reviewing.md say passing checks' logs are fetched only on the reviewer's press; and when the extension tests that need VS Code run in CI only. Validation launches, downloads and installs neither VS Code nor any other tool: it goes through the engine protocol, the unit tests and rendered output, and the extension tests that need VS Code run only in CI.

What Changed

  • The engine now offers Read the CI log on an unverifiable claim a CI log could settle when the pull request has passing checks at its head (new offerCiLogReads), and a new readCiLog RPC method (ci-log-read.ts) fetches those passing GitHub Actions jobs' logs — trimmed and capped as failed jobs' logs are, nothing fetched before the press — and re-judges the claim with them as evidence (review result schema v25, verdicts prompt v7), citing the log lines labelled as from a passing check.
  • The extension adds the second-look.readCiLog command and a Read the CI log button on the claim's quiet step 5 row and in the overview, shows which passing checks' logs the re-judged verdict used, and labels citations of a passing check's log in the overview, side bar, finding bodies and draft comments.
  • Verdicts judged with passing checks' logs never carry over across new commits; docs/privacy.md, docs/reviewing.md, docs/agent-safety.md, docs/getting-started.md and the README state that passing checks' logs are fetched only on the reviewer's press (ADR 0003).

Risk Assessment

✅ Low: The change is well-bounded, mirrors the established library-fetch pattern at every layer (gated press, metered reads, redacted errors, versioned protocol/prompt, carried-over re-judging), satisfies each acceptance criterion with behavioral tests, and review found no substantiated defect.

Testing

The criteria that could be driven against the real running product were validated live: the fetch log from the live engine run proves the passing job's log is fetched only after the reviewer's press (jobs/9002/logs 0x before, 1x after, while the failed job's is read with the review); the re-judged verdict cites the passing check's log line with ciLog+passingCheck, and the rendered overview/side-bar surfaces label it 'CI log of check / lint (a passing check), line 11' and say the claim was judged again with the passing checks' logs; refused presses fail plainly and fetch nothing. The remaining criteria were covered by targeted tests and file reading only, not live: the library-fetch re-offer and no-carry-over rules pass as engine tests (server.test.ts, carry-over.test.ts, ci.test.ts, review.test.ts), the docs' press-only wording was read in docs/privacy.md and docs/reviewing.md, and the CI-only integration configuration was read from vitest.config.ts and .github/workflows/ci.yml. All 425 targeted engine and extension tests pass, and the round-1 recorded fix (offerLibraryFetches on the CI-log re-judged verdict) is implemented with its test. No failures, no unfixable setup issues.

  • Live validation: ✅ go - 5 of 10 scenarios driven live against the product
Scenario Result Live Evidence
A review offers Read the CI log on an unverifiable claim a CI log could settle when checks passed at the head, and offers nothing on claims only the author or outside things could settle ✅ pass live live-review-result-offered.json (claim 0 ciLogFetch {checks:['check / lint'], reason 'check / lint passed at the head commit; reading its log could settle the claim.'}; claim 1 settles author, no offe…
Nothing is fetched before the press: the review reads only the failed job's log, never the passing job's ✅ pass live live-fetch-log.jsonl: between the 'review requested' marker and the 'readCiLog pressed' marker, actions/jobs/9001/logs is fetched once and actions/jobs/9002/logs zero times; jobs/9002/logs appears onl…
Pressing Read the CI log fetches the passing checks' logs and judges the claim again, citing the log's lines labelled as from a passing check, keeping the pressed offer and recording which logs judged… ✅ pass live live-review-result-after-press.json: verdict verified from 'a CI log', evidence [{path 'check / lint', line 11, quote 'Lint passed with 0 errors.', ciLog: true, passingCheck: true}], ciLogs {checks [{…
The reviewer's surfaces show the offer before the press and the labelled citation after it: the overview button and quiet step-5 row, then 'CI log of check / lint (a passing check), line 11' and 'judg… ✅ pass live live-overview-offered.html (one button.pt.ci-log, offer reason), live-overview-read.html (labelled citation, judged-again line, zero buttons), live-side-bar-offered.html (data-finding="ciLog" Read the…
A press that cannot be served fails plainly and fetches nothing: a claim with no offer, an unknown claim index, and a request without a token; the GitHub token never appears in any answer ✅ pass live live-rpc-transcript.json: 'this claim offers no CI log to read', 'this engine has no reviewed claim 99 of …; review the pull request again', invalid-params for the missing token; live-fetch-log.jsonl…
A claim re-judged with the passing logs that turns out to need a library gets its library fetch offered again (or a plain reason why not), as a verdicts-pass verdict would ⏸️ untested no The prior payload recorded only the engine unit test packages/engine/test/server.test.ts 'offers the library fetch again, or says why not, when the claim judged with the log needs a library' (run this…
A verdict judged with the passing checks' logs never carries over to a later review, and an unpressed or stale offer is re-made from the new CI or withdrawn ⏸️ untested no The prior payload recorded only engine unit tests (carry-over.test.ts with verdictHolds false when ciLogs set, ci.test.ts 'makes an unpressed offer again from the CI it is given, or withdraws it' and…
docs/privacy.md and docs/reviewing.md say passing checks' logs are fetched only on the reviewer's press ⏸️ untested no The prior payload recorded only a static reading of docs/privacy.md ('Passing checks' logs, only when you press) and docs/reviewing.md ('Reading a passing check's CI log' ending 'Nothing is fetched un…
Extension tests that need VS Code run in CI only ⏸️ untested no The prior payload recorded only static readings of vitest.config.ts (default runs include only packages//test/.test.ts, with the comment that the integration suite 'runs in CI (under xvfb) … never i…
The reviewer presses the button inside the real VS Code window (webview click → command → engine → result shown in place) ⏸️ untested no The ticket's standing boundary for this machine forbids launching, downloading or installing the VS Code application, so the in-editor button click cannot be driven here; that check is owned by the CI…
Evidence: GitHub request log of the live engine run (review marker → failed job's log only; readCiLog press marker → passing job's log)
{"marker":"review requested","at":1791827712626}
{"url":"https://api.github.com/repos/example-org/example-repo/pulls/42","method":"GET","at":1791827712633}
{"url":"https://api.github.com/repos/example-org/example-repo/pulls/42","method":"GET","at":1791827712634}
{"url":"https://api.github.com/repos/example-org/example-repo/contents/.gitattributes?ref=f00dcafe1234567890abcdef1234567890abcdef","method":"GET","at":1791827712637}
{"url":"https://api.github.com/repos/example-org/example-repo/compare/0123456789abcdef0123456789abcdef01234567...f00dcafe1234567890abcdef1234567890abcdef?per_page=1","method":"GET","at":1791827712637}
{"url":"https://api.github.com/repos/example-org/example-repo/commits/f00dcafe1234567890abcdef1234567890abcdef/check-runs?per_page=100","method":"GET","at":1791827712638}
{"url":"https://api.github.com/graphql","method":"POST","at":1791827712638}
{"url":"https://api.github.com/repos/example-org/example-repo/check-runs/9001/annotations?per_page=50","method":"GET","at":1791827712639}
{"url":"https://api.github.com/repos/example-org/example-repo/check-runs/9002/annotations?per_page=50","method":"GET","at":1791827712639}
{"url":"https://api.github.com/repos/example-org/example-repo/actions/jobs/9001/logs","method":"GET","at":1791827712639}
{"url":"https://api.github.com/user","method":"GET","at":1791827712639}
{"url":"https://api.github.com/repos/example-org/example-repo/pulls/42/reviews?per_page=100","method":"GET","at":1791827712640}
{"url":"https://api.github.com/repos/example-org/example-repo/tarball/f00dcafe1234567890abcdef1234567890abcdef","method":"GET","at":1791827712642}
{"url":"https://api.github.com/repos/example-org/example-repo/tarball/4242424242424242424242424242424242424242","method":"GET","at":1791827712643}
{"marker":"readCiLog pressed on claim 0","at":1791827713619}
{"url":"https://api.github.com/repos/example-org/example-repo/commits/f00dcafe1234567890abcdef1234567890abcdef/check-runs?per_page=100","method":"GET","at":1791827713619}
{"url":"https://api.github.com/repos/example-org/example-repo/actions/jobs/9002/logs","method":"GET","at":1791827713620}
{"marker":"readCiLog pressed on claim 1, which offers no read","at":1791827713859}
Evidence: Full JSON-RPC transcript with the real engine serve process (initialize, review, readCiLog, refusals)
{
  "responses": {
    "1": {
      "jsonrpc": "2.0",
      "id": 1,
      "result": {
        "protocolVersion": 1
      }
    },
    "2": {
      "jsonrpc": "2.0",
      "id": 2,
      "result": {
        "version": 25,
        "pullRequest": {
          "url": "https://github.com/example-org/example-repo/pull/42",
          "number": 42,
          "title": "Update dependencies and rename the config module",
          "author": "reviewer-login",
          "description": "The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whatever its length. The full description must never be truncated by the companion, whateve

... [113082 bytes truncated] ...

 "fake-provider/fake-model",
              "effort": null,
              "runAt": "2026-10-12T17:55:13.536Z",
              "tokens": {
                "input": 100,
                "output": 20,
                "cacheRead": 5,
                "cacheWrite": 0,
                "total": 125
              },
              "costUsd": 0.01
            },
            "startedAt": "2026-10-12T17:55:13.536Z",
            "agent": {
              "agent": "pi",
              "model": null,
              "effort": null
            },
            "durationMs": 81
          },
          {
            "id": "criteria",
            "position": 8,
            "total": 9,
            "state": "skipped",
            "detail": "this change gave it nothing to do",
            "startedAt": "2026-10-12T17:55:13.617Z",
            "agent": {
              "agent": "pi",
              "model": null,
              "effort": null
            },
            "durationMs": 0
          },
          {
            "id": "docLinks",
            "position": 9,
            "total": 9,
            "state": "done",
            "startedAt": "2026-10-12T17:55:13.618Z",
            "agent": {
              "agent": "pi",
              "model": null,
              "effort": null
            },
            "durationMs": 0
          }
        ],
        "story": {
          "promptVersion": "1",
          "stamp": {
            "agent": "pi",
            "agentVersion": "9.9.9",
            "model": "fake-provider/fake-model",
            "effort": null,
            "runAt": "2026-10-12T17:55:13.138Z",
            "tokens": {
              "input": 200,
              "output": 40,
              "cacheRead": 10,
              "cacheWrite": 0,
              "total": 250
            },
            "costUsd": 0.02
          },
          "outcome": "fell back",
          "detail": "the agent gave no usable answer (invalid-answer: the answer was invalid twice: the answer is missing \"sentences\")",
          "sentences": []
        },
        "unexplained": {
          "promptVersion": "1",
          "parts": [],
          "described": [],
          "outcome": "fell back",
          "detail": "the agent gave no usable answer (invalid-answer: the answer was invalid twice: the answer is missing \"unexplained\"; the answer is missing \"described\")",
          "stamp": {
            "agent": "pi",
            "agentVersion": "9.9.9",
            "model": "fake-provider/fake-model",
            "effort": null,
            "runAt": "2026-10-12T17:55:13.295Z",
            "tokens": {
              "input": 200,
              "output": 40,
              "cacheRead": 10,
              "cacheWrite": 0,
              "total": 250
            },
            "costUsd": 0.02
          }
        },
        "claims": {
          "promptVersion": "1",
          "stamp": {
            "agent": "pi",
            "agentVersion": "9.9.9",
            "model": "fake-provider/fake-model",
            "effort": null,
            "runAt": "2026-10-12T17:55:13.453Z",
            "tokens": {
              "input": 100,
              "output": 20,
              "cacheRead": 5,
              "cacheWrite": 0,
              "total": 125
            },
            "costUsd": 0.01
          },
          "outcome": "listed",
          "detail": "every quote was found in its source, which locates the claim and, for a docstring or comment, its part",
          "claims": [
            {
              "quote": "The full description must never be truncated by the companion, whatever its length.",
              "source": "description",
              "location": {
                "kind": "description",
                "line": 1
              },
              "part": 0,
              "verdict": {
                "kind": "verified",
                "source": "a CI log",
                "reason": "The lint check passed.",
                "looked": [],
                "evidence": [
                  {
                    "path": "check / lint",
                    "line": 11,
                    "quote": "Lint passed with 0 errors.",
                    "ciLog": true,
                    "passingCheck": true
                  }
                ],
                "ciLogFetch": {
                  "checks": [
                    "check / lint"
                  ],
                  "reason": "check / lint passed at the head commit; reading its log could settle the claim."
                },
                "ciLogs": {
                  "checks": [
                    {
                      "name": "check / lint",
                      "url": "https://github.com/example-org/example-repo/actions/runs/700/job/9002",
                      "detail": "the log marks no failing step, so its last 13 lines are kept"
                    }
                  ],
                  "promptVersion": "7",
                  "stamp": {
                    "agent": "pi",
                    "agentVersion": "9.9.9",
                    "model": "fake-provider/fake-model",
                    "effort": null,
                    "runAt": "2026-10-12T17:55:13.779Z",
                    "tokens": {
                      "input": 100,
                      "output": 20,
                      "cacheRead": 5,
                      "cacheWrite": 0,
                      "total": 125
                    },
                    "costUsd": 0.01
                  }
                }
              }
            },
            {
              "quote": "The full description must never be truncated by the companion",
              "source": "description",
              "location": {
                "kind": "description",
                "line": 1
              },
              "part": 0,
              "verdict": {
                "kind": "unverifiable",
                "source": "the change itself",
                "reason": "Only the author knows.",
                "looked": [],
                "evidence": [],
                "settles": {
                  "kind": "author",
                  "note": "ask the author"
                }
              }
            }
          ],
          "judging": {
            "promptVersion": "7",
            "stamp": {
              "agent": "pi",
              "agentVersion": "9.9.9",
              "model": "fake-provider/fake-model",
              "effort": null,
              "runAt": "2026-10-12T17:55:13.536Z",
              "tokens": {
                "input": 100,
                "output": 20,
                "cacheRead": 5,
                "cacheWrite": 0,
                "total": 125
              },
              "costUsd": 0.01
            },
            "outcome": "judged",
            "detail": "every citation was re-read in the head copy or the CI log it names; one that did not match, or the model's memory alone, kept a claim from verified"
          }
        },
        "docLinks": {
          "links": [],
          "unlinked": [],
          "notes": [
            "Library APIs are linked to their documentation in Python and C# only; the change's .ts files are not read for them"
          ]
        },
        "budget": {
          "limits": {
            "agentRuns": 0,
            "filesFetched": 0,
            "downloadMiB": 0
          },
          "used": {
            "agentRuns": 11,
            "filesFetched": 15,
            "downloadBytes": 25265
          }
        }
      }
    },
    "4": {
      "jsonrpc": "2.0",
      "id": 4,
      "error": {
        "code": -32002,
        "message": "this claim offers no CI log to read"
      }
    },
    "5": {
      "jsonrpc": "2.0",
      "id": 5,
      "error": {
        "code": -32002,
        "message": "this engine has no reviewed claim 99 of https://github.com/example-org/example-repo/pull/42; review the pull request again"
      }
    },
    "6": {
      "jsonrpc": "2.0",
      "id": 6,
      "error": {
        "code": -32602,
        "message": "readCiLog needs params: { \"url\": string, \"token\": string, \"claim\": number, \"agent\"?: { \"agent\": \"pi\" | \"claude-code\", \"model\"?: string, \"account\"?: string } }"
      }
    }
  },
  "stderr": ""
}
  • Evidence: Live review result before the press: claim 0 unverifiable with the ciLogFetch offer, claim 1 (settles author) without (local file: ~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-review-result-offered.json)
  • Evidence: Live review result after the press: claim 0 verified from 'a CI log', citing check / lint line 11 with passingCheck, ciLogs metadata, offer kept (local file: ~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-review-result-after-press.json)
  • Evidence: The saved review on disk after the press (review-result.json in the engine's cache), holding the new verdict (local file: ~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-saved-review-result.json)
  • Evidence: Rendered overview before the press, with the 'Read the CI log' button and the offer's reason (local file: ~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-overview-offered.html)
  • Evidence: Rendered overview after the press: citation 'CI log of check / lint (a passing check), line 11' and 'judged again with the logs of the checks that passed', button withdrawn (local file: ~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-overview-read.html)
  • Evidence: Rendered side bar step 5 before the press, with Read the CI log on the quiet row of the claim a CI log could settle (local file: ~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-side-bar-offered.html)
  • Evidence: Rendered side bar step 5 after the press, with the re-judged claim folded into the verified count and no offer left (local file: ~/.no-mistakes/evidence/01M4RK1ECRBZCX06NRCD3E6QYJ/live-side-bar-read.html)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 5 of 10 scenarios driven live against the product
Scenario Result Live Evidence
A review offers Read the CI log on an unverifiable claim a CI log could settle when checks passed at the head, and offers nothing on claims only the author or outside things could settle ✅ pass live live-review-result-offered.json (claim 0 ciLogFetch {checks:['check / lint'], reason 'check / lint passed at the head commit; reading its log could settle the claim.'}; claim 1 settles author, no offe…
Nothing is fetched before the press: the review reads only the failed job's log, never the passing job's ✅ pass live live-fetch-log.jsonl: between the 'review requested' marker and the 'readCiLog pressed' marker, actions/jobs/9001/logs is fetched once and actions/jobs/9002/logs zero times; jobs/9002/logs appears onl…
Pressing Read the CI log fetches the passing checks' logs and judges the claim again, citing the log's lines labelled as from a passing check, keeping the pressed offer and recording which logs judged… ✅ pass live live-review-result-after-press.json: verdict verified from 'a CI log', evidence [{path 'check / lint', line 11, quote 'Lint passed with 0 errors.', ciLog: true, passingCheck: true}], ciLogs {checks [{…
The reviewer's surfaces show the offer before the press and the labelled citation after it: the overview button and quiet step-5 row, then 'CI log of check / lint (a passing check), line 11' and 'judg… ✅ pass live live-overview-offered.html (one button.pt.ci-log, offer reason), live-overview-read.html (labelled citation, judged-again line, zero buttons), live-side-bar-offered.html (data-finding="ciLog" Read the…
A press that cannot be served fails plainly and fetches nothing: a claim with no offer, an unknown claim index, and a request without a token; the GitHub token never appears in any answer ✅ pass live live-rpc-transcript.json: 'this claim offers no CI log to read', 'this engine has no reviewed claim 99 of …; review the pull request again', invalid-params for the missing token; live-fetch-log.jsonl…
A claim re-judged with the passing logs that turns out to need a library gets its library fetch offered again (or a plain reason why not), as a verdicts-pass verdict would ⏸️ untested no The prior payload recorded only the engine unit test packages/engine/test/server.test.ts 'offers the library fetch again, or says why not, when the claim judged with the log needs a library' (run this…
A verdict judged with the passing checks' logs never carries over to a later review, and an unpressed or stale offer is re-made from the new CI or withdrawn ⏸️ untested no The prior payload recorded only engine unit tests (carry-over.test.ts with verdictHolds false when ciLogs set, ci.test.ts 'makes an unpressed offer again from the CI it is given, or withdraws it' and…
docs/privacy.md and docs/reviewing.md say passing checks' logs are fetched only on the reviewer's press ⏸️ untested no The prior payload recorded only a static reading of docs/privacy.md ('Passing checks' logs, only when you press) and docs/reviewing.md ('Reading a passing check's CI log' ending 'Nothing is fetched un…
Extension tests that need VS Code run in CI only ⏸️ untested no The prior payload recorded only static readings of vitest.config.ts (default runs include only packages//test/.test.ts, with the comment that the integration suite 'runs in CI (under xvfb) … never i…
The reviewer presses the button inside the real VS Code window (webview click → command → engine → result shown in place) ⏸️ untested no The ticket's standing boundary for this machine forbids launching, downloading or installing the VS Code application, so the in-editor button click cannot be driven here; that check is owned by the CI…
  • npx vitest run packages/engine/test/ci.test.ts packages/engine/test/server.test.ts packages/engine/test/verdicts.test.ts — 118 passed, including 'runRpcServer reading a passing check's CI log' (offer with the review, no passing log fetched before the press, re-judged citation with passingCheck, library fetch re-offered, refusals) and verdicts prompt/citation labelling
  • npx vitest run packages/engine/test/carry-over.test.ts packages/engine/test/draft-comment.test.ts packages/engine/test/review.test.ts — 57 passed, including a CI-log-judged verdict never carrying over and the drafted citation labelled '(a passing check)'
  • npx vitest run packages/extension/test/overview.test.ts packages/extension/test/side-bar.test.ts packages/extension/test/findings.test.ts packages/extension/test/engine-client.test.ts packages/extension/test/review-result.test.ts — 250 passed, including the overview button and labelled citation, the step-5 quiet row's Read the CI log button and command wiring, no thread for such claims, the readCiLog round-trip, and isReviewResult v25 accepting/rejecting the new shapes
  • npm run build, then a live drive: node .tmp-live/drive.mjs — spawned the real second-look-engine serve process, spoke its JSON-RPC protocol over stdio (initialize → review → readCiLog), with a fixture-backed GitHub transport that logs every request and a disposable agent executable passed through the product's own agent path setting; pressed Read the CI log on claim 0 and adversarially on claim 1, index 99, and without a token
  • Rendered the reviewer's surfaces from the live drive's own results through the extension's real rendering code (temporary vitest run): overview and side-bar HTML before and after the press, both live results also passed through the extension's isReviewResult guard
  • Read docs/privacy.md ('Passing checks' logs, only when you press) and docs/reviewing.md ('Reading a passing check's CI log': 'Nothing is fetched until you press'), and confirmed vitest.config.ts excludes test/integration from every default run while .github/workflows/ci.yml runs it under xvfb (CI only) — these were static file readings, not live drives
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…r's press (#174)

An unverifiable claim whose `settles` is `ci-log` now offers Read the CI
log (`ciLogFetch`) when checks passed at the pull request's head. Nothing
is fetched with the review: only the reviewer's press, as the engine's new
`readCiLog` request names the claim with the reviewer's token, lists the
check runs at the reviewed head commit again, downloads the passing GitHub
Actions jobs' logs with the same trimming, size limits and handling as a
failed job's, and has the agent judge the claim again with them. Each line
the new verdict cites is labelled as a passing check's (`passingCheck`),
and the verdict names the logs it read (`ciLogs`); the saved review
follows it.

A claim carried over from an earlier commit's review has its unpressed
offer made again from the CI at the new head, or withdrawn, and a carried
verdict judged with the earlier commit's passing logs never stands, so it
is judged again.

The verdicts prompt goes to version 7: when the logs shown are passing
checks', it names them so. The result version goes to 25, and the
extension's validator checks the new fields.

The extension offers Read the CI log on step 5's row and as a link on the
finding's thread, says so on the overview, and labels a passing check's
cited line on every surface. The privacy, reviewing and agent-safety pages
and the README say passing checks' logs are fetched only on the press.

Closes #174
…one (#174)

A comment drafted from a finding settled by a passing check's log now
cites that line as 'CI log of <check> (a passing check), line N', as the
overview, the finding's thread and step 5 label it; the draft's checks
accept the labelled location.
#174)

Master's #185 lists an unverifiable claim only a CI log could settle
quietly on step 5, with no thread and as no finding, so the thread's Read
the CI log link could no longer be reached. The press moves to that
claim's quiet step 5 row and to its overview row, a button each, and the
thread keeps only the note of the passing checks' logs a re-judged
finding was judged with; step 5 shows that note too. The result version
stays above master's, at 25.
@lbildzinkas lbildzinkas changed the title feat: read a passing check's CI log to settle a claim, on the reviewer's press feat(engine): read a passing check's CI log to settle a claim on the reviewer's press (#174) Oct 12, 2026
@lbildzinkas
lbildzinkas merged commit 23af070 into master Oct 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read a passing check's CI log to settle a claim, on the reviewer's press

1 participant