Repository navigation
feat(extension): show each claim's reason and evidence on its step 5 row, and link where it is made (#172) - #179
Merged
Merged
Conversation
…row, and link where it is made (#172) Each refuted or unverifiable claim's step 5 row now shows its evidence source with its reason, each line it cites as a chip — a head-copy line opening the line read-only as the overview's chips do, a CI log line opening its check run on GitHub, the one outside link, taken only for github.com addresses — and the recheck when the engine dropped a citation, so an unverifiable claim says why it was not verified without opening anything else. Where a claim is made becomes a link, on step 5 and in the overview's claims table: a docstring's or comment's line opens in the head copy, a description claim's line opens the overview's description scrolled to it and marked, since a pull request description has no line anchors on GitHub, and a story claim's opens the story at its sentence. Every piece of agent-derived text in the new places reaches the page escaped.
…nsion/test/integration/extension.test.ts:2168 asserted the old step 5 claim-row markup where the claim's 'where' was plain text (<span class="where">). This PR (#172) changed claimFindingRow in packages/extension/src/side-bar/page.ts to render 'where' as a link (<button type="button" class="where" id="step-5-where-claim-1" data-claim-where="1">), and updated the unit tests (side-bar.test.ts, side-bar-page.test.ts) but missed the CI-only integration test, which npm test/npm run check never run locally. Invariant fixed: every assertion of the rendered step 5 claim-row markup must match the renderer; the single stale sibling assertion was updated to expect the button markup, identical to the already-updated side-bar.test.ts:302 assertion. Verified locally: npm run test:integration 79/79 pass (including the failing test), npm run check (build, tsc test config, lint, 1879 unit tests) all pass. No production code changed; smallest root-cause fix
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
The ticket is #172 ("Show each claim's reason and evidence on its step 5 row, and link where it is made"), one ticket of the approved v1 order of lbildzinkas/second-look, a public MIT VS Code companion for human pull request review (default branch master). Read CONTEXT.md (glossary), docs/adr/ (decisions) and docs/ux/ (the chosen reviewing surface) before starting; the engine and the extension are TypeScript, the engine a separate local process. The ticket, quoted in full:
What to build
A claim's row in side bar step 5 (
side-bar/page.ts) shows only its verdict, where the claim is made as plain text, and its quote. The result already holds the verdict's reason, its evidence source, its cited head-copy lines, its cited CI log lines and its recheck, but they show only in the overview's collapsed detail row or in a finding's thread. An unverifiable claim therefore gives the reviewer no reason and nothing to check it against.urlis already in the result), and its recheck when the engine dropped a citation.claimWherereturns plain text today): a docstring or comment claim opens its line in the head copy; "description, line N" opens the overview with the description scrolled to that line and the line marked, since a pull request description has no line anchors on GitHub; a story claim opens the story.https://github.com/addresses.How to try it by hand: review lbildzinkas/interview-coach#98 and read, on each unverifiable claim in step 5, why it was not verified; press a description claim's place and land on its marked line in the overview.
Acceptance criteria
Blocked by
Standing boundary for validating this ticket on this machine: the VS Code application is never launched, downloaded or installed here, and validation installs no tools; extension tests that need VS Code run only in CI, so live validation goes through the engine protocol, the unit tests and rendered output.
What Changed
https://github.com/addresses, opened through the extension), the recheck note when the engine dropped a citation, and plain chips for fetched-library citations.claimWhere/claimEvidencemessages flow through the side-bar provider and extension handlers, anddocs/reviewing.mdplus unit tests (escaping of agent-derived text, link targets, and CI-only VS Code tests) cover the new surfaces.Risk Assessment
✅ Low: A well-bounded UI wiring change that satisfies every stated acceptance criterion - reason and evidence on step 5 rows, head-copy and check-run chips, marked description line and story sentence, escaped agent text, and the single guarded outside link - with the only findings being two informational notes (markup duplication and name-based check-run matching).
Testing
Drove every acceptance criterion of the ticket against the real product code in this run: each unverifiable claim's step 5 row renders its reason and evidence source inline with cited head-copy lines as chips, CI log citations as chips whose only outside link is the check run's github.com page (an evil.example URL is refused), the dropped-citation recheck, and where-made as links in both step 5 and the overview claims table — a description claim reopens the overview with that description line marked and scrolled to, a story claim marks its sentence, a file claim opens the head copy line, a pipeline claim stays text, and fetched-library citations stay non-opening chips; hostile agent text in reason, recheck, paths and aria-labels stays escaped. Everything passed, verified through the ticket's standing boundary routes (real rendered output, real panels handling the real page scripts' messages, and the unit suites), with rendered-HTML artifacts as reviewer-visible evidence; no screenshot or GIF was possible because the boundary forbids launching VS Code (or any GUI) on this machine, and browser automation would require installing tools, which the boundary also forbids.
Evidence: Step 5 claim rows as the side bar webview renders them (product output, verbatim)
<li class="finding"><div class="head"><span class="pill tone-warn">? unverifiable</span><button type="button" class="where" id="step-5-where-claim-1" data-claim-where="1">pull request description, line 1</button></div><q class="quote">Retries failed sends.</q><div class="why">the model's memory: Nothing in the change shows it.</div></li> — plus the refuted rows' <div class="why">the change itself: The loop runs five times.</div> with chip buttons "src/retry.py:6" (title "Open the line in the head copy") and "CI log of check / test, line 2" (title "Open its check run on GitHub"), and the story claim's <div class="why">dropped to unverifiable: the model's memory never yields verified</div>~/.no-mistakes/evidence/01M4QS673C5RX3DAZWJ89P3B0Z/side-bar-step5-claims.preview.html)Evidence: Overview opened at a description claim's line: line 2 marked and scrolled to (product output, verbatim)
Source: Overview opened at a description claim's line: line 2 marked and scrolled to (product output, verbatim) (local file:
~/.no-mistakes/evidence/01M4QS673C5RX3DAZWJ89P3B0Z/overview-opened-at-description-line-2.html)~/.no-mistakes/evidence/01M4QS673C5RX3DAZWJ89P3B0Z/overview-opened-at-description-line-2.preview.html)Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
packages/extension/src/overview.ts:1290- descriptionLines (overview.ts:1290-1304) re-implements the hidden-run markup that sanitiseUntrusted already builds (flag label plus shown span, overview.ts:346-360), and descriptionSection (overview.ts:1315) runs hiddenContent a second time through sanitiseUntrusted only to obtain the hidden counts for the summary line. One pass that returns both the per-line markup and the counts, or a shared span builder used by both, would keep the flagged rendering and the summary from drifting apart. Mechanical, non-functional simplification; the line splitting itself is correct (verified against multi-line and unclosed HTML comments: pushes happen exactly per newline, so data-line numbering matches the engine's 1-based flattenText lines).packages/extension/src/side-bar/provider.ts:269- A CI-log citation carries only the check run's name, and the handler resolves the URL by matching ci.checks on name (provider.ts:269, mirroring the engine's id-to-name mapping in verdicts.ts:421); if a pull request has two failed check runs with the same name, the chip opens the first one's run. Rare (GitHub matrix jobs usually carry distinct names), self-evident on landing, and a durable fix needs the engine's Citation to carry the check run reference rather than its name, which extends beyond this change - noting the limitation, not blocking on it.✅ **Test** - passed
✅ No issues found.
npx vitest run packages/extension/test/overview.test.ts packages/extension/test/side-bar.test.ts packages/extension/test/side-bar-page.test.ts — 137 tests pass (drives the real page scripts and providers end to end)Temporary live driver packages/extension/test/live-render-driver.test.ts (written, run, then removed): executed the real sideBarHtml/sideBarBody and overviewHtml renderers, the real OverviewPanel (openClaimWhere for file/description/story claims, marked dline/sentence, scroll selector) and the real SideBarProvider (claimWhere and claimEvidence messages: head-copy line opens, CI check run opens via vscode.env.openExternal, https://evil.example URL refused, out-of-range indices and fetched-library citations open nothing) — 11 checks passRendered-output evidence: side-bar-step5-claims.html and overview-opened-at-description-line-2.html written verbatim from the product renderers, plus themed .preview.html copies for browser legibilitynpx vitest list (default config) — confirms no integration/real-host/package-smoke VS Code suite runs locally; npx vitest list --config vitest.integration.config.ts resolves only packages/extension/test/integration/extension.test.ts; .github/workflows/ci.yml runs those suites under xvfb in CI onlynpx tsc -b && npx tsc -p tsconfig.test.json — the change builds and all tests typecheck (build outputs cleaned up afterwards)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.