chore(ci): build the nightly preview even when Full Suite fails - #1404
Conversation
The nightly Preview Release stopped at "Release tests" whenever any Full Suite job failed, so a single failing test or live shard meant no preview at all. The build now continues and the failures travel with it. Release tests: instead of requiring a passing Full Suite, the job requires only that the suite finished (success or failure). It then downloads the full-suite-* evidence, lists this run's jobs, and renders scripts/ci-preview-test-report.ts into the run summary and a preview-test-report artifact. The report names the failed legs, the failed jobs (grouped by leg, linked), and the failing test files and cases. Each artifact contributes only its own run: the stamp its job recorded, else the earliest, so the runner tests' deliberately failing fixture runs never appear. Missing evidence or an unavailable job list narrows the report instead of blocking the build. The job alone gains actions: read, to list the jobs. Build chain: a job's implicit success() also covers its ancestors (actions/runner#2205), so with Full Suite failed every job below Release tests would still be skipped. native-smoke through release now state `!cancelled()` plus a success check of each direct need, which keeps the existing tests-then-build order. t332 fails if a job below Release tests lacks that explicit condition. Release: a preview built from a failing suite opens its notes with a warning and ends them with the report. Passing previews keep their notes unchanged. The report is capped at 30,000 characters so the notes stay under GitHub's body limit. Release result: a new final step fails the run when Full Suite failed, so the scheduled run stays red and still alerts, even though the preview published. An unchanged source still skips publication, and the run still fails when its tests fail. Stable releases are unaffected; they never consume Full Suite evidence. The native-smoke build checks (t238/t244 on the built binaries) remain gates. t332 now pins the non-blocking path, the notes staging, and the red result. t-ci-preview-test-report covers the report script. Docs (DEVELOPERS.md, the testing and supply-chain references, the install guide's release channels, tests/README.md) and the ci.yml header no longer say that Full Suite failures block the preview.
AIDA findings ledger
Open blocking findings (P0/P1): 0. Accepted and rejected findings never count toward the next action. Maintainer commands (repository write access) — put them on the first lines of a comment, one per line, several ids per line allowed: ledger.json{
"version": 4,
"pullRequest": 1404,
"nextId": 6,
"findings": [
{
"id": "F1",
"priority": "P1",
"category": "contracts",
"title": "Near-limit planned notes still abort failed-suite publication",
"anchors": [
{
"kind": "line",
"path": "scripts/ci-preview-test-report.ts",
"side": "RIGHT",
"sha256": "e87dd06be2ca5b772e3955d2412bc9e2f066abb2466ec28f1cc978303daf809f"
},
{
"kind": "line",
"path": ".github/workflows/preview-release.yml",
"side": "RIGHT",
"sha256": "19f97a73d97c3336ecddae48186661311cdbcfc1294ea33890017a1dc87538f1"
},
{
"kind": "line",
"path": "scripts/ci-preview-test-report.ts",
"side": "RIGHT",
"sha256": "0017b33c2a4b4f9d30d49c86f9df9ec51df2fd8e973e31b96161ef5046564370"
},
{
"kind": "line",
"path": "docs/reference/09-testing.md",
"side": "RIGHT",
"sha256": "8fcbb119bb3f1e793806135a09062eff4c07c1407537f09841a4a25e2690ae49"
}
],
"status": "resolved",
"firstSeen": {
"head": "4df8f704600f34f3ba8042844a755e411481f4a3",
"at": "2026-09-25T09:27:45.576Z"
},
"lastSeen": {
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b",
"at": "2026-09-25T10:02:30.342Z"
}
},
{
"id": "F2",
"priority": "P3",
"category": "user-experience",
"title": "Setup failures are inaccurately described as failing tests",
"anchors": [
{
"kind": "line",
"path": ".github/workflows/preview-release.yml",
"side": "RIGHT",
"sha256": "19f97a73d97c3336ecddae48186661311cdbcfc1294ea33890017a1dc87538f1"
},
{
"kind": "line",
"path": "scripts/ci-preview-test-report.ts",
"side": "RIGHT",
"sha256": "f901eac510a69f10b32b112ed295f60bb9239697b6a59175cf8bcee6be64debc"
}
],
"status": "resolved",
"firstSeen": {
"head": "4df8f704600f34f3ba8042844a755e411481f4a3",
"at": "2026-09-25T09:27:45.576Z"
},
"lastSeen": {
"head": "458a2488fd805c9cc82b5f92ec40adb57da3d4d2",
"at": "2026-09-25T09:46:23.113Z"
}
},
{
"id": "F3",
"priority": "P3",
"category": "contracts",
"title": "Contributor reference still says Full Suite gates preview publication",
"anchors": [
{
"kind": "line",
"path": ".github/workflows/preview-release.yml",
"side": "RIGHT",
"sha256": "55f9fa861ff6a48ca0d0738f7330aaa4b600f7d29dcdf9d6b889743f0b5b35bb"
},
{
"kind": "line",
"path": ".github/workflows/preview-release.yml",
"side": "RIGHT",
"sha256": "439647bfab762136d85af263456e76d52bae2ace63a93e07a9435cc3a624a5f8"
}
],
"status": "resolved",
"firstSeen": {
"head": "4df8f704600f34f3ba8042844a755e411481f4a3",
"at": "2026-09-25T09:27:45.576Z"
},
"lastSeen": {
"head": "458a2488fd805c9cc82b5f92ec40adb57da3d4d2",
"at": "2026-09-25T09:46:23.113Z"
}
},
{
"id": "F4",
"priority": "P3",
"category": "user-experience",
"title": "Job-level result still misstates report availability",
"anchors": [
{
"kind": "line",
"path": ".github/workflows/preview-release.yml",
"side": "RIGHT",
"sha256": "4853db26a78f92be2e74f14858d23b3674406493d51fa62585ed63350a50e613"
},
{
"kind": "line",
"path": ".github/workflows/preview-release.yml",
"side": "RIGHT",
"sha256": "b80097d28e476b0c2bd41078c752ea6c75b92df0121768787f6a99e01b945a21"
}
],
"status": "open",
"firstSeen": {
"head": "458a2488fd805c9cc82b5f92ec40adb57da3d4d2",
"at": "2026-09-25T09:46:23.113Z"
},
"lastSeen": {
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b",
"at": "2026-09-25T10:02:30.342Z"
}
},
{
"id": "F5",
"priority": "P2",
"category": "contracts",
"title": "Near-limit failed previews can lose every failure marker",
"anchors": [
{
"kind": "line",
"path": "scripts/ci-preview-test-report.ts",
"side": "RIGHT",
"sha256": "783312ac35eb7b3da84ac8abe2ea173389c730ddf7ce839461c7a2b62b934c5e"
},
{
"kind": "line",
"path": "tests/unit/t-ci-preview-test-report.test.ts",
"side": "RIGHT",
"sha256": "d551ebefbdb0ccbce41aea6c5a8bdd3de84be9008160855e06ad7fa35e04367d"
},
{
"kind": "line",
"path": "docs/reference/09-testing.md",
"side": "RIGHT",
"sha256": "9f84c94f31e671c0a35f1a10accb5b821aec93e1e8d6d1b9b28163ac3b26c27f"
}
],
"status": "open",
"firstSeen": {
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b",
"at": "2026-09-25T10:02:30.342Z"
},
"lastSeen": {
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b",
"at": "2026-09-25T10:02:30.342Z"
}
}
],
"events": [
{
"at": "2026-09-25T09:27:45.576Z",
"kind": "opened",
"by": "aida",
"id": "F1",
"head": "4df8f704600f34f3ba8042844a755e411481f4a3"
},
{
"at": "2026-09-25T09:27:45.576Z",
"kind": "opened",
"by": "aida",
"id": "F2",
"head": "4df8f704600f34f3ba8042844a755e411481f4a3"
},
{
"at": "2026-09-25T09:27:45.576Z",
"kind": "opened",
"by": "aida",
"id": "F3",
"head": "4df8f704600f34f3ba8042844a755e411481f4a3"
},
{
"at": "2026-09-25T09:46:23.113Z",
"kind": "seen",
"by": "aida",
"id": "F1",
"head": "458a2488fd805c9cc82b5f92ec40adb57da3d4d2"
},
{
"at": "2026-09-25T09:46:23.113Z",
"kind": "resolved",
"by": "aida",
"id": "F2",
"head": "458a2488fd805c9cc82b5f92ec40adb57da3d4d2",
"reason": "declared corrected by the judge; a cited line is gone"
},
{
"at": "2026-09-25T09:46:23.113Z",
"kind": "resolved",
"by": "aida",
"id": "F3",
"head": "458a2488fd805c9cc82b5f92ec40adb57da3d4d2",
"reason": "declared corrected by the judge; cited files changed since the last review"
},
{
"at": "2026-09-25T09:46:23.113Z",
"kind": "opened",
"by": "aida",
"id": "F4",
"head": "458a2488fd805c9cc82b5f92ec40adb57da3d4d2"
},
{
"at": "2026-09-25T10:02:30.342Z",
"kind": "seen",
"by": "aida",
"id": "F4",
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b"
},
{
"at": "2026-09-25T10:02:30.342Z",
"kind": "resolved",
"by": "aida",
"id": "F1",
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b",
"reason": "declared corrected by the judge; a cited line is gone"
},
{
"at": "2026-09-25T10:02:30.342Z",
"kind": "opened",
"by": "aida",
"id": "F5",
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b"
}
],
"review": {
"head": "2dbcf241bfc6c5e0adaad124b0512d80c0316d2b",
"readiness": 4,
"risk": 2,
"decision": "merge"
}
} |
There was a problem hiding this comment.
Reviewed 4df8f704600f34f3ba8042844a755e411481f4a3 against 5975205f6aff222b9ab85d515d138f53abb2fcb4 and current repository behavior.
Inspection: 10 changed files. Scope: full head (first review of this pull request).
Final Assessment
Human decision aid only: Readiness 5/5 is best; Risk 1/5 is best. These scores inform the maintainer; the next action below follows finding severity (any open P0/P1 → author/change) and does not approve or merge the PR.
Readiness: 3/5 — The main DAG change is coherent and well covered by focused tests, but the failed-suite publication path still has a release-body boundary that can defeat its primary purpose. Two smaller reporting and documentation inconsistencies also remain.
Risk: 3/5 — The publication failure requires unusually large but valid planned notes, so blast radius is moderate. It blocks the nightly preview rather than corrupting releases or weakening asset verification.
Decision required: Author — make changes before this PR proceeds. The combined release-body limit defect can still block preview publication after a Full Suite failure, contradicting the primary behavior introduced by this PR.
Validation performed:
- Inspected all ten changed paths, the full diff, review scope, metadata, discussion, empty findings ledger, and specialist candidates.
- Traced the preview workflow DAG, release planner/publisher, update discovery, Full Suite artifacts, log sanitization, test runner failure format, and related documentation contracts.
- Established statically that direct-need conditions allow a completed Full Suite failure through the build while preserving contract, build, lifecycle, publication, and final-result checks.
- Verified immutable head snapshots against the stated head and found no diff whitespace errors.
Findings: 1 blocking, 2 advisory.
Ledger: 3 open, 0 retained blocking, 0 accepted, 0 suppressed as rejected by a maintainer. Maintainers act on findings with /aida commands in the ledger comment.
Contracts & Compatibility
P1 [F1]: Combined notes and report can exceed the release-body limit
Evidence: scripts/ci-preview-test-report.ts:15, .github/workflows/preview-release.yml:690.
Problem: When the uncapped planner produces an otherwise valid release body above roughly 95,000 characters, the failure path appends a report of up to 30,000 characters plus its warning. The combined body then exceeds the stated 125,000-character GitHub limit, so publish-release receives an API failure and no preview is published.
Impact: A failing Full Suite can still block the exact preview publication this change promises to preserve. This is a concrete release-path correctness failure, although it requires large accumulated changelog notes.
Required correction: Apply the limit to the combined warning, planned notes, and report. Dynamically budget or truncate the report while preserving required note content and the source footer, validate the final body before publication, and add a near-limit failed-suite test.
P3 [F3]: Contributor reference still says Full Suite gates preview publication
Evidence: .github/workflows/preview-release.yml:103, .github/workflows/preview-release.yml:165.
Problem: The unchanged authoritative contributor guide at docs/reference/11-contributing.md:97 still says preview builds are gated through the full deterministic/live suite, while this head explicitly allows a failed suite through publication.
Impact: Contributors and release maintainers receive conflicting guidance about why a red Full Suite run can still publish a preview.
Required correction: Update docs/reference/11-contributing.md to state that contract and release-asset checks gate publication while Full Suite failures are reported and keep the run red; extend the documentation contract test to guard this wording.
User Experience
User experience change: Preview users may now receive a prerelease after Full Suite failure, while release maintainers receive a red workflow, summary, artifact, and release-note report.
Before: A Full Suite failure stopped the preview build and publication chain.
After: A completed Full Suite failure permits publication with a warning and diagnostic report, then the final result keeps the workflow red.
Example: A nightly test failure can publish a preview whose notes begin with a warning and end with failed-job and test evidence.
Assessment: This improves access to testable preview artifacts and failure context, but setup-only failures are described inaccurately as failing tests, and large notes can still prevent publication.
P3 [F2]: Setup failures are inaccurately described as failing tests
Evidence: .github/workflows/preview-release.yml:690, scripts/ci-preview-test-report.ts:158.
Problem: If Full Suite fails during planning, setup, or provisioning before a test failure is recorded, the report correctly says no failing test files were found and lists the failed job, but the release warning and user documentation still promise that failing tests appear at the end.
Impact: Preview users receive contradictory recovery information and may look for nonexistent test failures instead of following the failed-job link.
Required correction: Describe the appended content as a Full Suite failure report rather than failing tests, distinguish missing test evidence from actual test failures, and align the guide and test documentation.
Residual risk: Repository code and tests were not executed under the read-only review contract. GitHub Actions expression and hosted-workflow behavior were assessed statically.
Reviewed by AIDA (AI-DLC Developer Agent).
[AI-PR-REVIEWED] 4df8f70
Addresses AIDA on #1404 (F1, F2, F3). F1: the 30,000-character cap bounded only the report, so planned notes above about 95,000 characters plus the warning and report could exceed GitHub's 125,000-character release body and fail publication, the case this change exists to allow. The release job now stages the notes with `ci-preview-test-report.ts --stage-notes`, which keeps every planned line and the source footer and trims only the report to the remaining budget. It fails clearly if the planned notes alone leave no room. t-ci-preview-test-report covers the exact, trimmed and near-limit budgets and the command; t332 still runs the workflow step. F2: a Full Suite that fails in setup records no failing tests, yet the warning promised a list of them. The warning, the notes wording in the docs, and the report now call it a Full Suite failure report, and the report says when the failed jobs may have stopped before their tests. F3: docs/reference/11-contributing.md still said Full Suite gates the preview. It now names the contract and release-asset gates and states that a failing suite is reported instead of blocking publication; t332 pins that wording.
There was a problem hiding this comment.
Reviewed 458a2488fd805c9cc82b5f92ec40adb57da3d4d2 against 5975205f6aff222b9ab85d515d138f53abb2fcb4 and current repository behavior.
Inspection: 11 changed files. Scope: incremental — 9 files with lines changed since the review at 4df8f704; the security lenses reviewed the full head. Findings on lines unchanged since that review are deferred, not decisive.
Final Assessment
Human decision aid only: Readiness 5/5 is best; Risk 1/5 is best. These scores inform the maintainer; the next action below follows finding severity (any open P0/P1 → author/change) and does not approve or merge the PR.
Readiness: 3/5 — The main workflow and prior documentation defects are substantially corrected, but a near-limit release body still breaks the feature's required failed-suite publication path.
Risk: 3/5 — The blocking condition requires unusually large but valid notes, limiting frequency, but it prevents nightly preview publication. The remaining diagnostic issue is low impact.
Decision required: Author — make changes before this PR proceeds. The remaining P1 release-body defect can still prevent the failed-suite preview publication this change requires.
Validation performed:
- Inspected all 11 changed-file snapshots, the full diff, review scope, ledger, discussion, specialist candidates, AGENTS.md, and CONTRIBUTING.md.
- Confirmed every snapshot matches the immutable head and the diff has no whitespace errors.
- Traced the preview workflow DAG, report artifact, note staging, uncapped release-note planner, publisher, documentation, and focused tests.
- Established statically that failed-suite publication still aborts when planned notes leave insufficient warning-marker capacity.
Findings: 1 blocking, 1 advisory.
Ledger: 2 open, 0 retained blocking, 0 accepted, 0 suppressed as rejected by a maintainer, resolved F2, F3 (F2, F3 declared corrected by the judge). Maintainers act on findings with /aida commands in the ledger comment.
Contracts & Compatibility
P1 [F1]: Near-limit planned notes still abort failed-suite publication
Evidence: scripts/ci-preview-test-report.ts:216, docs/reference/09-testing.md:1252.
Problem: When the uncapped planner produces notes for which the warning, body, newline, and minimum truncation marker exceed 125,000 characters, stagePreviewNotes throws even after dropping the entire report. Thus a body that could otherwise publish still aborts specifically when Full Suite fails, contradicting the documented publication contract.
Impact: A large but valid changelog or commit-subject body can still suppress the nightly preview after failed tests, breaking the feature's primary required behavior.
Required correction: Bound planner-generated notes while preserving the source footer and reserving failed-suite warning/report overhead; validate the final body and add a test where the original body fits but the minimum staged failure body does not. Reconcile the claim that planned notes always remain whole.
User Experience
User experience change: Preview maintainers and users can receive a preview after Full Suite fails, with a warning and failure report while the workflow remains red.
Before: A Full Suite failure stopped the build and publication chain.
After: A completed Full Suite failure normally permits publication and appends diagnostic evidence to the release notes.
Example: A nightly test failure publishes a warned prerelease, then Release result fails the workflow.
Assessment: This improves preview availability and diagnostics, but oversized planned notes can still prevent publication, and one final error can reference a report that was never generated.
P3 [F4]: Final error can point to a nonexistent failure report
Evidence: .github/workflows/preview-release.yml:753.
Problem: If Full Suite fails and Release tests then fails during checkout, Bun setup, or report generation, its summary has no failure report. Release result nevertheless emits an error claiming that report exists.
Impact: Maintainers investigating a failed nightly are directed to absent recovery evidence.
Required correction: Condition the message on needs.test.result: reference the summary only after successful report generation; otherwise state that Release tests failed and direct maintainers to that job's logs. Cover both paths in the workflow test.
Residual risk: Repository code and tests were not executed under the read-only review contract; GitHub Actions behavior was assessed statically.
Reviewed by AIDA (AI-DLC Developer Agent).
[AI-PR-REVIEWED] 458a248
Superseded by AI review of 458a248
Addresses AIDA on #1404 (F1 round 2, F4). F1: staging threw when the planned notes sat within the warning and truncation marker of GitHub's 125,000-character limit, so a failing suite could still stop a preview whose notes would publish. Staging now never throws. The report is trimmed first, then the warning shrinks to a one-line pointer to the run, and a planned body that fits only on its own publishes unchanged. A failing suite therefore never causes the overflow; notes the planner makes too long fail exactly as before. t-ci-preview-test-report covers the pointer, the body-only fallback and an already-oversized plan. F4: Release result pointed at the failure report even when Release tests failed before writing it. The message now depends on the Release tests result, and t332 covers both paths.
There was a problem hiding this comment.
Reviewed 2dbcf241bfc6c5e0adaad124b0512d80c0316d2b against 5975205f6aff222b9ab85d515d138f53abb2fcb4 and current repository behavior.
Inspection: 11 changed files. Scope: incremental — 5 files with lines changed since the review at 458a2488; the security lenses reviewed the full head. Findings on lines unchanged since that review are deferred, not decisive.
Final Assessment
Human decision aid only: Readiness 5/5 is best; Risk 1/5 is best. These scores inform the maintainer; the next action below follows finding severity (any open P0/P1 → author/change) and does not approve or merge the PR.
Readiness: 4/5 — The primary failed-suite publication path and prior blocking boundary failure are corrected and well covered. A near-limit disclosure gap and a smaller recovery-message defect remain.
Risk: 2/5 — The remaining defects affect rare near-limit notes and failure diagnostics for preview releases. Stable publication, asset verification, and workflow failure signaling remain intact.
Decision required: Maintainer — decide whether to merge this PR. No P0 or P1 finding survives; the remaining P2 and P3 findings are advisory under the repository decision contract.
Validation performed:
- Inspected all 11 changed paths, the immutable diff and snapshots, incremental review scope, ledger, discussion, repository instructions, and specialist outputs.
- Classified the change as a mixed workflow feature and recovery/reporting change and traced the preview DAG, Full Suite evidence, release planner, publisher, tests, and documentation contracts.
- Statically established that completed Full Suite failures permit the build and publication chain while Release result keeps the workflow red.
- Confirmed the previous near-limit publication failure is corrected when planned notes fit the release-body limit.
- Confirmed the diff has no whitespace errors.
Findings: 0 blocking, 2 advisory.
Ledger: 2 open, 0 retained blocking, 0 accepted, 0 suppressed as rejected by a maintainer, resolved F1 (F1 declared corrected by the judge). Maintainers act on findings with /aida commands in the ledger comment.
Contracts & Compatibility
P2 [F5]: Near-limit failed previews can lose every failure marker
Evidence: scripts/ci-preview-test-report.ts:219, tests/unit/t-ci-preview-test-report.test.ts:252, docs/reference/09-testing.md:1252.
Problem: When planned notes fit the 125,000-character limit but leave insufficient room for FAILED_SUITE_POINTER, stagePreviewNotes returns the original body unchanged. The resulting preview was built after a failed Full Suite but contains neither a warning nor a report, contradicting the documented release-note contract.
Impact: Preview consumers viewing the release cannot distinguish this failed-validation artifact from a normally reported preview and lose the traceability intended to make publishing after failed tests safe.
Required correction: Always preserve a minimal failure marker linked to the workflow run, reserving its space during planning or truncating planned content while retaining the source footer. Update the boundary test and reconcile all release-note documentation.
User Experience
User experience change: Preview users can receive a prerelease after Full Suite fails, while maintainers receive a red workflow plus failure evidence in the Release tests summary, artifact, and normally the release notes.
Before: A Full Suite failure stopped the preview build and publication chain.
After: A completed Full Suite failure normally permits publication with a warning and report, then Release result fails the workflow.
Example: A nightly test failure can publish a warned preview while the Actions run remains red.
Assessment: This improves preview availability and diagnostics, but near-limit notes can omit every failure marker and one report-job failure path gives inaccurate recovery guidance.
P3 [F4]: Job-level result still misstates report availability
Evidence: .github/workflows/preview-release.yml:757.
Problem: The report step can append the report to the job summary before the subsequent artifact upload fails. That makes needs.test.result failure, so Release result claims no report was produced even though the summary contains it.
Impact: Maintainers investigating that failure are directed away from available diagnostic evidence.
Required correction: Track successful summary generation independently from the overall job result, or use conservative wording that directs maintainers to both the summary and logs. Cover a failure after report generation.
Residual risk: Repository code and tests were not executed under the read-only review contract; GitHub Actions expression and hosted-workflow behavior were assessed statically.
Reviewed by AIDA (AI-DLC Developer Agent).
[AI-PR-REVIEWED] 2dbcf24
Superseded by AI review of 2dbcf24
…longpaths * origin/main: chore(ci): build the nightly preview even when Full Suite fails (#1404)
* origin/main: (93 commits) fix: tell reviewers the exact findings table so fewer reviews are refused (#1415) fix: improve Windows installation and preserve unowned files during uninstall (#1182) docs(install): say what a copy install asks you to trust (#1419) refactor(kiro-ide): target Kiro's unified agent harness (Kiro IDE 1.x and Kiro CLI v3) (#1416) feat(scopes): default express to Guard Policy off (#1397) fix(sensors): let reverse-engineering's declared sensors reach the CodeKB (#1403) fix(doctor): close the Kiro IDE ignore-sources follow-ups from #1391 (#1413) fix: name doctor through the harness skill prefix in state and registry refusals (#1410) fix(worktree): let Bolt worktrees pass MAX_PATH on Windows (#1399) chore(ci): build the nightly preview even when Full Suite fails (#1404) test: parse custom-harness SDK directive output on Windows (#1016) fix: normalize blank subagent types in completion audits (#1018) fix(ci): preserve full-suite evidence and verify completion (#1369) fix(doctor): probe every framework read and stop the repository search at filesystem boundaries (#1391) fix: keep open gates through scope-change and close the #1272 follow-ups (#1398) fix(windows): repair the cross-OS test failures on Windows (#1393) fix(doctor): compare the audit against the per-stage checkboxes (#1272) fix(sensor): drop a superseded detail file when the sensor passes (#1266) chore(ci): supersede Full Suite verification across branch heads (#1390) fix: never record an unreadable review findings table as no findings (#1163) ...
Problem
The nightly Preview Release stopped at Release tests whenever any Full Suite job failed. A single failing test or live shard meant no preview that night (for example, runs 36060429097 and 35785443087 skipped every build job after Release tests failed).
Change
full-suite-*evidence, lists this run's jobs, and rendersscripts/ci-preview-test-report.tsinto the run summary and apreview-test-reportartifact. The report lists the failed legs, the failed jobs (grouped by leg, with links) and the failing test files and cases. Each artifact contributes only its own run, so the runner tests' deliberately failing fixture runs (t112 calibration, t05'sNO_SUCHfilter) never show up. Missing evidence or an unavailable job list narrows the report instead of blocking the build. Only this job gainsactions: read.success()also covers its ancestors (Jobs skipped when NEEDS job ran successfully actions/runner#2205), so every job below Release tests would still have been skipped.native-smokethroughreleasenow use!cancelled()plus a success check on each direct need. That keeps the existing tests-then-build order. t332 fails if a job below Release tests is missing that condition.ci-preview-test-report.ts --stage-noteskeeps every planned line and the source footer. The report, then the warning, gives way to GitHub's 125,000-character release limit, so a failing suite never stops a preview whose planned notes fit.Unchanged: stable releases (they never consume Full Suite evidence), the native-smoke build checks (t238/t244 against the built binaries), the contract gate, and the unchanged-source publication skip.
Testing
tests/run-tests.ts: 206 cases. The smoke tier passes too.full-suite-*evidence, which is fine for the report job to download.bun run checkandbun tests/gen-coverage-registry.ts --checkpass.build's condition makes t332 fail with "build must not inherit the Full Suite result".preview-release.ymlonly runs onmain, so the first real proof is the next nightly (or a manual dispatch) after merge.Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.