test(audit): regression harness uses CI audit entry point (run_main_audit_workflow)#6917
Conversation
…nt) to close fidelity gap
Homeboy Results —
|
Empirical finding: entry point was NOT the gap; the regression is REAL-CODEBASE-SPECIFICI tested this directly: cherry-picked the #6915 structural migration onto THIS branch (workflow-entry harness) and ran the harness — it still PASSES (4 tests), even via Conclusion: the #6855 Phase 1 regression depends on the actual homeboy repository's scale/content (real cross-references, real file sizes hitting GodFile/HighItemCount thresholds, real This PR is still a real improvement (the harness should audit via the same entry point CI uses — better fidelity for the class of regressions a fixture CAN catch), so it's worth merging on its own merits. But the actual unblocker for #6855 is a different harness shape: a test/CI step that audits the REAL homeboy repo tree and asserts zero net drift vs baseline — essentially running the differential Audit gate locally/in-test. That's the only thing that reproduces real-codebase-specific regressions. Recommend that as the next step (it overlaps with #6858 'fast audit preview/reconcile'). |
Fixes the deep fidelity gap behind #6896/#6906/#6915: the harness audited via audit_path_with_id while CI Audit uses run_main_audit_workflow + reference_paths — so reference-path/symbol-graph regressions could not reproduce locally. Switches the harness to run_main_audit_workflow (matching src/commands/audit.rs), re-captures the snapshot, keeps the skips_test_paths invariant + determinism test. Now a #6855-style structural/symbol-graph regression fails locally. Harness test + fixture only; no detector/config change.