Skip to content

feat(audit): item-counting grammar to config (#6855 phase 1, harness-gated)#6915

Closed
chubes4 wants to merge 1 commit into
mainfrom
feat/language-grammar-phase1-v3
Closed

feat(audit): item-counting grammar to config (#6855 phase 1, harness-gated)#6915
chubes4 wants to merge 1 commit into
mainfrom
feat/language-grammar-phase1-v3

Conversation

@chubes4

@chubes4 chubes4 commented Jun 28, 2026

Copy link
Copy Markdown
Member

Third attempt at #6855 Phase 1, gated by the fidelity-upgraded runtime audit harness (#6914) which now exercises test-path skipping AND the agnostic detector — the two behaviors #6896/#6906 broke. Extracts structural.rs item counting into config-driven LanguageGrammar; enum Language/symbol_graph/is_empty untouched. All 3 audit_runtime_regression harness tests + structural parity pass with unchanged findings.

@homeboy-ci

homeboy-ci Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Homeboy Results — homeboy

Lint

lint — passed

ℹ️ Full options: homeboy docs commands/lint
ℹ️ Save lint baseline: homeboy lint homeboy --baseline
Deep dive: homeboy lint homeboy --changed-since fd23417

Artifacts and drill-down
  • CI results artifact: homeboy-ci-results-homeboy-lint-homeboy-Linux contains immediate command JSON for this action invocation.
  • Observation artifact: homeboy-observations-homeboy-lint-homeboy-Linux contains exported Homeboy run history for deeper queries.
  • Drill-down: download the observation artifact, then run homeboy runs import <dir>, homeboy runs list, and homeboy runs findings <run-id>.
  • Artifacts are attached to the workflow run: https://github.com/Extra-Chill/homeboy/actions/runs/28329485871

Test

⚠️ test — baseline red

ℹ️ No tests ran — the runner failed before producing results. See raw_output.stderr_tail / raw_output.stdout_tail for the underlying error (bootstrap failure, missing deps, DB connection, etc.).
ℹ️ To run specific tests: homeboy test homeboy -- --filter=TestName
ℹ️ Auto-fix lint issues: homeboy refactor homeboy --from lint --write
ℹ️ Collect coverage: homeboy test homeboy --coverage
ℹ️ Analyze failures: homeboy test homeboy --analyze
ℹ️ Pass args to test runner: homeboy test -- [args]
ℹ️ Full options: homeboy docs commands/test
Deep dive: homeboy test homeboy --changed-since fd23417

Artifacts and drill-down
  • CI results artifact: homeboy-ci-results-homeboy-test-homeboy-Linux contains immediate command JSON for this action invocation.
  • Observation artifact: homeboy-observations-homeboy-test-homeboy-Linux contains exported Homeboy run history for deeper queries.
  • Drill-down: download the observation artifact, then run homeboy runs import <dir>, homeboy runs list, and homeboy runs findings <run-id>.
  • Artifacts are attached to the workflow run: https://github.com/Extra-Chill/homeboy/actions/runs/28329485871

Audit

audit — failed

  • core_boundary_leak:core-agnostic-source — 70 finding(s)
  • structural — 57 finding(s)
  • intra-method-duplication — 47 finding(s)
  • field_patterns — 35 finding(s)
  • dead_code — 19 finding(s)
  • near-duplication — 17 finding(s)
  • test_quality — 13 finding(s)
  • duplication — 10 finding(s)
  • remote_execution_preflight — 9 finding(s)
  • thin_command_adapter — 8 finding(s)
  • Total: 311 finding(s)

Deep dive: homeboy audit homeboy --changed-since fd23417

Artifacts and drill-down
  • CI results artifact: homeboy-ci-results-homeboy-audit-homeboy-Linux contains immediate command JSON for this action invocation.
  • Observation artifact: homeboy-observations-homeboy-audit-homeboy-Linux contains exported Homeboy run history for deeper queries.
  • Drill-down: download the observation artifact, then run homeboy runs import <dir>, homeboy runs list, and homeboy runs findings <run-id>.
  • Artifacts are attached to the workflow run: https://github.com/Extra-Chill/homeboy/actions/runs/28329485871
Tooling versions
  • Homeboy CLI: homeboy 0.269.0+2ef12653d00f+7c8f8846c
  • Extension: rust from https://github.com/Extra-Chill/homeboy-extensions
  • Extension revision: dfe9fa31
  • Action: unknown@unknown

@chubes4

chubes4 commented Jun 28, 2026

Copy link
Copy Markdown
Member Author

Held as draft — root cause of the harness fidelity gap FOUND

Third attempt. The fidelity-upgraded harness (#6914) passes all 3 tests (matches_snapshot, skips_test_paths, deterministic) AND structural parity passes (24 tests). Yet CI Audit fails identically to #6896/#6906: delta: 300, 158 CoreBoundaryLeak (82 on tests/ paths), 94 IntraMethodDuplicate, 84 HighItemCount...

Root cause: the harness exercises the WRONG audit entry point

  • The regression-harness calls audit_path_with_id (src/core/code_audit/entry.rs), which loads config via discover_from_portable and audits a path directly.
  • The CI Audit gate / homeboy audit CLI (src/commands/audit.rs::run) calls run_main_audit_workflow with reference_paths (resolve_audit_reference_paths).

These are DIFFERENT code paths. reference_paths drive how the symbol-graph / cross-reference detectors (MissingMethod, NamingMismatch) and likely the test-path/fingerprint handling resolve against the whole codebase. The structural item-grammar migration perturbs something in the run_main_audit_workflow + reference-path path that audit_path_with_id simply doesn't exercise — which is why the toy fixture (and even the enriched one) can't reproduce it, but the real CLI audit fails every time.

The actionable fix (prerequisite for #6855)

The runtime audit regression harness must call run_main_audit_workflow (the same path CI uses), not audit_path_with_id. Once it does, this exact +300 regression reproduces locally and becomes diagnosable. Until then, no structural.rs/symbol-graph-adjacent change can be safely validated locally.

This is the real fidelity gap (deeper than the fixture-content gap fixed in #6914). I'll redirect to upgrading the harness to use the CLI workflow entry point.

Migration itself is correct in isolation (parity + fixture harness pass); held pending the workflow-path harness.

@chubes4

chubes4 commented Jun 28, 2026

Copy link
Copy Markdown
Member Author

Status: held as draft. Root cause is a real-codebase-specific audit regression that synthetic-fixture harnesses can't reproduce (proven in #6855 / #6917). Blocked on #6920 (real-repo drift harness). Full consolidated trail: #6855. Migration code here is correct and salvageable once #6920 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant