Change-aware GHDL/cocotb CI: run only affected cocotb tests on feature-branch pushes - #1449
Draft
ruck314 wants to merge 1 commit into
Draft
Change-aware GHDL/cocotb CI: run only affected cocotb tests on feature-branch pushes#1449ruck314 wants to merge 1 commit into
ruck314 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR makes the GHDL/cocotb CI regression “change-aware” on feature-branch pushes by resolving which cocotb tests are affected by the RTL (and wrapper) changes in the push, while preserving the full-suite behavior for main, tags, and PRs targeting main.
Changes:
- Add a dependency resolver (
tests/common/dep_map.py) that discovers cocotb toplevels, builds a GHDL-derived transitive source dependency map, and selects an affected test subset with conservative fail-open rules. - Add a CLI entrypoint (
python -m tests.common) to emit either selected pytest module targets or aFORCE_FULLsentinel. - Update CI workflow to run either the full suite or the resolver-selected subset depending on trigger type, including installing an LLVM GHDL only for selective runs.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/common/dep_map.py |
Implements discovery, dependency mapping, wrapper attribution, and test selection logic for selective CI runs. |
tests/common/__main__.py |
CLI wrapper that executes the resolver and prints selected targets or FORCE_FULL. |
tests/common/test_dep_map.py |
Unit tests covering resolver behavior, fail-open conditions, wrapper attribution, and CLI fail-open behavior. |
.github/workflows/surf_ci.yml |
Adds selective/full run mode selection and uses the resolver output to constrain pytest targets on feature-branch pushes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Run only the cocotb tests whose RTL dependencies actually changed on feature-branch pushes, while tags, main, and PRs into main still run the complete suite. The dependency graph is derived from the actual build (build/*.cf + `ghdl --gen-depends`) at CI time, so it cannot drift from the RTL, and every indeterminate case fails safe toward running more tests. - tests/common/dep_map.py (importable core) + `python -m tests.common` CLI: map each cocotb test (by its toplevel) to its transitive RTL sources via build/*.cf and `ghdl --gen-depends`, diff against the merge-base with origin/main, and print the affected pytest targets (or a FORCE_FULL sentinel). Any indeterminate case falls back to always-run or a full run. - Wrapper / IP-integrator resolution: most cocotb tests wrap their DUT in a **/wrappers/ or **/ip_integrator/ entity (cocotb cannot drive VHDL records), and `make import` never compiles those sources -- so their toplevels are absent from the surf library and `ghdl --gen-depends` cannot resolve them. Import those sources into the resolver's library first (with absolute paths, so the .cf entries stay REPO_ROOT-resolvable) so each such test resolves to its real transitive dependency set instead of falling back to always-run. Subsystems whose core RTL is family-gated out of the generic GHDL build (pgp2b, pgp2fc, coaxpress) remain always-run by fail-safe. - Changed-wrapper attribution: a changed **/wrappers/ file selects exactly the test(s) whose toplevel instantiates it (or FORCE_FULL if unattributable) and never fans out as another test's production dependency. - Shared base-package changes (e.g. StdRtlPkg, AxiLitePkg, AxiStreamPkg, or any high-fan-out hub) force a full run, detected by a data-driven fan-out threshold over the built dependency map. - Strict-subset guarantee: a runtime guard intersects the selection with the discovered test universe before emitting it, and a randomized property test asserts the selected set is always a subset of the full suite. - .github/workflows/surf_ci.yml: trigger on push and pull_request into main (preserving concurrency cancel-in-progress); selective on feature-branch pushes, full on tag/main/PR->main; fail open to the full suite on resolver error. The regression job installs GHDL's official LLVM build via ghdl/setup-ghdl (v6.0.0) for --gen-depends only, and runs the cocotb simulations on the apt mcode ghdl. The lint/syntax job is unchanged. - tests/common/test_dep_map.py: unit and property tests for the parser, the wrapper index, wrapper/ip_integrator source discovery and import, selection routing, base-package force-full, and the strict-subset invariant.
ruck314
force-pushed
the
ci-test-cherry-pick-using-GHDL-cf-file
branch
from
July 3, 2026 04:30
a5a9de4 to
d4795fb
Compare
ruck314
marked this pull request as ready for review
July 3, 2026 04:48
ruck314
marked this pull request as draft
July 8, 2026 18:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Make the GHDL/cocotb regression job change-aware: on ordinary feature-branch pushes it runs only the cocotb tests whose RTL dependencies actually changed, while the release path (tags,
main, and PRs intomain) still runs the complete suite. Previously every push re-ran the entire suite.The dependency graph is derived from the actual build (
build/*.cf+ghdl --gen-depends) at CI time — never a hand-maintained list — so it cannot drift from the RTL. Every indeterminate case fails safe toward running more tests, so selective mode can never skip a test the full suite would have run. Because cocotb cannot drive VHDL records, most tests wrap their DUT in a**/wrappers/or**/ip_integrator/entity thatmake importnever compiles; those sources are imported into the resolver's library first so their toplevels resolve to real dependency sets instead of falling back to always-run. Subsystems whose core RTL is family-gated out of the generic GHDL build (pgp2b, pgp2fc, coaxpress) remain always-run by fail-safe.CI trigger behavior
When the full suite runs vs. only the affected tests:
mainand not a tag)mainrefs/tags/*)mainmainbranch (e.g.pre-release)pull_requesttrigger is scoped tomain; the branch's own push already ran the selective job