Goal
Add automated coverage proving that an invalid, malformed, or mismatched
release BOM blocks basectl release check and basectl release publish, and
that bom_finding maps every ReleaseBomError class to a blocking finding.
Background
From the 2026-09-09 deep review of the release-BOM feature (Base 1.9.0):
cli/python/base_release/tests/test_engine.py::test_publish_with_bom_uploads_and_verifies_stable_release_assets
mocks base_release.engine.release_findings (returns READY_FINDINGS) and
writes a non-BOM stub as the --bom input
(bom_bytes = b'{"release":"base-1.9.0"}\n'). It therefore never exercises
bom_finding / validate_bom_file, and it would pass even if BOM validation
were removed entirely.
cli/python/base_release/release_readiness.py::bom_finding has no direct
tests. There is no test_release_readiness.py at all, so
inspect_release_provenance and the expected_commit=provenance.commit_sha
wiring are only covered indirectly through heavily-mocked engine tests.
release_bom.validate_bom itself is well covered by
cli/python/base_release/tests/test_release_bom.py. The gap is the wiring
that turns a bad BOM into a failed release.
Net: the property the feature exists for -- "a bad BOM fails the release" -- is
unverified end to end.
Scope
cli/python/base_release/tests/test_release_readiness.py (new) or additions
to test_engine.py: unit-test bom_finding for each branch --
ctx.bom_path is None (ok), missing file (error), malformed JSON (error),
schema-invalid document (error), repository / version / commit mismatch
(error), happy path (ok).
- Engine-level tests using a real BOM file on disk (no mock of
release_findings):
release check --bom <invalid> --format json exits non-zero and emits a
bom finding with status: error.
release publish --bom <invalid> is blocked and performs no git / gh
mutations.
- Replace the non-BOM stub in the existing publish-with-BOM test with a
canonical, schema-conformant BOM fixture (or add a parallel test that uses
one).
- Add a small helper/fixture that builds a valid
basefoundry/base-owned BOM;
current fixtures (tests/fixtures/release-bom-valid.json) model a
base-bash-libs release.
Acceptance Criteria
Validation
python -m pytest cli/python/base_release/tests -q
bin/base-test
git diff --check
Non-Goals
- No changes to
validate_bom rules, the BOM schema, or the wire format
(tracked separately in the schema-reconciliation and validator-tightening
issues).
- No new CLI flags or manifest fields (BOM enforcement is a separate issue).
Project Fields
- Status: Backlog
- Priority: P1
- Size: S
- Area: CI
- Initiative: Contract Hardening
- Milestone: v1.10.0
Agent Assignment
Human first. Straightforward test-authoring task once the fixture shape is
agreed; suitable for a Copilot candidate after triage.
Goal
Add automated coverage proving that an invalid, malformed, or mismatched
release BOM blocks
basectl release checkandbasectl release publish, andthat
bom_findingmaps everyReleaseBomErrorclass to a blocking finding.Background
From the 2026-09-09 deep review of the release-BOM feature (Base 1.9.0):
cli/python/base_release/tests/test_engine.py::test_publish_with_bom_uploads_and_verifies_stable_release_assetsmocks
base_release.engine.release_findings(returnsREADY_FINDINGS) andwrites a non-BOM stub as the
--bominput(
bom_bytes = b'{"release":"base-1.9.0"}\n'). It therefore never exercisesbom_finding/validate_bom_file, and it would pass even if BOM validationwere removed entirely.
cli/python/base_release/release_readiness.py::bom_findinghas no directtests. There is no
test_release_readiness.pyat all, soinspect_release_provenanceand theexpected_commit=provenance.commit_shawiring are only covered indirectly through heavily-mocked engine tests.
release_bom.validate_bomitself is well covered bycli/python/base_release/tests/test_release_bom.py. The gap is the wiringthat turns a bad BOM into a failed release.
Net: the property the feature exists for -- "a bad BOM fails the release" -- is
unverified end to end.
Scope
cli/python/base_release/tests/test_release_readiness.py(new) or additionsto
test_engine.py: unit-testbom_findingfor each branch --ctx.bom_path is None(ok), missing file (error), malformed JSON (error),schema-invalid document (error), repository / version / commit mismatch
(error), happy path (ok).
release_findings):release check --bom <invalid> --format jsonexits non-zero and emits abomfinding withstatus: error.release publish --bom <invalid>is blocked and performs nogit/ghmutations.
canonical, schema-conformant BOM fixture (or add a parallel test that uses
one).
basefoundry/base-owned BOM;current fixtures (
tests/fixtures/release-bom-valid.json) model abase-bash-libsrelease.Acceptance Criteria
bom_findinghas direct unit tests covering every error class and the okpath.
release publish --bom <invalid>returnsFAILUREand issues no git/gh commands.release check --bom <invalid> --format jsonreports
status: errorand abomfinding.schema-conformant BOM.
base-BOM fixture/builder exists for release tests.Validation
Non-Goals
validate_bomrules, the BOM schema, or the wire format(tracked separately in the schema-reconciliation and validator-tightening
issues).
Project Fields
Agent Assignment
Human first. Straightforward test-authoring task once the fixture shape is
agreed; suitable for a Copilot candidate after triage.