ci: Add Packit CI with RPM builds and TMT integration tests by henrywang · Pull Request #3576 · ostreedev/ostree

henrywang · 2026-03-20T08:51:59Z

Add Packit-based CI pipeline that builds RPMs via COPR and runs TMT integration tests on bootc image-mode systems. This follows the bootc-dev/bootc pattern of per-plan VM isolation using bcvk.

Key changes:

Dockerfile: Add rpmbuild stage, use RPM overlay for rootfs, run provision-derived.sh for VM provisioning (cloud-init, rsync, etc.)
Justfile: Add package target, test-tmt target with bcvk per-plan VMs, longer SSH wait for cloud-init first boot
.github/workflows/bootc.yaml: Split into unit-tests and integration jobs, archive TMT logs with PR number in artifact name
.packit.yaml: COPR builds + TMT tests for centos-stream-9/10 and fedora-43/44 on x86_64/aarch64
tmt/: FMF test plans and shell-based tests (booted verification, privileged ostree tests) translated from Rust integration tests
hack/: Packit provisioning scripts to convert package-mode VMs to image-mode via bootc install to-filesystem

openshift-ci · 2026-03-20T08:52:10Z

Hi @henrywang. Thanks for your PR.

I'm waiting for a ostreedev member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

gemini-code-assist

Code Review

This pull request introduces a Packit-based CI pipeline, which is a significant enhancement. The changes are extensive, adding new configurations for Packit and TMT, modifying the Dockerfile, and including new test scripts. My review has identified a critical bug in a provisioning script that needs to be addressed, as well as several instances of code duplication and opportunities to improve the robustness and maintainability of the new CI infrastructure. Addressing these points will help ensure the new pipeline is reliable and easy to manage.

gemini-code-assist · 2026-03-20T08:53:33Z

+# Stock extra cleaning of logs and caches in general (mostly dnf)
+rm /var/log/* /var/cache /var/lib/{dnf,rpm-state,rhsm} -rf
+# And clean root's homedir
+rm /var/roothome/.config -rf


The arguments for the rm command are in a non-standard order. While this may work with GNU rm, the standard and more portable syntax is rm [options] [file...]. Please adjust for correctness and clarity.

Suggested change

rm /var/roothome/.config -rf

rm -rf /var/roothome/.config

gemini-code-assist · 2026-03-20T08:53:33Z

+    playbook:
+      - https://github.com/bootc-dev/bootc/raw/refs/heads/main/hack/packit-reboot.yml


Fetching the Ansible playbook from an external GitHub URL makes the tests fragile. The URL could change, or the resource could become temporarily unavailable, causing test failures. It would be more robust to vendor this playbook within the repository to ensure test reliability.

packit-as-a-service · 2026-03-20T09:10:51Z

Failed to load packit config file:

Cannot parse package config. ValidationError({'jobs': {0: {'packages': defaultdict(<class 'dict'>, {'ostree': {'value': {'actions': ["Unknown action(s) provided: ['post_upstream_clone']"]}}})}, 1: {'packages': defaultdict(<class 'dict'>, {'ostree': {'value': {'actions': ["Unknown action(s) provided: ['post_upstream_clone']"]}}})}}, 'packages': defaultdict(<class 'dict'>, {'ostree': {'value': {'actions': ["Unknown action(s) provided: ['post_upstream_clone']"]}}})})

For more info, please check out the documentation or contact the Packit team. You can also use our CLI command config validate or our pre-commit hooks for validation of the configuration.

cgwalters · 2026-03-20T17:33:53Z

+        if tmt run --id "${vm_name}" --all \
+            provision --how connect \


That said it is probably time to open up the conversation with tmt upstream about supporting bcvk directly...it would remove a lot of gnarly code.

cgwalters · 2026-03-20T21:45:05Z

One approach may be for us to factor out like a testutils shared crate - doesn't have to be published on crates.io but could be bootc-dev/testutils and we reuse that in various repos?

➡️ bootc-dev/bcvk#231

Add Packit-based CI pipeline that builds RPMs via COPR and runs TMT integration tests on bootc image-mode systems. This follows the bootc-dev/bootc pattern of per-plan VM isolation using bcvk. Key changes: - Dockerfile: Add rpmbuild stage, use RPM overlay for rootfs, run provision-derived.sh for VM provisioning (cloud-init, rsync, etc.) - Justfile: Add package target, test-tmt target with bcvk per-plan VMs, longer SSH wait for cloud-init first boot - .github/workflows/bootc.yaml: Split into unit-tests and integration jobs, archive TMT logs with PR number in artifact name - .packit.yaml: COPR builds + TMT tests for centos-stream-9/10 and fedora-43/44 on x86_64/aarch64 - tmt/: FMF test plans and shell-based tests (booted verification, privileged ostree tests) translated from Rust integration tests - hack/: Packit provisioning scripts to convert package-mode VMs to image-mode via bootc install to-filesystem Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>

VM deployment is now handled externally by `just test-tmt` (bcvk + tmt) or `just integration-container` (bcvk direct SSH). The Rust test binary runs inside the VM as root, so the require_root/RunMode dispatch logic is no longer needed. - Remove require_root(), RunMode enum, and bcvk dispatch code - Simplify booted_test! and privileged_test! macros to just register and run tests directly - Remove rustix dependency (no longer checking getuid) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>

Replace the inline bash script in the Justfile test-tmt target with a Rust xtask crate (tests/xtask/) that handles TMT plan discovery, bcvk VM lifecycle, SSH readiness polling, and tmt invocation. This follows the bootc-dev/bootc cargo xtask run-tmt pattern. Also fix tests.fmf to pass test names individually with --exact, since libtest_mimic only accepts a single filter argument. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>

Set CARGO_HOME=/var/tmp/.cargo to avoid conflict with /root/.cargo which exists as a non-directory on Fedora 43/44 base images, causing "failed to create directory: File exists (os error 17)". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Xiaofeng Wang <henrywangxf@me.com>

cgwalters · 2026-03-30T18:21:30Z

Looks like it needs cargo fmt, we should also sync over from bootc the linting/style checks.

openshift-ci Bot added do-not-merge/work-in-progress needs-ok-to-test labels Mar 20, 2026

gemini-code-assist Bot reviewed Mar 20, 2026

View reviewed changes

henrywang force-pushed the tmt-integration branch from 271e988 to fbd56f9 Compare March 20, 2026 09:10

henrywang force-pushed the tmt-integration branch 2 times, most recently from 7d096f7 to 2b2a175 Compare March 20, 2026 10:42

henrywang changed the title ~~[WIP] ci: Add Packit CI with RPM builds and TMT integration tests~~ ci: Add Packit CI with RPM builds and TMT integration tests Mar 20, 2026

henrywang marked this pull request as ready for review March 20, 2026 11:21

openshift-ci Bot removed the do-not-merge/work-in-progress label Mar 20, 2026

cgwalters requested changes Mar 20, 2026

View reviewed changes

henrywang mentioned this pull request Mar 23, 2026

itest: Add reusable integration test library crate bootc-dev/bcvk#231

Merged

henrywang force-pushed the tmt-integration branch from 2b2a175 to b0cd80d Compare March 23, 2026 01:30

henrywang and others added 3 commits March 23, 2026 16:56

henrywang requested a review from cgwalters March 30, 2026 13:10

cgwalters approved these changes Mar 30, 2026

View reviewed changes

cgwalters merged commit eb27be7 into ostreedev:main Mar 30, 2026
43 of 46 checks passed

henrywang deleted the tmt-integration branch March 31, 2026 01:33

		playbook:
		- https://github.com/bootc-dev/bootc/raw/refs/heads/main/hack/packit-reboot.yml

		if tmt run --id "${vm_name}" --all \
		provision --how connect \

Conversation

henrywang commented Mar 20, 2026

Uh oh!

openshift-ci Bot commented Mar 20, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist Bot Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

packit-as-a-service Bot commented Mar 20, 2026

Uh oh!

Uh oh!

Uh oh!

cgwalters Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

cgwalters commented Mar 20, 2026

Uh oh!

cgwalters commented Mar 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants