Skip to content

bootupd CI: composefs e2e boot broken by bootc composefs.digest= karg vs 1.16 initramfs #34

Closed as not planned

Description

@cgwalters-bot

coreos/bootupd CI bootc-e2e (ubuntu-24.04, 0, 10) composefs UEFI boot now fails in the initrd: initrd-switch-root.service fails and it drops to emergency mode; bootc-root-setup.service never runs.

Cause: the test image upgrades bootc from bootc main (now 202609250143.gf62a0ed3a3; the last green run on 9/21 used g65321aed79) but keeps the centos-bootc stream10 initramfs (bootc 1.16.13). Since bootc ba10fb16fc (Generate both V1 and V2 EROFS images on install), install writes only composefs.digest=v1-sha512-12:... instead of composefs=..., and the 1.16.13 bootc-root-setup.service has ConditionKernelCommandLine=composefs, so it is skipped. The new unit also accepts composefs.digest, but that initramfs is not regenerated.

Evidence: failing https://github.com/coreos/bootupd/actions/runs/36194645623/job/108275112081 (cmdline has composefs.digest=, no bootc-root-setup) vs green main https://github.com/coreos/bootupd/actions/runs/35635888015/job/106453076598 (composefs=, bootc-root-setup ran). First seen on coreos/bootupd#1163, which does not touch install.

Q: should bootc keep writing composefs= on install for older initramfs (compat), or should bootupd CI regenerate the initramfs after upgrading bootc?

Activity

  1. converted this from a draft issue on Sep 26, 2026
  2. moved this from In Review to Done in Workstreamon Oct 9, 2026
  3. cgwalters-bot commented on Oct 17, 2026

    @cgwalters-bot
    CollaboratorAuthor

    Board scheduler: no plan. A triage run is asked whether this is done; it answers here. Delete this comment to ask again.

  4. github-actions commented on Oct 17, 2026

    @github-actions

    ✅ dispatch ended well. Result: #34 (comment)

  5. cgwalters-bot commented on Oct 17, 2026

    @cgwalters-bot
    CollaboratorAuthor

    Conclusion: partly fixed, but the rest can't be planned yet. The fix the issue asks for has merged in coreos/bootupd. The composefs boot that the issue says is broken is still skipped on stream10 in bootupd CI, so nothing shows it passes. Re-enabling it is a one-line change in coreos/bootupd, and that repository isn't in the dispatch bounds (.github/agentic-job/dispatch.toml only lists cgwalters-forge/*). So I can't write an implement plan for it that would be accepted.

    What landed

    • coreos/bootupd#1168 (merged 9 Oct, 97658ead): "ci: Regenerate the initramfs after installing bootc from copr". It runs dracut --force in the Dockerfile after dnf install bootc, so the initramfs's bootc-root-setup.service also starts on composefs.digest=. This is the second option in the issue's question. The PR says a newer bootc with an older initramfs isn't supported. Nothing in bootc changed to write composefs= again for older initramfs; I found no such PR in bootc-dev/bootc since 26 Sep. The PR author booted composefs UEFI to login: on a devspace, but only for stream10 x86_64.
    • On main, bootc-e2e (ubuntu-24.04, 0, 10) passes: run 38204910006, job 114666138189. The log shows the dracut --force --kver 6.12.0-275.el10... step.

    What's left

    • coreos/bootupd#1164 (merged 1 Oct, commit 6969be576c "ci: Skip composefs tests for C10S") added if [[ "${{ matrix.stream }}" != "10" ]] around test-uefi-vm-boot.sh "$IMG_NAME" "composefs" in .github/workflows/ci.yml. Because of that, the green stream10 job above only boots ostree. That skip is still on main.
    • Fix: remove that guard (and its TODO), now that #1168 regenerates the initramfs, then confirm (ubuntu-24.04, 0, 10) and (ubuntu-24.04-arm, 0, 10) boot composefs to login. That's a PR to coreos/bootupd.
    • Unrelated: the latest main CI run (38906095547) failed in (ubuntu-24.04-arm, 1). dnf download grub2-efi-aa64-cc from Fedora ELN failed after a mirror returned HTTP 500. It isn't this issue (ci: Fetch grub-cc from Fedora Koji bootc-dev/bootc#2575 moved bootc's own CI to fetch grub-cc from Koji).

    Needed before it can be planned: a decision from cgwalters on either

    1. adding coreos/bootupd to the dispatch bounds so an implement run can open the PR above, or
    2. opening that PR outside the board (or closing this as done upstream, if the stream10 skip is meant to stay).
    {"request":"563cf514-a4d3-4681-a5f5-64f649f8fb1f","verdict":"needs-design"}
    

    Generated by dispatch · ◷

  6. cgwalters-bot commented on Oct 17, 2026

    @cgwalters-bot
    CollaboratorAuthor

    The fix this issue asked for merged as coreos/bootupd#1168, which regenerates the initramfs after installing bootc, and the board already shows Done. The one remaining step is to re-enable the stream10 composefs boot leg in bootupd CI. That change is in coreos/bootupd, which is outside the dispatch bounds, and it does not gate the composefs-stable epic. See #34 (comment). Reopen this if the operator wants it tracked here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions