You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Rollback drill image drifts from the release it is built on, with no signal #691
The rollback drill image is built FROM one bosun release, pinned by digest, and does not track releases. DRILL_REF in scripts/rollback-drill.sh:85 is a hardcoded digest whose base is 0.43.1.
Nothing notices when it drifts, and the symptom is easy to misread. A drifted base makes the shadow render report RENDER-DIFFERS and list files, because the base and the running incumbent are different releases. That is expected noise, not a failure — answering y still cuts over and still exercises rollback. But an operator who did not build the image has no way to tell that from a real render difference, and the safe-looking answer, n, ends the run at exit 0 DECLINED at RENDER-DIFFERS with the rollback still unverified.
(Corrected: an earlier version of this issue said the symptom was exit 5 CANDIDATE-FAILED. It is not. That verdict means the drill image failed to render at all — a broken shim, not drift.)
Why this is low severity
A stale drill image is not dangerous. It is pinned by digest, lives in a private package, is never pulled by anything expecting a release, and only runs when an operator starts a drill. The cost is a judgement call the operator has no information to make.
Suggested Fix
Not a scheduled rebuild. A job that publishes a deliberately broken bosun image on a timer, unwatched, is a worse failure mode than a stale one, and the drill needs an operator at a prompt regardless.
Make the drill say what it knows.rollback-drill.sh can read the drill image's bosun --version and the running incumbent's before it touches anything, and print the comparison up front:
drill image is built on 0.43.1, incumbent is 0.45.0 — RENDER-DIFFERS below is expected. Answer y to drill anyway, or rebuild first: docs/rollback-drill-image.md
That converts the ambiguous prompt into an informed one without refusing anything. It costs one docker run and one docker exec, both of which the script already does elsewhere.
A CI freshness check comparing the base digest against the current release is possible but more machinery; only worth it if the above proves insufficient. Neither option should rebuild or publish automatically.
Context
Rebuild recipe: docs/rollback-drill-image.md. First and only drill to date: 2026-09-20, exit 1 ROLLED-BACK, against a base matching the then-current release.
Activity
cameronsjo commentedon Oct 14, 2026
Claimed by a local chief-of-staff sweep; a draft PR will reference this issue.
fix(ops): exclude stderr from drill version probes