Repository navigation
fix(docker): Add platform specification for bbot-docker.sh helper - #3638
Conversation
Specify platform `linux/amd64` so Docker emulates correctly on any architecture
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA. |
|
recheck |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #3638 +/- ##
=====================================
- Coverage 91% 91% -0%
=====================================
Files 465 465
Lines 52188 52188
=====================================
- Hits 47280 47279 -1
- Misses 4908 4909 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
recheck |
|
@liquidsec It seems that the bot is requiring you to sign the CLA as well. I have a feeling that you may actually be affiliated with blacklanternsecurity, so there might be a second fix in the CLA pipeline that wasn't addressed yet by #3659 . |
|
recheck |
|
I have read the CLA Document and I hereby sign the CLA. |
|
recheck |
en0f
left a comment
There was a problem hiding this comment.
Reviewed at e89dde8.
🟢 The pin matches what ships, and docker is what receives it
blacklanternsecurity/bbot:stable is an OCI index holding only linux/amd64 plus an attestation manifest, and neither the 3.1.0 build nor the new docker.yml declares platforms:, so an arm64 Docker cannot resolve the tag without a platform. --platform sits before the image ref, so docker takes it and bbot never sees it. On an arm64 Mac (Docker 29.2.1), docker pull --platform linux/amd64 succeeds and the image prints 3.1.0 under emulation, while at the base the same pull fails with no matching manifest for linux/arm64/v8 in the manifest list entries.
Reproduce this review
# save the script below as verify-3638.sh, then (V04/V05 need Docker on an arm64 host; they SKIP elsewhere):
bash verify-3638.sh blacklanternsecurity/bbot e89dde821a3d4794fa500c9ad574efcee58cfe8b
# expected: SUMMARY 6 PASS, 0 FAIL, 0 SKIP, 0 BROKEN
# control: bash verify-3638.sh blacklanternsecurity/bbot c339f34c56b51b1a513f94ac2bce46937b32325a -> V01 V02 V04 V05 FAIL#!/usr/bin/env bash
# verify-3638.sh: reproducible checks for blacklanternsecurity/bbot#3638
# (bbot-docker.sh: pass --platform linux/amd64 to docker run)
# Usage: bash verify-3638.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
# Never executes bbot-docker.sh. Its docker arguments are parsed from the tree with
# shlex, and only the PUBLISHED image (not the author's code) is pulled and run, with no
# mounts, no env and no credentials. Registry reads are anonymous HTTPS.
#
# CONTROL: sha c339f34c56b51b1a513f94ac2bce46937b32325a
# CONTROL-EXPECT: V01=FAIL V02=FAIL V04=FAIL V05=FAIL
# AUTHOR-CODE: none
set -uo pipefail
REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass() { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail() { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip() { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }
echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "python3 $(python3 -c 'import sys;print(sys.version.split()[0])')"
echo "curl $(curl --version | head -1 | cut -d' ' -f2)"
echo "jq $(jq --version)"
echo "docker $(docker version --format '{{.Client.Version}}/{{.Server.Version}}' 2>/dev/null || echo unavailable)"
echo "host_arch $(uname -m)"
WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
export DOCKER_CLI_HINTS=false
GIT=(git -c credential.helper= -c 'credential.helper=!gh auth git-credential')
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi
# ---- parse the docker run line of bbot-docker.sh (never executed) -------------
# Prints three lines: image ref, platform value ('' if none), 1/0 platform-before-image.
python3 -I - "$WORK/src/bbot-docker.sh" > "$WORK/parsed" 2>"$WORK/parse.err" <<'PY'
import shlex, sys
line = next(l for l in open(sys.argv[1]) if l.lstrip().startswith("docker run"))
t = shlex.split(line, posix=True)
# docker run options taking a value, as used in this script
VAL = {"-v", "--volume", "--platform", "-e", "--env", "--name", "--network"}
i, img, plat, plat_idx = 2, None, "", None
while i < len(t):
a = t[i]
if a.startswith("--platform="):
plat, plat_idx = a.split("=", 1)[1], i; i += 1; continue
if a in VAL:
if a == "--platform":
plat, plat_idx = t[i + 1], i
i += 2; continue
if a.startswith("-"):
i += 1; continue
img = a; break
# a --platform after the image would be passed to bbot, not docker
after = any(x == "--platform" or x.startswith("--platform=") for x in t[i + 1:])
print(img or "")
print(plat)
print("1" if plat and plat_idx is not None and not after else "0")
PY
[ -s "$WORK/parsed" ] || { broken V00 "parse bbot-docker.sh" "$(head -c 200 "$WORK/parse.err")"; summary; exit 1; }
IMAGE="$(sed -n 1p "$WORK/parsed")"; PLATFORM="$(sed -n 2p "$WORK/parsed")"; BEFORE="$(sed -n 3p "$WORK/parsed")"
echo "parsed image=$IMAGE platform=${PLATFORM:-<none>} platform_before_image=$BEFORE"
[ -n "$IMAGE" ] || { broken V00 "parse bbot-docker.sh" "no image ref found"; summary; exit 1; }
# ---- V01: docker (not bbot) receives --platform linux/amd64 --------------------
if [ "$PLATFORM" = "linux/amd64" ] && [ "$BEFORE" = "1" ]; then
pass V01 "bbot-docker.sh passes --platform linux/amd64 to docker run, before the image ref"
else
fail V01 "bbot-docker.sh passes --platform linux/amd64 to docker run, before the image ref" "platform='${PLATFORM}' before_image=$BEFORE"
fi
# ---- registry: what the image tag the script uses actually publishes ------------
REPO_PATH="${IMAGE%%:*}"; TAG="${IMAGE##*:}"
ACCEPT='application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json'
TOKEN="$(curl -fsS "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${REPO_PATH}:pull" | jq -r .token 2>/dev/null)"
if [ -n "$TOKEN" ] && curl -fsS -D "$WORK/idx.hdr" -H "Authorization: Bearer $TOKEN" -H "Accept: $ACCEPT" \
"https://registry-1.docker.io/v2/${REPO_PATH}/manifests/${TAG}" -o "$WORK/idx.json"; then
echo "registry ${IMAGE} digest $(grep -i '^docker-content-digest' "$WORK/idx.hdr" | tr -d '\r' | cut -d' ' -f2)"
RUNNABLE="$(jq -r '[.manifests[]? | select(.platform.os != "unknown") | "\(.platform.os)/\(.platform.architecture)"] | join(",")' "$WORK/idx.json")"
echo "registry ${IMAGE} runnable_platforms=${RUNNABLE:-<single manifest>}"
# V02 (modeled arm64 host): the platform docker resolves on an arm64 host is published.
WANT="${PLATFORM:-linux/arm64}"
if printf '%s\n' "${RUNNABLE//,/$'\n'}" | grep -qx "$WANT"; then
pass V02 "on an arm64 host, the platform the script resolves ($WANT) is published for $IMAGE"
else
fail V02 "on an arm64 host, the platform the script resolves ($WANT) is published for $IMAGE" "published: ${RUNNABLE}"
fi
else
skip V02 "on an arm64 host, the platform the script resolves is published" "registry unreachable"
fi
# ---- V03: the publisher builds amd64 only, so the pin matches what ships --------
RW_REF="$(grep -oE 'release-workflows/\.github/workflows/docker\.yml@[0-9a-f]{40}' .github/workflows/publish.yml | head -1 | cut -d@ -f2)"
if [ -n "$RW_REF" ] && gh api --method GET -H 'Accept: application/vnd.github.raw+json' \
"repos/blacklanternsecurity/release-workflows/contents/.github/workflows/docker.yml?ref=$RW_REF" > "$WORK/rw_docker.yml" 2>/dev/null; then
echo "release-workflows docker.yml@$RW_REF sha256 $(shasum -a 256 "$WORK/rw_docker.yml" | cut -d' ' -f1)"
if grep -q 'docker/build-push-action@' "$WORK/rw_docker.yml" && ! grep -qE '^[[:space:]]+platforms:' "$WORK/rw_docker.yml"; then
pass V03 "the release docker build (docker.yml@${RW_REF:0:8}) declares no platforms:, so it ships the runner's amd64 only"
else
fail V03 "the release docker build declares no platforms:" "platforms: present or build-push-action missing"
fi
else
skip V03 "the release docker build declares no platforms:" "publish.yml pin or docker.yml read failed"
fi
# ---- V06 (challenge C1): today's :stable (3.1.0) came from tests.yml at tag v3.1.0, ---
# not docker.yml; that build path declares no platforms: either.
V310=beca7d2fadeae998acb23b6c5c375f42acceb316
if "${GIT[@]}" cat-file -e "$V310^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "refs/tags/v3.1.0:refs/tags/v3.1.0" 2>/dev/null; then
git show "$V310:.github/workflows/tests.yml" > "$WORK/tests310.yml" 2>/dev/null
if grep -q 'Publish to Docker Hub (stable)' "$WORK/tests310.yml" && grep -q 'docker/build-push-action@' "$WORK/tests310.yml" && ! grep -qE '^[[:space:]]+platforms:' "$WORK/tests310.yml"; then
pass V06 "the v3.1.0 stable image build (tests.yml publish_code at ${V310:0:8}) declares no platforms:"
else
fail V06 "the v3.1.0 stable image build declares no platforms:" "platforms: present or publish step missing"
fi
else
skip V06 "the v3.1.0 stable image build declares no platforms:" "v3.1.0 commit unavailable"
fi
# ---- V04/V05: behaviour on this host (arm64 only; published image, no mounts) ---
if [ "$(uname -m)" != "arm64" ] && [ "$(uname -m)" != "aarch64" ]; then
skip V04 "docker pull with the script's platform args succeeds on an arm64 host" "host is $(uname -m)"
skip V05 "the image runs bbot --version on an arm64 host with the script's platform args" "host is $(uname -m)"
elif ! docker info >/dev/null 2>&1; then
skip V04 "docker pull with the script's platform args succeeds on an arm64 host" "docker daemon unavailable"
skip V05 "the image runs bbot --version on an arm64 host with the script's platform args" "docker daemon unavailable"
else
PARGS=(); [ -n "$PLATFORM" ] && PARGS=(--platform "$PLATFORM")
if docker pull -q ${PARGS[@]+"${PARGS[@]}"} "$IMAGE" > "$WORK/pull.out" 2>&1; then
pass V04 "docker pull ${PARGS[*]-} $IMAGE succeeds on an arm64 host"
if docker run --rm --network none ${PARGS[@]+"${PARGS[@]}"} "$IMAGE" --version > "$WORK/run.out" 2>&1; then
pass V05 "the image runs bbot --version on an arm64 host with the script's platform args: $(tail -1 "$WORK/run.out" | head -c 80)"
else
fail V05 "the image runs bbot --version on an arm64 host with the script's platform args" "$(tail -2 "$WORK/run.out" | tr '\n' ' ' | head -c 200)"
fi
else
fail V04 "docker pull ${PARGS[*]-} $IMAGE succeeds on an arm64 host" "$(tail -2 "$WORK/pull.out" | tr '\n' ' ' | head -c 200)"
fail V05 "the image runs bbot --version on an arm64 host with the script's platform args" "pull failed"
fi
fi
summary
Specify platform
linux/amd64so Docker emulates correctly on any architectureWhat changed
Adds in a
--platform "linux/amd64"option in thedocker runcommand. Because the bbot Docker image is built for a single architecture, this ensures that that image is pulled and emulated on local hardware, regardless of what system it's running on.Closes #
Test artifacts
Before
Doesn't run on Docker on macOS
After
Pulls image from Docker repository
Installs all dependencies without error
Runs test
-p kitchen-sinkscan successfullyTested on Apple Silicon (MacOS 27.3 beta, M3 Pro/arm64) and Raspberry Pi 5 (Trixie, arm64).
Checklist
AI Use Disclosure
No AI used.