Skip to content

ci: CLA caller v1.3.0 on stable - #3659

Merged
pjhartlieb merged 1 commit into
stablefrom
ci/cla-v1.3.0-stable
Nov 18, 2026
Merged

pjhartlieb merged 1 commit into
stablefrom
ci/cla-v1.3.0-stable

Conversation

@pjhartlieb

Copy link
Copy Markdown
Contributor

Puts the v1.3.0 CLA caller that dev already carries (identical file) on stable, the default branch. CLA runs from the default branch's copy, and the old copy saves signatures where they can no longer be written, so outside contributors cannot sign until the next release (#3658 today). Only this file changes; the next release has no conflict on it. Org CLA rollout; same change as MANSPIDER#138 and badsecrets#472.

After merge: cla / CLAAssistant becomes required on stable and dev. Contributors whose signature failed to save comment recheck or sign again.

Puts dev's cla.yml (release-workflows cla-reusable.yml v1.3.0) on stable,
the default branch, which is the copy pull_request_target and
issue_comment run. The old copy saves signatures on CLA main, which no
longer accepts writes, so outside contributors cannot sign until this
lands.
@pjhartlieb
pjhartlieb requested review from en0f and liquidsec November 18, 2026 14:31
@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91%. Comparing base (beca7d2) to head (adda64f).
⚠️ Report is 2 commits behind head on stable.

Additional details and impacted files
@@          Coverage Diff           @@
##           stable   #3659   +/-   ##
======================================
+ Coverage      91%     91%   +1%     
======================================
  Files         465     465           
  Lines       51927   51927           
======================================
+ Hits        47031   47048   +17     
+ Misses       4896    4879   -17     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@singlerider singlerider added the github_actions Pull requests that update GitHub Actions code label Nov 18, 2026

@en0f en0f left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at adda64f.

🟢 Signature saves move off CLA's write-refused main, in the copy the triggers actually load

issue_comment and pull_request_target load stable's cla.yml, so only this copy decides where a signature is saved. Today it calls CLA/cla-reusable.yml@4f85d3c, which saves on CLA's main, and run 44293705592 this morning failed with Could not update the JSON file: Repository rule violations found. v1.3.0 saves on the signatures branch instead (V12). Targeting stable directly is the right exception to "target dev": the file is byte-identical to dev's (blob 31bc4d38, V07), so the next release merges nothing in it.

🟢 Least privilege for a pull_request_target job

secrets: inherit is gone. The caller passes exactly APP_ID and APP_PRIVATE_KEY, the two secrets v1.3.0 declares, and both reach bbot as org secrets. contents: read is dropped because the called job asks only for pull-requests: write and statuses: write, which is exactly what's granted here. The called job checks out no code, and the pin is tag v1.3.0's commit (ci.md § Permissions, § Pin everything to a SHA; V01 to V06).

📝 Test artifact pending: the first signature saved on CLA's signatures branch

No run of this caller can exist before merge, and nowhere in the org has v1.3.0 saved a signature yet: signatures holds only its seed commit f8010baf (V11), and the one v1.3.0 run that minted the App token (MANSPIDER 43966084823, V10) took the all-exempt path. The save needs the App to be a bypass actor on CLA's signatures-bot-only ruleset, and my token can't see bypass actors. Nothing is needed in this PR. Confirm the App is a bypass actor there, or link the first post-merge run that saves an outside contributor's signature. I'll check that run.

Reproduce this review
# save the script below as verify-3659.sh, then:
bash verify-3659.sh blacklanternsecurity/bbot adda64f3ab18b4a059e77d82ad118d55c3c40f75
# expected: SUMMARY 12 PASS, 2 FAIL, 0 SKIP, 0 BROKEN   (V11 until a signature is saved; V14 is release-workflows' 100-commit cap)
# control:  bash verify-3659.sh blacklanternsecurity/bbot beca7d2fadeae998acb23b6c5c375f42acceb316   -> V01 V02 V03 V04 V07 V08 V12 FAIL
#!/usr/bin/env bash
# verify-3659.sh: reproducible checks for blacklanternsecurity/bbot#3659
# Usage: bash verify-3659.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
# Reads the called reusable workflow, org secrets names, rules and CI runs via gh api --method GET.
#
# CONTROL: sha beca7d2fadeae998acb23b6c5c375f42acceb316
# A fix to stable's cla.yml: the base carries the old caller.
# CONTROL-EXPECT: V01=FAIL V02=FAIL V03=FAIL V04=FAIL V07=FAIL V08=FAIL V12=FAIL
# AUTHOR-CODE: none
set -uo pipefail

REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
BASE=beca7d2fadeae998acb23b6c5c375f42acceb316
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass()   { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail()   { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip()   { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }

echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "gh $(gh --version | head -1 | cut -d' ' -f3)"
echo "jq $(jq --version)"

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
GIT=(git -c credential.helper= -c 'credential.helper=!gh auth git-credential')
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi

CALLER=.github/workflows/cla.yml
[ -f "$CALLER" ] || { broken V00 "caller exists" "$CALLER missing"; summary; exit 1; }
RW=blacklanternsecurity/release-workflows
USES=$(awk '/^[[:space:]]*uses:[[:space:]]/{sub(/^[[:space:]]*uses:[[:space:]]*/,""); sub(/[[:space:]]*#.*/,""); print; exit}' "$CALLER")
PIN=${USES##*@}
echo "info caller uses: $USES"

# The called workflow, at whatever ref the caller pins
CALLED="$WORK/called.yml"
CREPO=$(cut -d/ -f1-2 <<<"${USES%@*}"); CPATH=$(cut -d/ -f3- <<<"${USES%@*}")
gh api --method GET "repos/$CREPO/contents/$CPATH" -f ref="$PIN" -H 'Accept: application/vnd.github.raw+json' > "$CALLED" 2>/dev/null || : > "$CALLED"
[ -s "$CALLED" ] || { broken V00 "read called workflow" "could not read $USES"; summary; exit 1; }
echo "info called_sha256 $(shasum -a 256 "$CALLED" | cut -d' ' -f1)"

# V01: the caller pins release-workflows' cla-reusable.yml to the full SHA that tag v1.3.0 names
TAG_SHA=$(gh api --method GET "repos/$RW/git/ref/tags/v1.3.0" --jq '.object.sha' 2>/dev/null || true)
if [ -z "$TAG_SHA" ]; then skip V01 "cla.yml pins release-workflows cla-reusable.yml to the v1.3.0 commit" "could not read tag v1.3.0"
elif [ "${USES%@*}" = "$RW/.github/workflows/cla-reusable.yml" ] && [ "$PIN" = "$TAG_SHA" ]; then
  pass V01 "cla.yml pins release-workflows cla-reusable.yml to the v1.3.0 commit ($TAG_SHA)"
else fail V01 "cla.yml pins release-workflows cla-reusable.yml to the v1.3.0 commit" "uses=$USES tag=$TAG_SHA"; fi

# V02: no secrets: inherit into a pull_request_target job
if grep -Eq '^[[:space:]]*secrets:[[:space:]]*inherit' "$CALLER"; then fail V02 "cla.yml does not pass secrets: inherit" "secrets: inherit present"
else pass V02 "cla.yml does not pass secrets: inherit"; fi

# V03: the caller passes exactly the secrets the called workflow declares under workflow_call
DECL=$(awk '/^  workflow_call:/{w=1;next} w&&/^    secrets:/{s=1;next} s&&/^      [A-Za-z_]+:/{gsub(/[ :]/,"");print;next} s&&/^    [a-z]/{exit} /^[a-z]/&&w{exit}' "$CALLED" | sort | tr '\n' ' ')
PASSED=$(awk '/^    secrets:[[:space:]]*$/{s=1;next} s&&/^      [A-Za-z_]+:/{sub(/^      /,"");sub(/:.*/,"");print;next} s{exit}' "$CALLER" | sort | tr '\n' ' ')
if [ -z "$DECL" ]; then fail V03 "cla.yml passes exactly the secrets the called workflow declares" "called workflow declares none; passed=[$PASSED]"
elif [ "$DECL" = "$PASSED" ]; then pass V03 "cla.yml passes exactly the secrets the called workflow declares ($DECL)"
else fail V03 "cla.yml passes exactly the secrets the called workflow declares" "declared=[$DECL] passed=[$PASSED]"; fi

# V04: the called job declares its permissions, and the caller grants exactly those (least privilege, ci.md)
CALLER_PERMS=$(awk '/^permissions:/{p=1;next} p&&/^  [a-z-]+:/{gsub(/ /,"");print;next} p&&!/^  /{exit}' "$CALLER" | sort | tr '\n' ' ')
JOB_PERMS=$(awk '/^jobs:/{j=1} j&&/^    permissions:/{p=1;next} p&&/^      [a-z-]+:/{gsub(/ /,"");print;next} p&&!/^      /{exit}' "$CALLED" | sort | tr '\n' ' ')
if [ -z "$JOB_PERMS" ]; then fail V04 "caller grants exactly the called job's declared permissions" "called job declares none; caller=[$CALLER_PERMS]"
elif [ "$JOB_PERMS" = "$CALLER_PERMS" ]; then pass V04 "caller grants exactly the called job's declared permissions ($CALLER_PERMS)"
else fail V04 "caller grants exactly the called job's declared permissions" "caller=[$CALLER_PERMS] job=[$JOB_PERMS]"; fi

# V05: the called workflow checks out no code (pull_request_target safety, ci.md)
if grep -Eq 'actions/checkout|git (clone|checkout|fetch)' "$CALLED"; then fail V05 "called workflow checks out no code" "$(grep -Ein 'actions/checkout|git (clone|checkout|fetch)' "$CALLED" | head -2 | tr '\n' ' ')"
else pass V05 "called workflow checks out no code"; fi

# V06: APP_ID and APP_PRIVATE_KEY reach this repository as org secrets
NAMES=$(gh api --method GET "repos/$REPO/actions/organization-secrets" --jq '[.secrets[].name]|sort|join(" ")' 2>/dev/null || true)
if [ -z "$NAMES" ]; then skip V06 "APP_ID and APP_PRIVATE_KEY reach this repository" "org secrets list unreadable with this token"
elif grep -qw APP_ID <<<"$NAMES" && grep -qw APP_PRIVATE_KEY <<<"$NAMES"; then pass V06 "APP_ID and APP_PRIVATE_KEY reach this repository as org secrets"
else fail V06 "APP_ID and APP_PRIVATE_KEY reach this repository" "org secrets: $NAMES"; fi

# V07: this cla.yml is byte-identical to dev's (so the next dev->stable release has nothing to merge in it)
"${GIT[@]}" fetch --quiet origin "+refs/heads/dev:refs/remotes/origin/dev" || true
DEV_BLOB=$(git rev-parse --verify --quiet "origin/dev:$CALLER" || true); HEAD_BLOB=$(git hash-object "$CALLER")
if [ -z "$DEV_BLOB" ]; then skip V07 "cla.yml equals dev's cla.yml" "could not read dev"
elif [ "$DEV_BLOB" = "$HEAD_BLOB" ]; then pass V07 "cla.yml equals dev's cla.yml (blob ${HEAD_BLOB:0:8})"
else fail V07 "cla.yml equals dev's cla.yml" "here ${HEAD_BLOB:0:8}, dev ${DEV_BLOB:0:8}"; fi

# V08: against the base, the change touches only .github/workflows/cla.yml
"${GIT[@]}" cat-file -e "$BASE^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "$BASE" || true
CHANGED=$(git diff --name-only "$BASE" HEAD 2>/dev/null | tr '\n' ' ')
if [ "$CHANGED" = "$CALLER " ]; then pass V08 "the change touches only $CALLER"
else fail V08 "the change touches only $CALLER" "changed=[$CHANGED]"; fi

# V09: no required status check on stable or dev names the old job check, so the job rename blocks nothing
REQ=""
for BR in stable dev; do
  R=$(gh api --method GET "repos/$REPO/rules/branches/$BR" --jq '[.[] | select(.type=="required_status_checks") | .parameters.required_status_checks[].context] | join("|")' 2>/dev/null) || { REQ="ERR"; break; }
  REQ="$REQ $BR:[$R]"
done
if [ "$REQ" = "ERR" ]; then skip V09 "no required check names 'check the CLA is signed / CLAAssistant'" "rules unreadable"
elif grep -q 'check the CLA is signed' <<<"$REQ"; then fail V09 "no required check names 'check the CLA is signed / CLAAssistant'" "$REQ"
else pass V09 "no required check names 'check the CLA is signed / CLAAssistant' ($REQ )"; fi

# V10: a run of release-workflows cla-reusable.yml@v1.3.0 in this org minted the App token with these secrets
# (MANSPIDER run 43966084823, pull_request_target, the only v1.3.0 run found that reached the token step)
MRUN=43966084823
MREF=$(gh api --method GET "repos/blacklanternsecurity/MANSPIDER/actions/runs/$MRUN" --jq '[.referenced_workflows[]?.path]|join(",")' 2>/dev/null || true)
MTOK=$(gh api --method GET "repos/blacklanternsecurity/MANSPIDER/actions/runs/$MRUN/jobs" --jq '[.jobs[].steps[] | select(.name=="Generate token from GitHub App") | .conclusion] | join(",")' 2>/dev/null || true)
if [ -z "$MREF" ] || [ -z "$MTOK" ]; then skip V10 "an org run of cla-reusable.yml@v1.3.0 minted the App token" "could not read MANSPIDER run $MRUN"
elif grep -q "cla-reusable.yml@$TAG_SHA" <<<"$MREF" && [ "$MTOK" = "success" ]; then pass V10 "an org run of cla-reusable.yml@v1.3.0 minted the App token (MANSPIDER run $MRUN)"
else fail V10 "an org run of cla-reusable.yml@v1.3.0 minted the App token" "ref=$MREF token_step=$MTOK"; fi

# V11: the App has saved a signature to CLA's signatures branch (a commit after the seed f8010baf)
NSIG=$(gh api --method GET "repos/blacklanternsecurity/CLA/commits?sha=signatures&per_page=100" --jq '[.[] | select(.sha | startswith("f8010baf") | not)] | length' 2>/dev/null || true)
if [ -z "$NSIG" ]; then skip V11 "the App has saved a signature to CLA's signatures branch" "could not read CLA signatures"
elif [ "$NSIG" -gt 0 ]; then pass V11 "the App has saved a signature to CLA's signatures branch ($NSIG commits after the seed)"
else fail V11 "the App has saved a signature to CLA's signatures branch" "only the seed commit f8010baf; no run has saved one yet"; fi

# V12: the called workflow saves signatures on CLA's signatures branch, not on main (main refuses writes)
BRANCH=$(awk '/^[[:space:]]*branch:[[:space:]]/{gsub(/["[:space:]]/,""); sub(/^branch:/,""); print; exit}' "$CALLED")
if [ "$BRANCH" = "signatures" ] && grep -q 'remote-repository-name: "CLA"' "$CALLED"; then pass V12 "called workflow saves signatures on CLA's signatures branch"
else fail V12 "called workflow saves signatures on CLA's signatures branch" "branch=${BRANCH:-unset}"; fi

# V13 (challenge C11): classic branch protection on stable and dev does not require the old job check either
CREQ=""
for BR in stable dev; do
  R=$(gh api --method GET "repos/$REPO/branches/$BR" --jq '[.protection.required_status_checks.contexts[]?] | join("|")' 2>/dev/null) || { CREQ="ERR"; break; }
  CREQ="$CREQ $BR:[$R]"
done
if [ "$CREQ" = "ERR" ]; then skip V13 "classic protection requires no 'check the CLA is signed' check" "branches unreadable"
elif grep -q 'check the CLA is signed' <<<"$CREQ"; then fail V13 "classic protection requires no 'check the CLA is signed' check" "$CREQ"
else pass V13 "classic protection requires no 'check the CLA is signed' check"; fi

# V14 (challenge C12): the called workflow can pass a pull request with more than 100 commits, as bbot's
# dev->stable release merges have (346 at beca7d2f, 121 at eabac72a, 1407 at 5355bd1f)
"${GIT[@]}" fetch --quiet origin eabac72a4 5355bd1f9 2>/dev/null || true
RC=$(for c in "$BASE" eabac72a4 5355bd1f9; do git rev-list --count "$c^1..$c^2" 2>/dev/null; done | tr '\n' ' ')
if grep -Eq '"\$N_COMMITS" -gt 100' "$CALLED"; then fail V14 "called workflow passes pull requests with more than 100 commits" "exit 1 when commits > 100, before exemption; recent release merges: $RC"
else pass V14 "called workflow passes pull requests with more than 100 commits"; fi

summary

@liquidsec liquidsec left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving: this restores CLA signing on bbot. Outside contributors can't sign today: #3658 (into dev) still ran stable's old CLA/cla-reusable.yml@4f85d3c (run 44284637473) and failed. GitHub runs this workflow from the default branch, so dev's v1.3.0 copy has never run in bbot.

Please hold off on making cla / CLAAssistant required on stable or dev until release-workflows fixes the first two points below. The next release can't merge without a bypass until then.

  1. More than 100 commits fails the check. v1.3.0 exits on N_COMMITS > 100, and that runs before the org-member exemption, so a dev→stable release opened by a maintainer fails too. The last two releases had 346 commits (#3584) and 121 (#3271). Paginating the commits (--paginate) instead of capping them fixes it.
  2. Signing or recheck can't clear a failed check. After saving the signature, the action re-runs the PR's failed run, which needs actions: write. Neither this caller nor the reusable grants it, so the call returns 403 and is only logged. The contributor stays red until they push again. This was already the case with the old workflow, but it only matters once the check is required.
  3. The check name changes from check the CLA is signed / CLAAssistant to cla / CLAAssistant. Open PRs only report the old name, so each one needs a new push before a required check can pass.
  4. A commit whose author email isn't linked to a GitHub account fails the whole check. One such commit in a release's hundreds blocks it.

Also: when the opener isn't an org member, v1.3.0 requires every commit author to have signed, members included. That's deliberate, since an author email can be spoofed. But #3638 includes a maintainer's merge commit from #3584, and no maintainer is in cla.json, so that PR will ask a maintainer to sign. Maintainers should sign the ICLA once.

@pjhartlieb
pjhartlieb merged commit 0ade542 into stable Nov 18, 2026
20 checks passed
@pjhartlieb
pjhartlieb deleted the ci/cla-v1.3.0-stable branch November 18, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants