Skip to content

fix(aws-lambda): a network blip no longer stops the release ffmpeg build - #6569

Merged
miguel-heygen merged 1 commit into
mainfrom
fix/ffmpeg-build-retry
Oct 27, 2026
Merged

miguel-heygen merged 1 commit into
mainfrom
fix/ffmpeg-build-retry

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

The publish workflow's arm64 ffmpeg build failed once on curl: (35) Recv failure: Connection reset by peer, which skipped Publish for v0.8.209 until the job was re-run. The build's downloads now live in one script, packages/aws-lambda/image/fetch.sh:

  • fetch (source tarballs) uses curl --retry 5 --retry-all-errors --retry-delay 3 --connect-timeout 20 --max-time 600. The old --retry 3 did not retry a reset connection. The sha256 check against sources.txt is unchanged.
  • retry wraps the x264 git clone and both microdnf install calls in the Dockerfile, which had no retry.
    Other downloads in the release and publish workflows: the curl calls in lambda-arm64-image.yml already retry 30 times, and install-ffmpeg-linux already uses --retry-all-errors with a deliberate one-minute cap; the rest are gh release and artifact actions.

What I measured

  • fetch.test.ts serves a tarball from a server that resets the first two connections, and fetch still ends with the file extracted and checked. fails without the fix: with the old flags curl exits 56 and is not retried (run on the devbox). Green with the fix, 3 runs in a row on the Mac with a sha256sum shim for the missing GNU tool.
  • microdnf --setopt=retries is rejected by the base image (Unable to handle), so retry wraps the command instead. Ran . /fetch.sh && retry microdnf install ... and retry false (exit 1) in the pinned Lambda base image.
  • This PR touches the image directory, so the Lambda arm64 image workflow builds the whole image.

What I did NOT exercise

A full publish run. Why every release rebuilds ffmpeg: lambda-arm64-image.yml seeds the cache only on a push to main that touches packages/aws-lambda/image/**. That push trigger arrived in #6070, after the last image change (#6064), so the key has never been saved (no push run exists for the workflow, gh cache list has no hf-media entry). This PR touches that directory, so its merge push saves the key that the next release restores. Not verified until it merges; I will check gh cache list then.

Why under 100 lines

One lane-owned release-pipeline fix with nothing else to carry it. Its merge also seeds the ffmpeg cache that releases restore.

No visible change

Build and CI only.

@heygengenesis heygengenesis Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: APPROVE · 0 blocker · 0 important · 0 nits · head 5052aec5

Scope

I read all four files. I ran fetch.test.ts locally: it passes with this change and fails with the old --retry 3 flags, so the reset-connection repro is real. retry sits in an && list, so set -e does not cut a retry short. A failed git clone cleans up its own directory, so the retried clone starts clean. On this head the Lambda arm64 image workflow ran a full uncached ffmpeg build twice, cmp shows the two builds match, and the image smoke test passed. CI and Windows render verification are green.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 27, 2026
Merged via the queue into main with commit 8cd9c51 Oct 27, 2026
208 of 209 checks passed
@miguel-heygen
miguel-heygen deleted the fix/ffmpeg-build-retry branch October 27, 2026 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant