[miguel-heygen]

Merged with main after #5772

#5772 added a loop that sets download targets for files chunks read by URL; it still passed the old chunksCarryAudio flag this PR replaced with the chunkAudio mode. The merge passes chunkAudio there (planV2.ts); git flagged no conflict, the typecheck did. The size budget needs no change now: main's producer budget (768,210 B packed) covers this branch, which packs to 733,045 B.

What changes for users

A distributed render whose mp4 chunks carry their own audio no longer makes every chunk download the whole mixed audio track. Each chunk downloads the track's index (its moov) and only the AAC packet bytes it encodes. On a 30-minute render that is about 0.5 MB per chunk instead of about 43 MB. The output is unchanged: every chunk carries the same audio packets as before, and the finished video's audio is the same.

How

• A v2 plan whose chunks carry audio also writes audio-chunks/:
  • frame.m4a: the audio file without its mdat payload (ftyp, moov and the mdat header). It goes to every chunk.
  • layout.json: the file's size, where the payload sits, and where each chunk's packets start. It goes to every chunk.
  • NNNN.bin: chunk N's packet bytes, cut with the same audioCutForChunk the chunk uses. It goes to chunk N only.
• A chunk rebuilds a sparse copy of the whole file at the original offsets (bytes it does not own are holes) and runs the unchanged mux and cut on it. ffmpeg sees the same moov and the same packet bytes at the same offsets, so the chunk writes the same packets on every ffmpeg version.
• The whole audio.m4a still ships in the plan, so HasAudio is unchanged, but no role downloads it.
• A chunk whose plan has no audio-chunks/ (a v1 plan, or one written before this change) reads the whole track as before.
• v1 plans are unchanged. Their chunks download the whole plan tarball, so slices there would only grow it.

Why not cut each slice into its own m4a

That was the first version of this PR. Decoded audio stayed identical, but on ffmpeg 6.0 (the Lambda's ffmpeg-static) and 7.0.2, re-reading a stream-copied slice changes its last packet's duration (the slice's own edit list trims it). The stitch sums sample durations, so a 30-minute render's audio track came out about 2.2 s short of the video. ffmpeg 8.x does not show it, which is why it passed CI. The sparse rebuild does not depend on how any ffmpeg version writes or reads edit lists.

Tests

• chunkAudio.test.ts: for three layouts (three uneven chunks, one chunk, and a 60 fps one-frame tail that owns no packets), a chunk muxed from a sliced plan has the same packets per stream (timestamps, durations, sizes and payload sha256) as one muxed from the whole file. Durations are 91 frames at 30 fps and 51 frames at 60 fps, which are not whole 1024-sample frames. Passes with ffmpeg 6.0, 7.0.2 and 8.1.1. The first version of this PR fails it on 6.0.
• planV2.test.ts: a sliced plan sends each chunk its own packet file plus the shared frame and layout, and nothing gets audio.m4a. A real planV2WithPublisher run (2.1 s tone, three chunks) writes and targets them.
• Each new assertion was broken once and went red: writing a chunk's bytes one byte late, a frame without its moov, sending packet files to every chunk, and turning the plan option off.

Measured

Local v2 distributed render on a Linux dev box (8 cores, shared, ffmpeg 7.0.2): a 30-minute composition (1800.5 s at 30 fps, 54,015 frames, so not a whole number of AAC frames), 226 chunks of 240 frames. Each run goes through the producer's own plan, per-chunk materialization, renderChunk and assemble, with the same fixture, once on main and once on this branch. Chunks read from local disk, so the download time does not show here; the bytes are what a Lambda chunk fetches from S3.

	main	this PR
plan audio bytes each chunk downloads (median)	43,209,643	534,190 (339 KB moov frame + 3 KB layout + 192 KB packets)
all plan files each chunk downloads (median)	73.1 MB	30.5 MB
plan audio downloaded across the render	9,765 MB	120 MB
new plan step that writes the slices	none	0.10 s
chunk materialize (median)	134 ms	58 ms
chunk encode + mux stage (median)	665 ms	359 ms
chunk wall time (median, shared box)	5.94 s	6.05 s
audio packets in all 226 chunks (timestamps, durations, sizes, payload sha256)		identical: 84,400 of 84,400
finished video's audio packets		identical: 84,400 of 84,400
decoded audio, 48 kHz, pan=mono|c0=FL and stereo f32 hashes		identical (86,424,576 samples per channel)
audio and video stream durations of the finished video	1800.500 s / 1800.500 s	1800.500 s / 1800.500 s

• At the roughly 83 MB/s a render Lambda reads from S3, 43 MB is about 0.5 s of each chunk's start; 0.53 MB is a few milliseconds plus two more small GETs.
• Chunk wall time on the shared box is within its run-to-run noise; the box reads local files, so it cannot show the download saving.
• The remaining ~30 MB each chunk downloads is mostly a project <audio> source (tone.m4a, 29.6 MB) that chunks never play: capture does not wait on <audio> and the plan mixes the audio. That is a separate change to how compiled assets are targeted.
• 90 of 226 chunks have a few different video packets between the two runs. This PR does not touch capture or the video encode; two runs of main on this loaded box have differed the same way before. Not re-proven with a second main run here.
• Not measured on Lambda: no AWS session was available for this run.

[github-actions]

Edit accuracy: accurate 271 (base branch 271), smooth 166 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

[somanshreddy]

Reviewed at 8ff43a6d. No blocker, so I'm approving. There is one should-fix for a deploy that happens mid-render.

What I checked

• Slice writer: the packet offsets are walked from stco/stsc/stsz, and each NNNN.bin is cut with the same audioCutForChunk that mux uses. frame.m4a is the file minus [payloadStart, payloadEnd). chunkAudioSource writes the head, the tail at payloadEnd and the chunk's part at its original offset, then ftruncates to sizeBytes. So the moov offsets stay valid, and any packet the chunk doesn't own reads as zeros. The cut never touches those packets, and the payload-sha256 test proves that.
• Targeting: frame.m4a and layout.json go to every chunk, NNNN.bin goes only to chunk N, and audio.m4a now goes to no chunk and no assembler when the plan is sliced. A v1 plan, or a v2 plan with no audio-chunks/, falls back to the whole file.
• Tests: I ran bun test src/services/distributed/ with ffmpeg 8.0.1: 268 pass and 1 fail. The failure is plan.masterBus.test.ts, a vitest-only file (vi.hoisted) that this PR doesn't touch. chunkAudio.test.ts and planV2.test.ts pass 71 of 71.
• Mutation checks:
  • Writing the chunk's part at start + 1 fails 3 of 4 tests in chunkAudio.test.ts.
  • Sending every NNNN.bin to all chunks fails 2 tests in planV2.test.ts.
• API surface: the plan format is internal. No public SDK or CLI export changed.

1. Old chunk worker with a new plan: silent audio gaps (should-fix)
Nothing stops a worker from before this PR from running a chunk of a sliced plan. renderChunk only gates on the ffmpeg version, and the Lambda target is unqualified ($LATEST). Cloud Run can also split traffic across revisions. Such a worker calls resolvePlanAudioPath(planDir), gets null because audio.m4a is no longer materialised for chunks, and muxes a video-only chunk. Its probe then only expects video, so it passes.

• planTracksOf accepts a chunk with no soun track as "owns no audio packets", so the stitched video gets silent stretches with no error.
• It only fails loudly if chunk 0 itself is the old one.

This needs a deploy or rollback while a render is running, so it's rare, and I didn't reproduce it. I inferred it from the code. Two cheap fixes:

• Bump something an old chunk refuses: a plan v2 manifest or protocol version, or a required capability.
• Or keep shipping audio.m4a to chunks for one release, so the fallback still works.

Nits

• rmSync(audioScratchPath) only runs on the success path. On a mux failure the sparse scratch file stays next to outputChunkPath. It's small on disk but has the full apparent size. Moving it into the finally would fix that.
• track! is used in three places in writeChunkAudioSlices. An audio file with no track would throw a bare TypeError. A one-line if (!track) throw new Error("[chunkAudio] plan audio has no track") would give a clearer error.

I didn't measure this on Lambda or Cloud Run. CI was 13 pass, 47 pending and 0 failing when I approved.

— Somu

[somanshreddy]

Re-approving at aeebedfc. It adds one commit on top of 8ff43a6d, which I approved in review 5574472941. The only change is in scripts/package-size-budgets.json: the packed budget for @hyperframes/producer goes from 729,380 to 767,334 bytes (+5.2%). The unpacked budget is unchanged. No source changed, so my earlier findings stand, including the should-fix for an old chunk worker running a new plan. CI was 21 pass, 28 pending and 0 failing.

— Somu

[somanshreddy]

Re-approving at 91fc255c. It's a merge of main (through #5903) into aeebedfc, which I approved in review 5574981230. git range-diff shows all four of the PR's commits as =.

The merge resolution (git show --remerge-diff) has two hunks:

• planV2.ts: #5772's new remote-artifact loop now passes chunkAudio instead of chunksCarryAudio, to match this PR's new artifactTargets signature. The remote paths are project assets from shared storage, not plan audio or audio-chunks/. chunkAudio is still derived from the local files, where the slices live.
• scripts/package-size-budgets.json: takes main's @hyperframes/producer budget (768,210 packed / 2,807,938 unpacked).

Tests: bun test src/services/distributed/ passes 282 and fails 1. The failure is the same vitest-only plan.masterBus.test.ts (vi.hoisted) as before, unrelated to this PR. CI was 30 pass, 25 pending and 0 failing.

My earlier should-fix still applies: an old chunk worker given a new sliced plan finds no audio.m4a and leaves silent gaps.

— Somu