fix(engine): motion blur works over video by holding each video frame across samples

[miguel-heygen]

Fixes #5144.

What changes

Motion blur now works in compositions that contain a <video>. Each output frame's video frame is held for every sub-frame sample of that frame, so the footage stays as shot (it already carries its own camera blur) and the graphics moving over it are blurred. No new option: this is simply what motion blur does over video now, and the refusal is gone.

How

• The before-capture hook (BeforeCaptureHook) takes an optional third argument, heldVideoTime. Motion-blur samples and the adaptive probes pass the output frame's time; a frame without blur passes nothing, so that path is unchanged.
• The video frame injector (createVideoFrameInjector) decides which videos show at the sample's own time, like every other layer. For a video that is also on screen at heldVideoTime, it uses that time's frame. The injector already skips re-injecting an unchanged frame, so samples reuse the injected frame: no re-extraction, no extra decode.
• Each blurred frame starts with one injection at the frame's own time, right after its seek, before any sample. The injector copies a video's own style (its transform, for example) only when its frame changes, so this keeps the held frame's style at the frame's time; without it, a video moving by its own transform sat a quarter frame behind.
• The GPU re-render the injector triggers (for WebGL/WebGPU layers that sample a video) runs at the time it is given, which is always the page's own time.
• resolveSessionMotionBlur no longer throws when a session has a before-capture hook.

Scope notes

• Held means the whole video layer: its pixels, and any style animated directly on the <video> element, taken at the frame's time. Animation on a wrapper follows the samples like any other layer.
• A cut inside a frame's shutter window cross-fades, video included: samples before the cut show the outgoing scene with its video's last frame, samples after show the incoming scene's held frame. This is the same blend the non-video layers of a cut already get with blur on.

Tests

• frameCapture-motionBlur.test.ts: every sample and adaptive probe calls the hook at its own sub-frame time with the frame time held; a frame without blur holds nothing. The old "rejects injected video frames" case now asserts the session is accepted.
• videoFrameInjector.test.ts: a video on screen at both times shows the held frame (frame 15, where the sample alone would pick 14); across a cut, a sample just before it shows the outgoing scene's video.
• motionBlurVideo.integration.test.ts (producer, integration lane, real Chrome + ffmpeg):
  • a 160x90 testsrc2 video filling the frame and drifting left by its own CSS transform, with a white bar sliding across its bottom third, rendered without blur, with blur and with blur again. Every frame's video rows match the unblurred render, the bar's rows differ, and the two blurred renders are byte-identical.
  • the repo's two-scene fixture nested-sequential-video-local-start (two sub-composition hosts, each a full-frame video, cut at 1 s, 24 fps) with a generated clip: every non-cut frame matches the unblurred render, and the cut frame's brightness stays within 10% of it (and the unblurred cut frame is checked to have real video in it).
• Not covered by a real render: a WebGL/WebGPU layer sampling a video, and the rule that a held frame applies only to a video also on screen at the sample's time (that one is covered by the injector unit test).

Verified on a Linux box with real Chrome (headless shell, software GL):

Check	Result
frameCapture-motionBlur.test.ts, frameCapture-gpuCompletion.test.ts, videoFrameInjector.test.ts, 3 runs each	38, 2 and 18 passed every run
motionBlurVideo.integration.test.ts, 3 runs	5 passed each
integration test against the first version of this PR (frame held for visibility too)	the cut case fails: the cut frame's brightness is off by 54.9 against a limit of 11 (it went about half dark)
integration test against main's engine	fails in setup with the old "cannot run with injected video frames" error
injector ignores the held time (mutation)	the video-rows and cut cases fail at frame 1, and the injector's held-frame unit case fails
no injection at the frame's time before the samples (mutation)	the video-rows case fails at frame 1 (the drifting video lags), and 2 unit cases fail
oxfmt, oxlint, tsc --noEmit in engine and producer, test reachability, comment ratchet	clean

Before / After

A lower third sliding in over generated test footage (testsrc2, 960x540, 30 fps), frame 6, rendered to PNG. On main, the same render with motionBlur fails at once with "[MotionBlur] sub-frame motion blur cannot run with injected video frames", so the only thing you could ship was the unblurred frame.

Before

Main, blur refused, so the lower third ships unblurred (2x crop, then the full frame):

After

This branch, motionBlur: { samplesPerFrame: 32 }. The lower third smears along its motion; the footage, including the grey bar next to it, stays exactly as shot:

[github-actions]

Edit accuracy: accurate 2059 (base branch 2059), smooth 1612 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

• crop-none-px-r0-nested-z50: tracking 0.05, pressJump 0, drop 40.07, reload 40.12, render 40.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0.1, reload 0.12, render 0.03, renderKey -, undo true, teleport true / tracking 0.05, pressJump 0, drop 0.1, reload 0.12, render 0.03, renderKey -, undo true, teleport true

[jrusso1020]

Approve. The fix is small and sits at the right layer. It reuses the injector's existing "skip an unchanged frame" path (lastInjectedFrameByVideo, videoFrameInjector.ts:224), so holding a frame costs no extra decode or injection. It adds no new option or mode.

Reuse and simplicity

• The optional heldVideoTime on BeforeCaptureHook is the smallest change that works. Only the injector reads it. The HDR loops (captureHdrSequentialLoop, captureHdrHybridLoop, captureHdrResources) call the hook with two arguments, so their behavior is unchanged.
• Removing the engine refusal is safe. Every blurred frame goes through captureAccumulatedFrame (frameCapture.ts:4247-4251). The one route that can't accumulate, hdr_layered, is still refused in motionBlurRoute.ts. Distributed renders can't request blur.
• Merging held payloads into the active map is safe because getActiveFramePayloads returns a fresh Map on every call (videoFrameExtractor.ts:2547).
• The pre-sample injection at frameTime (frameCapture.ts:4150-4156) is the right fix for the transform lag the body describes. Its time is added to beforeCaptureMs, so the perf totals stay honest.

Non-blocking notes

1. refreshActiveSet keeps a forward cursor and does a full rescan whenever time goes backward (videoFrameExtractor.ts:2508). The hook now looks up the sample time and then the held time, so every sample after the frame time triggers a rescan. That's correct, and the cost is O(videos) per sample, which is small next to a screenshot. If it ever shows up in a profile, the held lookup can be done once per frame instead of once per sample.
2. Wording only: before a cut, the outgoing video isn't on screen at the held time, so its samples follow the sample time. They show the frame at each sample time, not the video's last frame as the "Scope notes" say. The cut-frame blend still behaves as described, and the injector test 0.98 -> /a/9 pins exactly this.

Verified against the source: the injector already skips unchanged frames; resolveSessionMotionBlur no longer throws on a hook; the probes and samples all pass frameTime. I didn't run the integration test locally.

Verdict: APPROVE
Reasoning: A minimal and correct change that reuses the injector's existing dedup. The sites that call the hook without blur are unchanged, and the routes that can't blur are still refused.

— Rames