Skip to content

fix(core): stop script loading time from shifting WAAPI frames - #5017

Open
cooleryu wants to merge 1 commit into
heygen-com:mainfrom
cooleryu:fix/4559-frame-stable
Open

cooleryu wants to merge 1 commit into
heygen-com:mainfrom
cooleryu:fix/4559-frame-stable

Conversation

@cooleryu

@cooleryu cooleryu commented Oct 4, 2026 •

Copy link
Copy Markdown

What

Keep initial WAAPI frames on composition time, rather than on how long the page took to load. Feedback on the initialization contract and compatibility decision below is especially welcome.

Why

The first baseline currently retains a running animation's currentTime. If the animation starts before the runtime and more scripts load afterward, that elapsed wall-clock time becomes a permanent seek offset. Repeated exports can therefore put the same object at different positions in the same frame.

I reproduced this through the producer's HTML compilation, capture, MP4 encoding, and decoded-frame path, not just the adapter. A 4-second, 100 px translation with an initial left edge of 40 px should land at 90 px at 2 seconds:

Local scripts loaded after creation Before: three exports After: three exports
0 90, 90, 90 px 90, 90, 90 px
8 92, 92, 92 px 90, 90, 90 px
32 98, 92, 92 px 90, 90, 90 px

The scripts are tiny local files, with no artificial sleep. Paused-offset controls retain their authored position. These are macOS/Chrome measurements, not a cross-platform performance claim.

Related work

Refs #4559. Proposed contract and reproduction: #4559 (comment)

How

At initial discovery only, anchor running or already-finished JavaScript animations on document.timeline at normal playback rate to composition zero. Including finished animations lets short, filled effects rewind after a slow load. Keep existing baselines for paused animations, CSS animations/transitions, other timelines, non-default playback rates, late creation, and rediscovery.

Compatibility decision for review: a pre-runtime currentTime assignment left running is indistinguishable from elapsed loading time. This proposal does not retain it as a baseline offset. The authoring guide now documents positive/negative delay, or pause() followed by currentTime, as explicit offset mechanisms. There is a browser test for this behavior; it is not an accidental side effect. No new API or timing heuristic is introduced.

Test plan

  • Added 21 adapter cases: initial progress, direct seek/pause, finished/idle states, paused offsets, exclusions, repeated discovery, late creation, and revert.
  • Added 9 real-Chrome cases using the built runtime and both public seek methods. Seven fail against the original artifact; all nine pass with the patch. Existing browser tests also pass: 17/17 total.
  • Full test:hyperframe-runtime-ci passes, including 1,849 runtime tests, type checking, contract/behavior/seek/duration/parity/security checks, coverage gates, and linter.
  • Separate local integration experiment: 36 browser contexts / 288 seek samples, plus 18 final MP4 exports / 720 decoded frames, all within 1 px of expected positions. Across 0/8/32 scripts and three repetitions, each paused/running group has identical decoded RGB output. This experiment is supplementary; the committed browser regressions are CI-runnable.
  • Changed-file lint/format and diff whitespace checks pass; authoring documentation updated.
  • Comments describe test intent and runtime boundaries, not change history.
bun run --cwd packages/core test:hyperframe-runtime-ci
cd packages/producer
bunx vitest run src/services/coreRuntimeBrowser.test.ts

The browser suite needs Chrome; locally I used PUPPETEER_EXECUTABLE_PATH for the installed browser. I have not validated Linux BeginFrame/GPU capture, distributed workers, or the Studio UI. This is not a general fix for non-default playback rates or custom timelines.

@cooleryu
cooleryu marked this pull request as ready for review October 4, 2026 13:55

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant