Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
608dc22
feat(core): support deterministic async frame sources
jrusso1020 Oct 6, 2026
30f6c08
feat(core): bridge preserved film HTML to native capture
jrusso1020 Oct 6, 2026
789b2ee
feat(core): bound imported scene timing to source intervals
jrusso1020 Oct 6, 2026
7503e04
test(producer): verify film bridge pixels in real browser
jrusso1020 Oct 6, 2026
37b9f4c
fix(runtime): retain queued draws and disable unsafe dedup
jrusso1020 Oct 6, 2026
6985b8a
Merge branch 'codex/external-renderer-bridge' into codex/film-runtime…
jrusso1020 Oct 6, 2026
d3e7bcb
fix(runtime): fail promptly on runner reloads and send errors
jrusso1020 Oct 6, 2026
04ab9a1
Merge branch 'codex/film-runtime-bridge' into codex/film-scene-timing
jrusso1020 Oct 6, 2026
b29b84a
fix(runtime): align film draws with export windows
jrusso1020 Oct 6, 2026
31c2d2c
docs(core): mark frame-source hosts with data-no-timeline
jrusso1020 Oct 7, 2026
4d01382
Merge branch 'codex/external-renderer-bridge' into codex/film-runtime…
jrusso1020 Oct 7, 2026
42414f2
docs(core): require data-no-timeline on imported film hosts
jrusso1020 Oct 7, 2026
40e53e2
Merge branch 'codex/film-runtime-bridge' into codex/film-scene-timing
jrusso1020 Oct 7, 2026
29a5c02
fix(runtime): avoid transport redraws after render seeks
jrusso1020 Oct 7, 2026
c4b8763
Merge branch 'codex/external-renderer-bridge' into codex/film-runtime…
jrusso1020 Oct 7, 2026
c73af06
Merge branch 'codex/film-runtime-bridge' into codex/film-scene-timing
jrusso1020 Oct 7, 2026
3a7a350
chore: merge main after film bridge support landed
jrusso1020 Oct 7, 2026
1a13421
test: set explicit film fixture canvas background
jrusso1020 Oct 7, 2026
3b471b8
test: diagnose film capture timeout phase
jrusso1020 Oct 7, 2026
7a3d335
test: foreground film pages before screenshot capture
jrusso1020 Oct 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/guides/frame-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ const unregister = window.__hyperframes.registerFrameSource({

The host uses the usual composition attributes: `data-composition-id`, `data-start`, `data-duration`, and `data-track-index`. Declare the root's dimensions, FPS, and duration. A frame source does not need a dummy GSAP timeline. Add `data-no-timeline` to the host, and to a root that registers no timeline; otherwise `hyperframes lint` reports `missing_timeline_registry` and each render waits 45 seconds for timeline registration.

The callback receives seconds in source time: playback inpoint (`data-playback-start`) plus the rate-adjusted time since the host's resolved start. Existing playback rate and speed-ramp semantics apply. Timing is read again on every seek, so move, trim and rate edits do not require rewriting the animation code. Each overlapping scene needs its own source instance.
The callback receives seconds in source time: playback inpoint (`data-playback-start`) plus the rate-adjusted time since the host's resolved start. Optional `sourceRange: { start, duration, fps }` adds an original source offset and clamps this time to the last source frame within that interval. Existing playback rate and speed-ramp semantics apply. Timing is read again on every seek, so move, trim and rate edits do not require rewriting the animation code. Each overlapping scene needs its own source instance.

`ready` holds initialization and export capture. Return a promise from `render` when drawing is asynchronous: capture waits for it before taking pixels. Repeated and out-of-order times must produce the same frame. HyperFrames owns playback; do not start a second clock.

Expand Down
24 changes: 23 additions & 1 deletion docs/guides/imported-film-html.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,30 @@ const unregister = window.__hyperframes.registerFrameSource({

The bridge installs its listener before setting `srcdoc`. It accepts messages only from that iframe's window with opaque origin `"null"`. It sends `load` after `hello`, waits for `ready`, and sends explicit `frame` requests with increasing sequence IDs. Capture waits for the matching frame acknowledgement; stale replies do not release it. The wrapper never calls the original player's autoplay path.

Startup has a 20-second deadline and a frame has a 15-second deadline. Startup, runner and timeout failures reject capture. An individual `frame-error` can recover on the next seek. Unregistering rejects outstanding bridge work and removes its listener. Use a separate bridge and runner iframe for each independently timed or overlapping scene.
Startup has a 20-second deadline and a frame has a 15-second deadline. Startup, runner and timeout failures reject capture. An individual `frame-error` can recover on the next seek. Unregistering rejects outstanding bridge work, removes its listener and clears the runner document. Use a separate bridge and runner iframe for each independently timed or overlapping scene.

Use screenshot capture for iframe pixels. This bridge does not read the sandbox DOM, expose internal scene nodes as Studio layers, or provide audio mixing and video extraction. Font/asset readiness depends on the preserved runner's `ready` and `frame` guarantees and must be checked for each supported export version.

The compatibility target is the current protocol, not every arbitrary animated web page. Visual comparison against the partner preview remains an integration acceptance step after MCP packaging.

## Editable scene timing

To expose individual scenes on the HyperFrames timeline, mount one timed host and independent runner iframe per scene. Each runner can load the preserved full film. Register the original scene's immutable source interval separately from its editable host timing:

```javascript
window.__hyperframes.registerFrameSource({
element: sceneHost,
ready: bridge.ready,
render: bridge.render,
dispose: bridge.dispose,
sourceRange: { start: 3, duration: 4, fps: 60 },
});
```

For a scene originally covering `[3, 7)`, source time is `3 + data-playback-start + rateAdjustedLocalTime`. The source interval clamps that result between 3 and the last source frame before 7. Moving the host changes `data-start`; trimming changes its duration and source inpoint; stretching uses the existing playback-rate attributes. Extending beyond available source footage holds the last frame. Changing duration alone does not automatically stretch the animation.

Adjacent clips use half-open timeline windows. At the final composition instant, the final scene holds its last frame. Different source instances allow reordered or overlapping scenes to request different original film times without sharing mutable runner state.

Persist the preserved runner/load payload and `sourceRange` in the wrapper's initialization data. Host timing stays in ordinary HyperFrames attributes, so serialization, save/reopen, undo and redo preserve it without rewriting scene modules. Do not use the original player's `recut` field as a replacement for host timing: that protocol does not represent arbitrary trim, reorder and overlap edits.

This provides scene-level timing and placement. Text, colors and individual animation changes still require editing the preserved scene modules. The runtime does not expose iframe internals as manually editable Studio objects or keyframes.
1 change: 1 addition & 0 deletions packages/core/src/runtime/filmBridge.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,7 @@ describe("film runner bridge", () => {
first.bridge.dispose();
first.initialize();
await expect(first.bridge.ready).rejects.toThrow("disposed");
expect(first.iframe.srcdoc).toBe("");
expect(first.send).not.toHaveBeenCalled();
const second = setup();
second.initialize();
Expand Down
1 change: 1 addition & 0 deletions packages/core/src/runtime/filmBridge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ export function createFilmBridge(options: {
disposed = true;
window.removeEventListener("message", receive);
fail(new Error("Film bridge was disposed"));
iframe.srcdoc = "";
},
};
}
109 changes: 108 additions & 1 deletion packages/core/src/runtime/frameSources.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,13 @@ function deferred() {
}

const adapters: ReturnType<typeof createFrameSourceAdapter>[] = [];
const adapter = () => {
const adapter = (compositionDuration = 20, exportRenderSeek = false) => {
const runtime = createFrameSourceAdapter({
start: (element) => createRuntimeStartTimeResolver({}).resolveStartForElement(element, 0),
duration: (element) => createRuntimeStartTimeResolver({}).resolveDurationForElement(element),
compositionDuration: () => compositionDuration,
canonicalFps: () => 30,
exportRenderSeek: () => exportRenderSeek,
});
adapters.push(runtime);
return runtime;
Expand Down Expand Up @@ -215,6 +218,110 @@ describe("frame sources", () => {
b.resolve();
await waitForSeekCompletion();
});
it("keeps trimmed, sped-up and extended clips inside their original scene", async () => {
const host = mount("5", "8");
host.setAttribute("data-playback-start", "0.5");
host.setAttribute("data-playback-rate", "2");
const render = vi.fn();
disposers.push(
registerFrameSource({
element: host,
render,
sourceRange: { start: 3, duration: 4, fps: 60 },
}),
);
const runtime = adapter();
runtime.seek({ time: 5 });
await waitForSeekCompletion();
expect(render).toHaveBeenLastCalledWith(3.5, expect.any(AbortSignal));
runtime.seek({ time: 6 });
await waitForSeekCompletion();
expect(render).toHaveBeenLastCalledWith(5.5, expect.any(AbortSignal));
runtime.seek({ time: 12 });
await waitForSeekCompletion();
expect(render).toHaveBeenLastCalledWith(7 - 1 / 60, expect.any(AbortSignal));
host.setAttribute("data-start", "0");
host.setAttribute("data-playback-start", "0");
runtime.seek({ time: 0 });
await waitForSeekCompletion();
expect(render).toHaveBeenLastCalledWith(3, expect.any(AbortSignal));
});

it("uses half-open cut boundaries and holds the terminal scene's last source frame", async () => {
const first = vi.fn();
const last = vi.fn();
disposers.push(registerFrameSource({ element: mount("0", "3"), render: first }));
disposers.push(
registerFrameSource({
element: mount("3", "4"),
render: last,
sourceRange: { start: 10, duration: 4, fps: 60 },
}),
);
const runtime = adapter(7);
runtime.seek({ time: 3 });
await waitForSeekCompletion();
expect(first).not.toHaveBeenCalled();
expect(last).toHaveBeenLastCalledWith(10, expect.any(AbortSignal));
runtime.seek({ time: 7 });
await waitForSeekCompletion();
expect(last.mock.lastCall?.[0]).toBeCloseTo(14 - 1 / 60, 10);
});

it("does not redraw an ended scene across a gap or seek backward before its start", async () => {
const render = vi.fn();
disposers.push(registerFrameSource({ element: mount("2", "2"), render }));
const runtime = adapter(8);
for (const time of [0, 1, 4, 6, 8]) runtime.seek({ time });
await waitForSeekCompletion();
expect(render).not.toHaveBeenCalled();
runtime.seek({ time: 2 });
await waitForSeekCompletion();
expect(render).toHaveBeenCalledTimes(1);
});

it("matches snapped export visibility at near-frame starts and ends", async () => {
const render = vi.fn();
disposers.push(registerFrameSource({ element: mount("1.00001", "1"), render }));
const runtime = adapter(3, true);
runtime.seek({ time: 1 });
await waitForSeekCompletion();
expect(render).toHaveBeenLastCalledWith(0, expect.any(AbortSignal));
runtime.seek({ time: 59 / 30 });
await waitForSeekCompletion();
expect(render.mock.lastCall?.[0]).toBeCloseTo(59 / 30 - 1.00001, 10);
runtime.seek({ time: 2 });
await waitForSeekCompletion();
expect(render).toHaveBeenCalledTimes(2);
});

it("retains unsnapped authored timing during interactive preview", async () => {
const render = vi.fn();
disposers.push(registerFrameSource({ element: mount("1.00001", "1"), render }));
const runtime = adapter(3);
runtime.seek({ time: 1 });
await waitForSeekCompletion();
expect(render).not.toHaveBeenCalled();
runtime.seek({ time: 2 });
await waitForSeekCompletion();
expect(render).toHaveBeenCalledTimes(1);
});

it("rejects invalid original ranges before registering a source", () => {
const element = mount();
for (const sourceRange of [
{ start: -1, duration: 4, fps: 60 },
{ start: 0, duration: 0, fps: 60 },
{ start: 0, duration: 4, fps: NaN },
{ start: Infinity, duration: 4, fps: 60 },
]) {
expect(() => registerFrameSource({ element, render: vi.fn(), sourceRange })).toThrow(
"ranges",
);
}
disposers.push(registerFrameSource({ element, render: vi.fn() }));
});

it("drains a queued seek after an earlier draw fails and retains the failure for capture", async () => {
const first = deferred();
const render = vi
Expand Down
29 changes: 27 additions & 2 deletions packages/core/src/runtime/frameSources.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
import { exportClipWindow } from "../inline-scripts/parityContract";
import { sourceTimeAt } from "../speedRamp";
import { readElementRateSpec, readMediaStart } from "./playbackRate";
import { registerSeekCompletion } from "./adapters/seek-dispatch";
import { isClipVisibleAt } from "./clipWindow";
import type { RuntimeDeterministicAdapter } from "./types";

export interface FrameSource {
element: Element;
ready?: PromiseLike<unknown>;
sourceRange?: { start: number; duration: number; fps: number };
render: (sourceTime: number, signal: AbortSignal) => void | PromiseLike<unknown>;
dispose?: () => void;
}
Expand All @@ -24,6 +27,18 @@ export const hasFrameSources = (): boolean => sources.size > 0;
/** Bind a frame source to a timed host. Unregister before replacing its source. */
export function registerFrameSource(source: FrameSource): () => void {
if (sources.has(source.element)) throw new Error("This element already has a frame source");
const range = source.sourceRange;
if (
range &&
(!Number.isFinite(range.start) ||
range.start < 0 ||
!Number.isFinite(range.duration) ||
range.duration <= 0 ||
!Number.isFinite(range.fps) ||
range.fps <= 0)
) {
throw new Error("Frame source ranges require a nonnegative start, positive duration and FPS");
}
const controller = new AbortController();
const cancelled = new Promise<void>((resolve) => {
controller.signal.addEventListener("abort", () => resolve(), { once: true });
Expand Down Expand Up @@ -55,7 +70,9 @@ export function registerFrameSource(source: FrameSource): () => void {
element: source.element,
ready,
seek(time) {
pending = time;
pending = range
? range.start + Math.min(Math.max(0, time), Math.max(0, range.duration - 1 / range.fps))
: time;
work ??= drain();
return work;
},
Expand All @@ -74,6 +91,9 @@ export function registerFrameSource(source: FrameSource): () => void {
export function createFrameSourceAdapter(timing: {
start: (element: Element) => number;
duration: (element: Element) => number | null;
compositionDuration: () => number;
canonicalFps: () => number;
exportRenderSeek: () => boolean;
}): RuntimeDeterministicAdapter {
const owned = new Set<RegisteredSource>();
let readySources: RegisteredSource[] = [];
Expand Down Expand Up @@ -111,7 +131,12 @@ export function createFrameSourceAdapter(timing: {
for (const [element, source] of current()) {
const start = timing.start(element);
const duration = timing.duration(element);
if (time < start || (duration !== null && time > start + duration)) continue;
const end = start + (duration ?? Infinity);
const clipWindow = timing.exportRenderSeek()
? exportClipWindow(start, end, timing.canonicalFps())
: { start, end };
if (!isClipVisibleAt(time, clipWindow.start, clipWindow.end, timing.compositionDuration()))
continue;
const localTime = Math.max(0, time - start);
const sourceTime =
readMediaStart(element) + sourceTimeAt(readElementRateSpec(element), localTime);
Expand Down
3 changes: 3 additions & 0 deletions packages/core/src/runtime/init.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4025,6 +4025,9 @@ export function initSandboxRuntimeModular(): void {
createFrameSourceAdapter({
start: (element) => resolveStartForElement(element, 0),
duration: (element) => resolveDurationForElement(element),
compositionDuration: () => getSafeTimelineDurationSeconds(state.capturedTimeline, 0),
canonicalFps: () => state.canonicalFps,
exportRenderSeek: () => Boolean(window.__HF_EXPORT_RENDER_SEEK_CONFIG),
}),
createWaapiAdapter(),
createCssAdapter({
Expand Down
Loading
Loading