perf(studio): long clips mount only the picture tiles in view (#4993)

* perf(studio): long clips mount only the picture tiles in view

A clip's picture strip mounted one tile per frame width, so a 10-minute
clip at full zoom mounted 18,000 tiles. The strip-size hook now also
reports the strip's on-screen span (refreshed on scroll, resize and clip
moves), and the video, image and composition strips mount only the tiles
inside it.

* refactor(studio): read the strip span outside the state updater

* fix(studio): a dragged long clip keeps its picture across the screen

The strip refreshed its span only on scroll, resize or a style change on
its own clip, but a drag moves the ghost through a wrapper, so a long
clip's ghost went blank past about 600 px of drag. Each strip now marks
its unmounted ends and re-measures when either nears the screen,
whatever moved it, and commits before paint. The tile row is one shared
ThumbnailTiles for the video, image and composition strips.

* fix(studio): warn of a tile gap inside the timeline scroller too

* fix(studio): clip strips re-measure together, one layout per frame

With row virtualization off the timeline mounts every clip, and each
strip had its own scroll listener and frame that read layout and then
committed. A scroll jump changed every strip's span, so a 1,000-clip
timeline laid out about once per clip per frame (the viewport gate's
1,000-row arm went to ~3 s frames). All strips now share one scroll
listener, observers and frame: every position is read first, then every
update commits in one batch.

* perf(studio): a scroll re-renders only the clip strips crossing the screen edge

A strip's span is clamped to its own measured width, and each strip keeps
its current size outside React so an unchanged size never calls setState.
A clip wholly on or wholly off screen then does no work on a scroll; only
strips straddling the edge re-render.

* perf(studio): a scroll re-measures only clip strips near the screen

The shared frame read every mounted strip's position, so a timeline of
1,000 unvirtualized clips forced about 1,000 style-and-layout passes per
scroll frame. An IntersectionObserver now tracks which strips are within
256 px of the screen; a scroll re-measures only those, and a strip is
measured as it comes near. A clip wholly off screen mounts no tile.

* test(studio): share the strip test setup through the root harness

* test(studio): a strip is measured as it comes near the screen

* fix(studio): a scroll jump measures the clips it brings on screen in the same frame

Each frame moves every strip's last measured box by its scroller's offset
change and reads only the boxes that land within 256 px of the screen. A
jump therefore measures exactly the strips it brings into view, with no
blank frame, at the cost of a short scroll; strips far away get their span
from the moved box without a layout read.

* perf(studio): a strip off either side of the screen keeps one empty span

* perf(studio): a clip strip far above or below the screen mounts no tiles

* perf(studio): a scroll frame leaves off-screen clip strips untouched

* perf(studio): short clips keep every tile and do no work on a scroll

A strip up to 4,096 px wide keeps all its tiles, as before this change,
and a scroll frame skips it; only long clips mount the tiles in view.
Short clips gain nothing from virtualizing and paid a re-render each
time they crossed the screen's edge.

* fix(studio): a scroll is never applied twice to a clip strip's position

Each strip remembers the scroll offset its box was read at, so a strip read
between a scroll and that scroll's frame is not shifted again. The empty-span
guards are removed: a near strip always starts inside itself and a far one
already gets nothing.

* fix(studio): a clip on screen keeps its pictures after a move and a scroll

A strip showing tiles is re-read on every scroll frame instead of trusting
its last box shifted by the scroll, which a move without a scroll (rows
removed above it, a dock resize) leaves stale. Tests now pin the vertical
shift, the 256 px band, the 4,096 px short-clip limit and the tile padding.

Author: miguel-heygen

packages/studio/src/components/sidebar/CompositionsTab.thumbnails.test.tsx

@@ -39,6 +39,7 @@ class ViewportObserver {
     const entry = { isIntersecting: true } as IntersectionObserverEntry;
     this.callback([entry], this as unknown as IntersectionObserver);
   }
+  unobserve() {}
   disconnect() {}
 }
 

packages/studio/src/hooks/intersectionObserverTestUtils.ts

@@ -0,0 +1,16 @@
+// Test-only IntersectionObserver stand-in: everything observed is reported near the screen, asynchronously.
+export function NearScreenIntersectionObserver(callback: IntersectionObserverCallback) {
+  const observer = {
+    observe(target: Element) {
+      queueMicrotask(() =>
+        callback(
+          [{ isIntersecting: true, target } as IntersectionObserverEntry],
+          observer as unknown as IntersectionObserver,
+        ),
+      );
+    },
+    unobserve() {},
+    disconnect() {},
+  };
+  return observer;
+}

packages/studio/src/hooks/resizeObserverTestUtils.ts

@@ -1,14 +1,21 @@
 // Test-only ResizeObserver stand-in: a test delivers a size by hand with `reportResize`.
 let resizeCallback: ResizeObserverCallback | null = null;
+let observed: Element | null = null;
 
 export function MockResizeObserver(callback: ResizeObserverCallback) {
   resizeCallback = callback;
-  return { observe() {}, disconnect() {}, unobserve() {} };
+  return {
+    observe(target: Element) {
+      observed = target;
+    },
+    disconnect() {},
+    unobserve() {},
+  };
 }
 
 export function reportResize(width: number, height: number): void {
   resizeCallback!(
-    [{ contentRect: { width, height } } as ResizeObserverEntry],
+    [{ contentRect: { width, height }, target: observed } as ResizeObserverEntry],
     {} as ResizeObserver,
   );
 }

packages/studio/src/hooks/useThumbnailStripSize.test.tsx

@@ -1,7 +1,9 @@
 // @vitest-environment happy-dom
 import { act } from "react";
 import { createRoot } from "react-dom/client";
-import { expect, it } from "vitest";
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { createHappyDomRootHarness } from "../player/components/testRootHarness";
+import { NearScreenIntersectionObserver } from "./intersectionObserverTestUtils";
 import { MockResizeObserver, reportResize } from "./resizeObserverTestUtils";
 import { useThumbnailStripSize } from "./useThumbnailStripSize";
 
@@ -45,3 +47,236 @@ it("does not re-render the strip when the observer reports the size it already h
     globalThis.ResizeObserver = originalResizeObserver;
   }
 });
+
+describe("on a scroll", () => {
+  const originalResizeObserver = globalThis.ResizeObserver;
+  const originalIntersectionObserver = globalThis.IntersectionObserver;
+  const frames: FrameRequestCallback[] = [];
+  const harness = createHappyDomRootHarness();
+  let host: HTMLDivElement;
+  let root: ReturnType<typeof createRoot>;
+  let left = 0;
+  let top = 0;
+  let scrolled = 0;
+  let scrolledDown = 0;
+  let stripWidth = 1_000_000;
+  let renders = 0;
+
+  function Strip() {
+    renders += 1;
+    const [size, ref] = useThumbnailStripSize();
+    return (
+      <div>
+        <div ref={ref} data-in-view={`${size.inViewStart}-${size.inViewEnd}`} />
+      </div>
+    );
+  }
+
+  const mountStrips = (count: number) =>
+    act(async () =>
+      root.render(Array.from({ length: count }, (_, index) => <Strip key={index} />)),
+    );
+
+  const spans = () =>
+    [...host.querySelectorAll("[data-in-view]")].map((strip) => strip.getAttribute("data-in-view"));
+
+  /** The timeline scrolls so the strips' left edge lands at `nextLeft`, then one frame runs. */
+  const scrollTo = (nextLeft: number) => {
+    scrolled += left - nextLeft;
+    left = nextLeft;
+    act(() => host.dispatchEvent(new Event("scroll")));
+    expect(window.requestAnimationFrame).toHaveBeenCalledTimes(1);
+    act(() => frames.splice(0).forEach((frame) => frame(0)));
+  };
+
+  const neverReportedNear = () => {
+    globalThis.IntersectionObserver = class {
+      observe() {}
+      unobserve() {}
+      disconnect() {}
+    } as unknown as typeof IntersectionObserver;
+  };
+
+  beforeEach(() => {
+    globalThis.ResizeObserver = MockResizeObserver as unknown as typeof ResizeObserver;
+    globalThis.IntersectionObserver =
+      NearScreenIntersectionObserver as unknown as typeof IntersectionObserver;
+    vi.spyOn(window, "requestAnimationFrame").mockImplementation((frame) => frames.push(frame));
+    vi.spyOn(Element.prototype, "getBoundingClientRect").mockImplementation(
+      () => ({ left, top, width: stripWidth, height: 40 }) as DOMRect,
+    );
+    vi.spyOn(HTMLElement.prototype, "clientWidth", "get").mockImplementation(() => stripWidth);
+    left = 0;
+    top = 0;
+    scrolled = 0;
+    scrolledDown = 0;
+    stripWidth = 1_000_000;
+    renders = 0;
+    host = document.body.appendChild(document.createElement("div"));
+    host.setAttribute("data-timeline-scroll-viewport", "");
+    Object.defineProperty(host, "scrollLeft", { configurable: true, get: () => scrolled });
+    Object.defineProperty(host, "scrollTop", { configurable: true, get: () => scrolledDown });
+    root = harness.mount(host);
+  });
+
+  afterEach(() => {
+    frames.length = 0;
+    vi.restoreAllMocks();
+    globalThis.ResizeObserver = originalResizeObserver;
+    globalThis.IntersectionObserver = originalIntersectionObserver;
+  });
+
+  it("re-measures every near strip in one shared frame, so a full timeline lays out once", async () => {
+    await mountStrips(50);
+
+    scrollTo(-10_000);
+
+    expect(spans()).toEqual(Array(50).fill("9216-11776"));
+  });
+
+  it("keeps a visible strip measured when it is read after a scroll but before that scroll's frame", async () => {
+    stripWidth = 5_000;
+    left = -1_000;
+    await mountStrips(1);
+
+    scrolled += 3_000;
+    left = -4_000;
+    act(() => reportResize(stripWidth, 40));
+    act(() => host.dispatchEvent(new Event("scroll")));
+    act(() => frames.splice(0).forEach((frame) => frame(0)));
+
+    expect(spans()).toEqual(["3072-5000"]);
+  });
+
+  it("keeps a strip on screen measured after a move without a scroll, then a scroll", async () => {
+    await mountStrips(1);
+    expect(spans()).toEqual(["0-1536"]);
+
+    left = -3_000;
+    scrollTo(-1_000);
+    expect(spans()).toEqual(["0-2560"]);
+  });
+
+  it("measures a strip a vertical scroll brings on screen in that same frame", async () => {
+    neverReportedNear();
+    top = 5_000;
+    await mountStrips(1);
+    expect(spans()).toEqual(["0-0"]);
+
+    scrolledDown += 4_900;
+    top = 100;
+    act(() => host.dispatchEvent(new Event("scroll")));
+    act(() => frames.splice(0).forEach((frame) => frame(0)));
+
+    expect(spans()).toEqual(["0-1536"]);
+  });
+
+  it("keeps the tiles of a strip that ends just off screen", async () => {
+    await mountStrips(1);
+
+    scrollTo(-(stripWidth + 100));
+
+    expect(spans()).toEqual(["999424-1000000"]);
+  });
+
+  it("measures the strips a jump brings on screen in that same frame", async () => {
+    neverReportedNear();
+    left = 50_000;
+    await mountStrips(50);
+    expect(spans()).toEqual(Array(50).fill("0-0"));
+
+    scrollTo(0);
+
+    expect(spans()).toEqual(Array(50).fill("0-1536"));
+  });
+
+  it("does not re-render a strip wholly on screen when it moves", async () => {
+    stripWidth = 300;
+    left = 100;
+    await mountStrips(1);
+    const settled = renders;
+
+    scrollTo(700);
+
+    expect(renders).toBe(settled);
+  });
+
+  it("does not re-render strips that a jump carries from one side of the screen to the other", async () => {
+    left = 50_000;
+    await mountStrips(50);
+    const settled = renders;
+
+    scrollTo(-2_000_000);
+
+    expect(renders).toBe(settled);
+  });
+
+  it("keeps every tile of a short clip, even far from the screen", async () => {
+    stripWidth = 4_096;
+    top = 5_000;
+    await mountStrips(1);
+
+    expect(spans()).toEqual(["0-4096"]);
+  });
+
+  it("never re-measures a short clip on a scroll", async () => {
+    stripWidth = 300;
+    left = 100;
+    await mountStrips(1);
+    const reads = vi.mocked(Element.prototype.getBoundingClientRect);
+    reads.mockClear();
+
+    scrollTo(50);
+
+    expect(reads).not.toHaveBeenCalled();
+  });
+
+  it("mounts nothing for a strip far above or below the screen", async () => {
+    top = 5_000;
+    await mountStrips(1);
+
+    expect(spans()).toEqual(["0-0"]);
+  });
+
+  it("measures a strip as it comes near the screen, without waiting for a frame", async () => {
+    const reports: IntersectionObserverCallback[] = [];
+    globalThis.IntersectionObserver = class {
+      constructor(callback: IntersectionObserverCallback) {
+        reports.push(callback);
+      }
+      observe() {}
+      unobserve() {}
+      disconnect() {}
+    } as unknown as typeof IntersectionObserver;
+    await mountStrips(1);
+    const strip = host.querySelector("[data-in-view]")!;
+
+    left = -10_000;
+    const [presence] = reports;
+    act(() =>
+      presence!(
+        [
+          {
+            isIntersecting: true,
+            target: strip.parentElement!,
+          } as unknown as IntersectionObserverEntry,
+        ],
+        {} as IntersectionObserver,
+      ),
+    );
+
+    expect(frames).toHaveLength(0);
+    expect(strip.getAttribute("data-in-view")).toBe("9216-11776");
+  });
+
+  it("reads no strip far from the screen, however many clips the timeline mounts", async () => {
+    left = 50_000;
+    await mountStrips(50);
+    const reads = vi.mocked(Element.prototype.getBoundingClientRect);
+    reads.mockClear();
+
+    scrollTo(49_900);
+
+    expect(reads).not.toHaveBeenCalled();
+  });
+});