Skip to content

test(studio): the edit bench render serves GSAP from the repo, never the CDN - #4996

Merged
miguel-heygen merged 5 commits into
mainfrom
test/edit-bench-render-local-gsap
Oct 4, 2026
Merged

miguel-heygen merged 5 commits into
mainfrom
test/edit-bench-render-local-gsap

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

What this fixes

The edit accuracy case rotate-size-px-r0-root-z200-on failed its render metric on some CI runs and passed on others with the same code (65.62 px vs 0.03 px). The failing runs also took about 30 s longer (60 s vs 29 s).

Cause: the bench's render step renders the saved project through the producer's own browser, and that browser fetched GSAP from cdn.jsdelivr.net. #4968 made the Studio half serve GSAP from the repo, but not this step. When jsdelivr stalled, the fixture's blocking GSAP script held the page load until Chrome gave the request up (about 30 s), GSAP never ran, the producer's short wait after a failed script found no timeline, and the frame showed the box at its 0 s size (240x160) instead of its 2 s size (300x200). The failing CI runs already included #4968.

A second page had the same problem: the producer's static-frame dedup opens a verification page that loads the project again from the network. The bench captures one frame, so it needs no dedup.

Change

  • render.mjs serves the fixture's CDN requests from the repo on the producer's page (the same serveFixtureAssetsLocally the Studio half uses, now exported from case.mjs), before the page loads.
  • render.mjs turns static-frame dedup off for its one-frame render, so no second page fetches GSAP.
  • render.test.mjs runs renderBox with the producer mocked and checks that the interception pattern covers the fixture's GSAP URL, that GSAP is served from the repo's gsap dist, and that dedup is off.

Evidence

Same saved project, a proxy that accepts connections and never answers standing in for a stalled jsdelivr:

Render step CDN render Case time
main answers 0.03 px 30 s
main stalls 65.62 px (the CI failure) 92 s
this branch stalls 0.03 px, 3 of 3 runs 27 s each
  • render.test.mjs: 3 of 3 runs exit 0. Six mutations each turn it red: no local serving ("expected [] to deeply equal ['Fetch.enable', …]"), dedup left on ("expected { playerReadyTimeout: 10000 } to match object { staticFrameDedup: false }"), an interception pattern for another host, the pattern without its trailing * (Chrome then matches only the exact URL; the test matches patterns the way Chrome does), and a pattern narrowed by requestStage: "Response" or by resourceType: "Document" (the test checks the whole pattern object).
  • The edit accuracy test folder: 11 files, 60 tests, exit 0.
  • oxfmt, oxlint, comment ratchet, comment citations and fallow audit: all exit 0.

Chrome still opens two speculative connections to jsdelivr per run with the CDN stalled. Nothing waits on them: the case time matches the run with the CDN answering.

No visible change

Test harness only; nothing a Studio user sees changes.

Why it ships alone

Under 100 lines on purpose: this flake can turn the required edit accuracy gate red on any unrelated Studio PR, and the next change in this area (the crop-scale render fix) touches the runtime and waits on its own decision, so nothing ready can carry it.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

Edit accuracy: accurate 1556 (base branch 1556), smooth 1426 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

@miguel-heygen
miguel-heygen marked this pull request as ready for review October 4, 2026 09:06

@terencecho terencecho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving cfb1a296. The bench's render step no longer waits on jsdelivr, and the fix reaches the real browser, not just the mocked test.

What I verified

  • End to end, with a stalled CDN. I ran renderBox on the rotate-size-px-r0-root-z200-on fixture with a proxy that accepts connections and never answers (stand-in for a stalled jsdelivr). Before this change (the two render.mjs edits removed): 33.2 s, box 240x160, which is the 0 s frame and the CI failure. At this head: 1.1 s, box 300x200, twice. With the network reachable, the old code gives 300x200, so the stall is what flips it.
  • Unit test. render.test.mjs passes 3 of 3 runs at this head (needs NODE_ENV=test and the dependency packages built, as for the other studio tests). It is picked up by the studio vitest config like its siblings.
  • Mutants. Five changes to render.mjs, each failing the one test: no local serving (expected [] to deeply equal ['Fetch.enable', …]), dedup option dropped, dedup set to true, serving started after initializeSession (so the first request is missed), and serving attached to a different page.
  • Scope. Three files under packages/studio/tests/e2e/edit-accuracy/: export added to serveFixtureAssetsLocally, two lines in render.mjs, and the new test. No product or runtime change, and case.mjs does not import render.mjs, so the new import adds no cycle.

Non-blocking

  • The Fetch pattern is https://cdn.jsdelivr.net/*, and any jsdelivr URL that is not the installed gsap's dist is now failed with BlockedByClient on the producer page too (and named once by the existing warning), same as the Studio half. Fine for today's fixtures; a fixture that loads another jsdelivr file would now fail its render alone.
  • The unit test mocks the producer, so it pins the wiring (pattern, order, dedup off), not real interception. My run above is the real-browser check.

CI at this head is not finished: 56 pass, 2 skipped, 0 failing, 35 still pending (20 Studio: edit accuracy shards, 10 regression-shards, and the Windows jobs). Nothing is red, so this approval is on code merit, not CI completion. The edit accuracy shards are the ones to watch, since they exercise this change. It is not authorization to merge or deploy beyond what the gate already does.

— Review by tai (pr-review)

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 4, 2026
Merged via the queue into main with commit 7f34649 Oct 4, 2026
169 of 170 checks passed
@miguel-heygen
miguel-heygen deleted the test/edit-bench-render-local-gsap branch October 4, 2026 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants