Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions bench/src/benchmark/environment.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import { afterEach, describe, expect, it, vi } from 'vitest';
import type * as THREE from 'three/webgpu';
import { collectBenchmarkEnvironment } from './environment';
afterEach(() => vi.unstubAllGlobals());
describe('benchmark environment adapter identity', () => {
it('uses the rendering device even when a fresh/default adapter would differ', async () => {
const requestAdapter = vi.fn(() => { throw new Error('Must not request a probe adapter'); });
vi.stubGlobal('navigator', { userAgent:'Chrome test', platform:'Win32', gpu:{requestAdapter} });
const renderer = { backend:{ isWebGPUBackend:true,
device:{ adapterInfo:{vendor:'nvidia',architecture:'ampere',device:'',description:''},
features:new Set(['timestamp-query','shader-f16']) },
adapter:{info:{vendor:'intel',architecture:'gen-12lp'}} } };
const config = {dimensions:{width:1280,height:720,devicePixelRatio:1},ratio:2,timestepSeconds:1/60};
const result = await collectBenchmarkEnvironment(renderer as unknown as THREE.WebGPURenderer,
config as BenchmarkRunConfig);
expect(result.adapter).toBe('nvidia ampere');
expect(result.webgpuFeatures).toEqual(['shader-f16','timestamp-query']);
expect(requestAdapter).not.toHaveBeenCalled();
});
});
49 changes: 23 additions & 26 deletions bench/src/benchmark/environment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,32 +27,29 @@ export function collectBenchmarkEnvironment(
): Promise<BenchmarkEnvironment> {
const backend = renderer.backend as RendererBackendDetails;
const device = backend.device;
return navigator.gpu
.requestAdapter()
.then((adapter) => {
const adapterInfo = backend.adapter?.info ?? adapter?.info;
const adapterName = adapterInfo
? [
adapterInfo.vendor,
adapterInfo.architecture,
adapterInfo.device,
adapterInfo.description,
]
.filter(Boolean)
.join(' ')
: 'adapter-info-unavailable';
return {
browser: navigator.userAgent,
operatingSystem: navigator.platform,
adapter: adapterName,
backend: backend.isWebGPUBackend === true ? 'WebGPU' : 'unknown',
webgpuFeatures: device ? [...device.features].sort() : [],
threeVersion: THREE.REVISION,
dimensions: { ...config.dimensions },
ratio: config.ratio,
fixedTimestep: config.timestepSeconds,
};
});
// Report the device used to render; a fresh request can select another GPU.
const adapterInfo = device?.adapterInfo ?? backend.adapter?.info;
const adapterName = adapterInfo
? [
adapterInfo.vendor,
adapterInfo.architecture,
adapterInfo.device,
adapterInfo.description,
]
.filter(Boolean)
.join(' ')
: 'adapter-info-unavailable';
return Promise.resolve({
browser: navigator.userAgent,
operatingSystem: navigator.platform,
adapter: adapterName,
backend: backend.isWebGPUBackend === true ? 'WebGPU' : 'unknown',
webgpuFeatures: device ? [...device.features].sort() : [],
threeVersion: THREE.REVISION,
dimensions: { ...config.dimensions },
ratio: config.ratio,
fixedTimestep: config.timestepSeconds,
});
}

/**
Expand Down
5 changes: 2 additions & 3 deletions bench/src/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -226,8 +226,8 @@ window.__UPSCALER_BENCH__ = createBenchmarkApi({
let pageDisposed = false;
window.addEventListener(
'pagehide',
() => {
if (pageDisposed) return;
(event) => {
if (event.persisted || pageDisposed) return;
pageDisposed = true;
renderer.setAnimationLoop(null);
controls?.dispose();
Expand All @@ -238,7 +238,6 @@ window.addEventListener(
renderer.dispose();
window.__UPSCALER_BENCH__ = undefined;
},
{ once: true },
);

//* Main Loop
Expand Down
5 changes: 5 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ Maintained, for contributors:

Live records, kept current as findings land:

- [Async initialization migration](async-init-migration.md): before/after examples
for the draft #83 API change and preparation/readiness rules.
- [Windows cross-device audit](windows-cross-device.md): adapter metadata, NVIDIA
and Intel verification, lifecycle fixes and remaining measurement limits.

- [FSR 3.1.5 parity report](research/PARITY.md): where this implementation matches
and diverges from FSR 3.1.5, and the measurements behind each choice.
- [Paper notes](research/PAPER-NOTES.md): a running tracker of write-up-worthy
Expand Down
113 changes: 113 additions & 0 deletions docs/async-init-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Async initialization migration

[PR #83](https://github.com/pmndrs/upscaler/pull/83) changes `Upscaler.init()` from
`void` to `Promise<void>`, and adds explicit async initialization to `UpscalePass`
and `MomentsPass`. These changes are not published yet.

The renderer still needs `await renderer.init()` before constructing these objects.
Configure the selected path first, then await preparation before starting dispatch.

## Raw Upscaler

Before:
```ts
const upscaler = new Upscaler({ renderer });
upscaler.init(); // synchronous compilation
upscaler.configure(config);
startRendering();
```

After:
```ts
const upscaler = new Upscaler({ renderer });
upscaler.configure(config); // still synchronous; allocates and queues preparation
await upscaler.init(); // waits for the selected path's required pipelines
startRendering();
```

The existing rendering/jitter recipe stays the same. Raw dispatch before mandatory
passes are ready throws `UpscalerNotReadyError` before encoding compute commands.
Its stable code is `UPSCALER_NOT_READY`; its reason is `preparing` or `device-lost`.
Actual premature dispatch also warns once per instance with the migration step.
Correctly awaited callers receive no migration warning. MomentsPass uses the same
diagnostic contract. This detects unready dispatch, not whether a Promise was ignored.

## UpscalePass

Before, the constructor initialized its internal upscaler synchronously:
```ts
const pass = new UpscalePass(renderer);
pass.configure(config);
startRendering();
```

After:
```ts
const pass = new UpscalePass(renderer);
pass.configure(config);
await pass.init();
startRendering();
```

Its `draw()` / `present()` methods also support an unjittered input fallback while
preparing, but awaiting startup makes the initial upscale ready before the loop.

## MomentsPass

Before:
```ts
const moments = new MomentsPass({ renderer });
moments.configure({ width, height, space: 'ycocg' });
moments.dispatch({ source: giTexture });
```

After:
```ts
const moments = new MomentsPass({ renderer });
moments.configure({ width, height, space: 'ycocg' });
await moments.init();
moments.dispatch({ source: giTexture });
```

## Path changes, readiness and errors

`Upscaler.prepare()` warms the currently selected path and requested optional
settings. `init()` uses the same preparation contract. For an existing raw loop:
```ts
async function changePath(path: 'temporal' | 'spatial') {
renderer.setAnimationLoop(null);
upscaler.configure({ ...config, path });
await upscaler.prepare();
renderer.setAnimationLoop(renderFrame);
}
```

Catch rejection to display a preparation error instead of resuming dispatch.
The library observes bare initialization rejections to prevent an ignored init
Promise from producing an additional unhandled rejection. Compilation failures
are logged, and the original Promise still rejects for callers that await/catch it.
Explicit `init()` / `prepare()` can retry compilation failures; dispatch does not
silently retry failed shaders. Existing pipelines are shared per device, so an
already prepared path or ordinary resize usually requires no new compilation.

`upscaler.isReady` means the mandatory passes are ready. A newly requested optional
debug/shading feature can still be preparing while it is true. Rendering continues
with that feature temporarily inactive; it activates at a frame boundary and resets
the affected history. Await `prepare()` if the host needs that feature warmed first.

For code that keeps rendering an input fallback, check `isReady` before raw dispatch.
You can also catch `UpscalerNotReadyError` (or check its code across bundles) in a
central error handler. A `device-lost` reason requires a working renderer/device;
awaiting the lost instance again does not restore it.

## TSL factories

These factories remain synchronous:
```ts
pipeline.outputNode = upscaleScene(scene, camera, { ratio: 2 });
renderer.setAnimationLoop(() => pipeline.render());
```

Do not add an await to `upscaleScene()`, `upscale()`, `upscaleSpatial()` or
`temporalGuides()`. Their nodes keep the input visible without jitter while
preparing; linked guides/upscale nodes share readiness.
7 changes: 4 additions & 3 deletions docs/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,10 @@ re-verified on r186.1.
- **`timestamp-query`** is optional and only used when GPU timing is opted into
(`gpuTiming: true`; off by default). Without it, `upscaler.gpuTimings` stays empty
and nothing else changes. It's often missing on mobile.
- **Measured hardware.** Performance and quality measurements so far come from one
adapter family (Apple Metal). Treat the published millisecond figures as indicative
on other GPUs, especially mobile tilers.
- **Measured hardware.** The original performance/quality program used Apple Metal.
The [Windows cross-device audit](windows-cross-device.md) adds NVIDIA and Intel
coverage, with explicit adapter identity and noise limits. Treat figures as
specific to their recorded workload and hardware; mobile tilers remain unverified.

## Out of scope by design

Expand Down
1 change: 1 addition & 0 deletions docs/windows-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
Baseline: v0.4.0 / bfb343c. Startup implementation: 3d018e1.

Current implementation status: [Windows follow-ups completed](windows-followups.md).
Latest coverage: [Windows cross-device audit](windows-cross-device.md).
The observations below are the original audit snapshot.

## Result and measurements
Expand Down
117 changes: 117 additions & 0 deletions docs/windows-cross-device.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Windows cross-device audit

This follow-up to [the initial audit](windows-audit.md) addresses the Windows part
of [issue #4](https://github.com/pmndrs/upscaler/issues/4). It is based on the draft
PR stack #83 → #84 → #85. No changes have been merged or published.

## Fixes

- Exposure captures copy GPUAdapterInfo's getter fields explicitly; serializing
the native object had produced `{}`.
- Benchmark environment metadata now reads the rendering device's adapterInfo.
Requesting another default adapter could report a different GPU on this laptop.
- Measurement scripts accept a power preference and expected vendor. The vendor
is checked against the returned adapter before allocating its device. Windows
Chrome warns that powerPreference is ignored, so the hint alone is insufficient.
- Demo 13 resizes its existing scene pass and RTT instead of rebuilding their
identities. It disposes all owned graph products when the page exits.
- Demo 13 updates its renderer DPR, including changes arriving after a resize
event or without a CSS-size change. Its output follows the drawing-buffer size.
- Page-exit handlers preserve pages marked as persisted, rather than disposing
their renderer while the browser may restore the same document.
- Owned browser shutdown uses the browser CDP session, releases inherited pipe
handles and tolerates Edge's successful launcher relaunch. The benchmark rejects
occupied CDP ports instead of silently attaching to another process.
- Premature imperative dispatch warns once and throws a recognizable
`UpscalerNotReadyError`, code `UPSCALER_NOT_READY`. Correctly awaited callers stay
quiet. Ignored bare preparation rejections are observed while awaited calls still
reject; device-loss diagnostics explain that a new working device is required.

## Verified hardware and coverage

Windows 11, Node 22.23.3; NVIDIA RTX 3060 Laptop (driver 32.0.15.9200) in Chrome
154.0.8037.98 and Intel UHD / Gen-12LP (driver 31.0.101.5333) in Edge
154.0.4258.53. Actual vendor/architecture and enabled device features are recorded.
Browser choice and GPU differ together here; this is not an isolated browser A/B.
The harness uses its own profiles and `--enable-unsafe-webgpu`.

| Area | Status | Evidence |
| --- | --- | --- |
| Adapter reporting | Verified | Getter serialization and actual-device unit coverage; vendor assertions on real devices. |
| Exposure | Verified | Three profiles per GPU, five bit-identical rgba16float readback fixtures per profile. NVIDIA retains about 44% lower pass time; Intel has no consistent runtime gain. Compile time improves on both. |
| Q12 convergence | Verified | 180 settle frames and 12 consecutive pairs, ratio 2: approximately 0.0221 NVIDIA / 0.0219 Intel, mean absolute RGB difference on the 0–255 scale. |
| Benchmark modes and debug views | Verified | Both GPUs run native, bilinear, spatial and temporal; five quality settings and all exposed debug views. Effective debug values match the selections. Zero synchronous compute creation or WebGPU validation errors. |
| Demo 13 resize / DPR | Verified | Thirty alternating size/DPR cycles per GPU retain 26 live textures, 46 buffers and eight query sets. At DPR 2, 800×600 CSS pixels produce 1600×1200 output and 800×600 input. |
| Page suspension | Verified, bounded | Built-gallery freeze/active test resumes the same document with a ready shared driver. Development-server reload trials are excluded. This is not physical OS sleep/resume. |
| Missing timestamp feature | Verified | A real NVIDIA device is requested without timestamp-query; 15 resize cycles render correctly and allocate zero query sets. |
| Fresh instance disposal | Reproduced upstream retention | Thirty cycles of four new UpscalePass instances release textures/query sets but retain six renderer binding buffers per instance, including after GC. See below. |
| Scatter and candidate timings | Smoke / noise-qualified | Sequential ABBA comparisons record their noise floors. Do not infer a Windows speed ranking where the delta fails to clear that floor. Candidate image quality was not regraded. |

NVIDIA dispatch-only timer medians were +0.24% temporal versus a 0.17% noise floor,
+0.84% spatial versus 0.55%, and +0.28% bilinear versus 7.10%. These do not clear
the guide's 3×-noise threshold. CPU encoding differences were roughly 5–10 µs.
Timing stays opt-in; these numbers do not establish a whole-application FPS gain.

[Compact evidence](windows-evidence/cross-device.json) records both completed
measurements and explicit limits. Screenshots and complete samples remain under
`bench/results/windows-local/cross-device` and `bench/results/raw`.

## Resource findings

Before the demo change, thirty resizes increased tracked live textures from
26 to 176 and buffers from 46 to 226. Disposing the abandoned targets fixes the
texture growth. Keeping scene-pass identities stable also avoids the binding
buffers retained by three's per-pass render-object cache.

A separate fresh-instance test still reproduces renderer-level retention:
46 → 70 buffers after four instances, then 766 after 120 instances. The same
pattern occurs on Intel and with timestamps disabled (30 → 390 after 60 instances).
Textures and query sets return to baseline. Weak references remain reachable after
forced GC, so this is more than a missing explicit destroy call in the counter.

This test intentionally reuses scene meshes/materials and one renderer. Three
r186.1's RenderObjects cache retains render objects for the retired target/pass
contexts; disposing their RenderTargets does not evict those objects. The library
should not evict other consumers' renderer caches or dispose shared scene materials.
An upstream lifecycle fix needs a focused reproduction and multi-consumer checks.

## Reproduce

Build the gallery before running its production lifecycle test:

```powershell
npm run examples:build
node scripts/audit-windows-lifecycle.mjs --cycles 30 --hold-seconds 60 --power-preference high-performance --expected-vendor nvidia
node scripts/audit-windows-lifecycle.mjs --cycles 15 --hold-seconds 10 --without-timestamps --power-preference high-performance --expected-vendor nvidia
node scripts/audit-windows.mjs --bench --exercise --power-preference high-performance --expected-vendor nvidia
node scripts/measure-convergence.mjs --scenario Q12 --ratio 2 --power-preference high-performance --expected-vendor nvidia --label windows-nvidia
node scripts/measure-timer-overhead.mjs --attach-only --frames 100 --blocks 12 --inflight 8 --conditions temporal:2:dispatch:none,temporal:2:dispatch,spatial:2:dispatch,bilinear:2:dispatch --power-preference high-performance --expected-vendor nvidia --label windows-nvidia
node scripts/run-benchmark.mjs --smoke --variant baseline --comparison reconstruct-cross-frame-v1 --ratios 1,2,3 --blocks 4 --warmup 60 --samples 120 --power-preference high-performance --expected-vendor nvidia
```

For the Intel runs here, use `--power-preference low-power --expected-vendor intel`
and the installed Edge executable. `--browser` selects it in the audit scripts;
`--chrome` selects it in the measurement scripts. A mismatch aborts the measurement.
Run GPU jobs sequentially and freeze source changes while measuring. A development
reload interrupted one timer trial; its incomplete results are excluded.

## Remaining priorities

1. **Renderer buffer lifetime:** upstream the retired-target reproduction. Accept
only bounded buffers across repeated target/instance rebuilds while another
consumer continues rendering correctly. This relates to the upstream tracker #35.
2. **DPR across the remaining demos:** audit their cached startup-DPR values and
move shared sizing into a consistent helper. Demo 13 is verified; that is not
proof of every demo or a physical mixed-monitor transition.
3. **Timing stability:** repeat noisy comparisons with longer warmup/sample windows
and recorded clock/power state before adopting shader optimizations. NVIDIA P8
at 210/405 MHz was observed during an earlier rendering hold; clocks were not
continuously recorded for every leg.
4. **Physical Windows lifecycle:** manual sleep/resume, minimize/restore and moving
between monitors remain untested. Check frame progress, adapter/device loss,
drawing-buffer/output size and console errors before and after each transition.
5. **Wider adapters and compilation:** AMD, mobile tilers, forced FXC, extended
device recovery and full Homefig integration remain outside this local matrix.

Lint, typecheck, portable tests, library artifact verification and gallery
build passed locally. The release-workflow cases run in the full CI matrix.
Loading
Loading