[Web] [WebGPU] Single-pass prefill of ~900 tokens crashes Safari's GPU process on iOS 27 (iPhone) — please split large workloads into smaller submissions / dispatches #33754

Describe the issue

On iPhone (Safari, iOS 27.2), running an LLM prompt of ~870–920 tokens through the WebGPU EP in a single forward pass reliably crashes Safari's GPU process (com.apple.WebKit.GPU). Short prompts and long replies (token-by-token decode) are fine.

Reading the same prompts in 256-token chunks (several smaller forward passes over the KV cache) never crashed: prompts of 2,763 and ~7,600 tokens then work. So the trigger looks like the amount of GPU work in flight for one session.run() (large matmul/attention dispatches over ~900 positions), not the prompt length or memory use.

Because a GPU process crash takes down WebGPU/WebGL for every page in the browser, and a WebKit change would only reach future iOS versions, it would help a lot if ORT Web could keep its submissions small enough on Safari to avoid this on the iOS versions already shipped.

What the page sees

• The GPUDevice is lost with reason "destroyed".
• ORT fails with: Failed to download data from buffer. wgpu::MapAsyncStatus value: 3, message: map async was not successful
• In one run there was no error at all: the page just hung, because the pending mapAsync (the output download) never resolved after the device was lost. We now race our own device.lost handler against inference to get out of that state.

What the device shows

• Device log: GPUProcessProxy::gpuProcessExited: reason=Crash, and the GPU process "hit an unrecoverable exception".
• No jetsam (memory) kill of the GPU process, and no IOGPU/AGX fault lines.
• Two crash reports, same signature: EXC_BREAKPOINT (SIGKILL), com.apple.WebKit.GPU 8625.2.8.10.7, iPhone OS 27.2 (24B119). Crashed thread (queue com.apple.root.default-qos):

0  WebGPU             WebGPU::crashGPUProcess(WTF::RefPtr<WebGPU::Device, ...> const&, NSString*) + 84
1  WebGPU             invocation function for block in WebGPU::Queue::commitMTLCommandBuffer(id<MTLCommandBuffer>) + 144
2  libdispatch.dylib  _dispatch_client_callout + 16
3  libdispatch.dylib  _dispatch_continuation_pop + 316
4  libdispatch.dylib  _dispatch_source_latch_and_call + 160
5  libdispatch.dylib  _dispatch_source_invoke + 1124
6  libdispatch.dylib  _dispatch_queue_override_invoke + 468
7  libdispatch.dylib  _dispatch_root_queue_drain + 640
8  libdispatch.dylib  _dispatch_worker_thread2 + 196
9  libsystem_pthread  _pthread_wqthread + 232
10 libsystem_pthread  start_wqthread + 8

Safari GPU process (com.apple.WebKit.GPU) crashes - crashing-thread backtraces
Two crash reports, same signature. Identifiers, addresses and logs removed.

=== Crash 1 (2026-10-29 15:01 local time)
Process:        com.apple.WebKit.GPU 8625.2.8.10.7
Hardware:       iPhone17,2
OS:             iPhone OS 27.2 (24B119)
Exception:      EXC_BREAKPOINT (SIGKILL)
Termination:    namespace FOUNDATION, code 1
Crashed thread: 2 (queue: com.apple.root.default-qos)
0  WebGPU                    WebGPU::crashGPUProcess(WTF::RefPtr<WebGPU::Device, WTF::RawPtrTraits<WebGPU::Device>, WTF::DefaultRefDerefTraits<WebGPU::Device>> const&, NSString*) + 84
1  WebGPU                    invocation function for block in WebGPU::Queue::commitMTLCommandBuffer(id<MTLCommandBuffer>) + 144
2  libdispatch.dylib         _dispatch_client_callout + 16
3  libdispatch.dylib         _dispatch_continuation_pop + 316
4  libdispatch.dylib         _dispatch_source_latch_and_call + 160
5  libdispatch.dylib         _dispatch_source_invoke + 1124
6  libdispatch.dylib         _dispatch_queue_override_invoke + 468
7  libdispatch.dylib         _dispatch_root_queue_drain + 640
8  libdispatch.dylib         _dispatch_worker_thread2 + 196
9  libsystem_pthread.dylib   _pthread_wqthread + 232
10 libsystem_pthread.dylib   start_wqthread + 8

=== Crash 2 (2026-10-29 15:08 local time)
Process:        com.apple.WebKit.GPU 8625.2.8.10.7
Hardware:       iPhone17,2
OS:             iPhone OS 27.2 (24B119)
Exception:      EXC_BREAKPOINT (SIGKILL)
Termination:    namespace FOUNDATION, code 1
Crashed thread: 8 (queue: com.apple.root.default-qos)
0  WebGPU                    WebGPU::crashGPUProcess(WTF::RefPtr<WebGPU::Device, WTF::RawPtrTraits<WebGPU::Device>, WTF::DefaultRefDerefTraits<WebGPU::Device>> const&, NSString*) + 84
1  WebGPU                    invocation function for block in WebGPU::Queue::commitMTLCommandBuffer(id<MTLCommandBuffer>) + 144
2  libdispatch.dylib         _dispatch_client_callout + 16
3  libdispatch.dylib         _dispatch_continuation_pop + 316
4  libdispatch.dylib         _dispatch_source_latch_and_call + 160
5  libdispatch.dylib         _dispatch_source_invoke + 1124
6  libdispatch.dylib         _dispatch_queue_override_invoke + 468
7  libdispatch.dylib         _dispatch_root_queue_drain + 640
8  libdispatch.dylib         _dispatch_worker_thread2 + 196
9  libsystem_pthread.dylib   _pthread_wqthread + 232
10 libsystem_pthread.dylib   start_wqthread + 8

Context from open-source WebKit (for what it's worth)

The shipping iOS build isn't public, so this is only an inference from the matching symbol. In WebKit main, Source/WebGPU/WebGPU/Queue.mm, crashGPUProcess(const RefPtr<Device>&, NSString*) is called from one place: a 3-second watchdog that commitMTLCommandBuffer arms with dispatch_after on the default-priority global queue when each command buffer is committed. If the buffer hasn't completed by then, the device is lost and the GPU process is deliberately crashed ("command buffer ... did not complete within 3s timeout"). This landed as "Long running command buffers can hang the system" (https://commits.webkit.org/325185@main, merged from a Safari release branch). Separately, Metal command-buffer errors of the page-fault class (invalid resource/input, page fault, protection violation, stack overflow) also crash the GPU process there, while other errors only lose the device.

Our crash matches the watchdog path: same NSString* overload, called from a block in commitMTLCommandBuffer, running from a dispatch source on com.apple.root.default-qos. If so, note that the timer starts at commit, not when the GPU starts the buffer, so time a command buffer spends queued behind earlier submissions counts toward the 3 s. A whole ~900-token prefill submitted as many back-to-back command buffers could trip it even if no single dispatch is that long, which would fit chunking fixing it.

Requests

1. Keep the GPU work in flight per run() bounded on Safari/WebKit. For example: cap work per command buffer, split very large matmul/attention dispatches during prefill, and/or limit how far submissions run ahead of the GPU (wait for onSubmittedWorkDone before queuing more), so no submitted command buffer goes ~3 s without completing. An option for this would also work. (We see ep.webgpuexecutionprovider.maxNumPendingDispatches, default 16, in the native EP. We haven't tested whether lowering it from ORT Web helps, and it doesn't limit how many command buffers are queued.)
2. Fail pending downloads/mapAsync promptly when the device is lost. BufferManager::Download waits on MapAsync, and after device loss that wait can sit forever, leaving session.run() hanging with no error. Rejecting with the device-lost status (the EP already records it in the device-lost callback) would let apps recover by recreating the session.

A related feature request for chunked prefill in Transformers.js: huggingface/transformers.js#1836. That fixes this one model pipeline, but the EP-level fix above would cover every ORT Web workload on Safari.

To reproduce

Public app where this was found: https://github.com/daveycodez/lofi-agent. The current app reads prompts in 256-token chunks to work around this, so the crash needs single-pass prefill:

1. iPhone 16 Pro Max, iOS 27.2, Safari.

2. In a Web Worker, load daveycodez/LFM3-1.7B-Web (dtype: "q4f16", ~900 MB of weights, ~1.1 GB of GPU buffers live) with Transformers.js 4.3.2 on device: "webgpu", with onnxruntime-web 1.31.0-dev.20260914-8d85527a0.

3. Call model.generate() with a ~900-token prompt in one pass (e.g. a chat message with a pasted article or web search results added), max_new_tokens small.

   import { AutoTokenizer, AutoModelForCausalLM } from "@huggingface/transformers";
   const id = "daveycodez/LFM3-1.7B-Web";
   const tokenizer = await AutoTokenizer.from_pretrained(id);
   const model = await AutoModelForCausalLM.from_pretrained(id, { device: "webgpu", dtype: "q4f16" });
   const inputs = tokenizer.apply_chat_template(
     [{ role: "user", content: longText /* ~900 tokens */ }],
     { add_generation_prompt: true, return_dict: true },
   );
   await model.generate({ ...inputs, max_new_tokens: 16 });

4. Result: the GPU process crashes during the prefill run(). The device is lost ("destroyed") and generate() rejects with the MapAsyncStatus value: 3 error, or occasionally never settles.

5. Feeding the same prompt as consecutive 256-token chunks (with the KV cache carried between runs) completes normally, including for ~7,600-token prompts.

Expected: the workload completes, or at worst the device is lost and the pending call rejects. The GPU process should not crash.

Urgency

Any web app running mid-size LLMs on iPhone hits this as soon as a prompt passes roughly 900 tokens (pasted text, RAG/search context), and it also breaks WebGPU/WebGL in the user's other tabs. We have an app-side workaround (chunked prefill), so this isn't blocking us.

ONNX Runtime Installation

Released Package (npm dev build)

ONNX Runtime Version or Commit ID

onnxruntime-web 1.31.0-dev.20260914-8d85527a0

Execution Provider

'webgpu' (WebGPU)