[Feature Request] [Web] Expose kvCacheQuantizationBits in the onnxruntime-web WebGPU EP options #33752

Describe the feature request

Please add kvCacheQuantizationBits (0 / 4 / 8) to InferenceSession.WebGpuExecutionProviderOption in onnxruntime-web, so it gets passed through like enableMatmulFp32Accumulation and the buffer cache modes already are.

Everything below comes from reading the source at 8d85527a0 and at current main (74bda515e). I haven't run any of it yet.

Where things stand

• The native WebGPU EP reads ep.webgpuexecutionprovider.kvCacheQuantizationBits in ParseEpConfig (core/providers/webgpu/webgpu_provider_factory.cc).
• js/web/lib/wasm/session-options.ts only maps a fixed list of WebGPU options to EP options: enableGraphCapture, preferredLayout, forceCpuNodeNames, validationMode, enableMatmulFp32Accumulation and the *BufferCacheMode keys. kvCacheQuantizationBits isn't one of them, and it doesn't appear in the js/ type definitions.
• A workaround looks possible: sessionOptions.extra entries go through _OrtAddSessionConfigEntry, and for EP-device builds WebGpuEpFactory::CreateIExecutionProvider builds the provider from session_options->value.config_options. So this would probably work:

  extra: { 'ep.webgpuexecutionprovider.kvCacheQuantizationBits': '8' }

  But it isn't documented, and I'm not sure it works on the minimal/legacy path, where the provider factory is created when the EP is appended, which happens before extra is applied.

Small related nit: sliding-window GQA layers now use flash attention (#32462, which shipped in 1.31.0), so KV cache quantization works with local_window_size. The error text in contrib_ops/webgpu/bert/group_query_attention.cc still says "Check that smooth_softmax and local_window_size are not set." Only smooth_softmax (and the other flash-attention preconditions) still rule out the flash path:

const int flash_local_window_size = kv_empty ? -1 : local_window_size_;
bool will_use_flash_attention = false;
if (!use_smooth_softmax_) {

At 8d85527a0 the gate was still if (!use_smooth_softmax_ && (!use_sliding_window || kv_empty)). That meant a quantized cache on a sliding-window layer would hit this error as soon as total_sequence_length passed the window. That part is fixed on main.

Describe scenario use case

We run small LLMs fully in the browser (Transformers.js + onnxruntime-web, WebGPU EP) on phones. One target is iPhone Safari, where a page gets about 2 GB in total, so memory, not compute, limits how long the context can be.

Our current model, LFM3-1.7B, is a hybrid: conv layers, full-attention GQA layers and sliding-window GQA layers (local_window_size 256), with 4 KV heads and head size 128. At 8K tokens its attention KV state is the biggest GPU cost after the weights, roughly 370 MB once both the past and present copies are counted for a step. Quantizing the KV cache to 8 or 4 bits would cut that by about 2x or 4x, which is enough to fit noticeably longer contexts on these devices.

A typed, documented JS option would make this usable from onnxruntime-web and from libraries built on it, like Transformers.js, which can only forward options that the JS API accepts.