[ivan-digital]

Repeated long system/tool prompts make each Gemma 4 12B request pay the full prefill cost. The runtime now skips discarded vocabulary projections and keeps one bounded, immutable system/tool prefix. Each request extends independent KV cache objects, including sliding-window position metadata; user turns and tool results are excluded from retention.

On one M5 Pro synthetic prompt (8,499 input tokens, 15 generated tokens), the original path took 10.180 and 10.292 seconds around the optimized pair; optimized cold took 9.352 seconds and warm took 0.778 seconds. All four generations had identical token IDs, raw output, and tool arguments. This is a local timing probe, not an accuracy result or a general latency guarantee.

The 12B prefix budget defaults to 1,024 MiB and uses the existing SPEECH_SWIFT_GEMMA4_PREFIX_CACHE_MB override (0 disables retention). E4B keeps its existing default. Native responses report cachedPrefixTokens, capabilities report cache support, and provenance records the effective budget. Local model documentation and a reproducible benchmark summary are included.

Validation: 10 focused config/prefix tests pass. Real-weight release tests pass for cold/warm token parity, a long second user followed by the original user (cache isolation), native tool/result history and cancellation, and capture of malformed output. The release server build passes. Complete-service HTTP integration remains unrun: two model-service startups hit the 2 GiB free-disk guard before readiness, including after removal of this task’s rebuildable artifacts. This limit occurs before requests or retained prefix state; no HTTP or speech-latency claim is made.

Regression risk: medium because retained KV state adds memory and depends on correct independent cache copying. The snapshot is bounded, token-matched, limited to system/tool input, explicitly releasable, and covered by reference-generation comparisons.