feat(cuda): GPU-native fused filtered sampling & lazy host KV materialization

[karan68]

Summary

This PR resolves a major throughput bottleneck during non-greedy sampling on CUDA and eliminates redundant synchronous host memory copies during prompt prefill:

1. GPU-Native Fused Filtered Sampling:

   • Implemented in providing parallel on-device logit filtering (, , ) with Gumbel noise perturbation and reduction.
   • Connected directly to the decode forward pipeline in so non-greedy sampling ( > 0, ext{min_p} > 0$) executes 100% on the GPU with zero device-to-host logit transfers (saving ~512 KB PCIe round-trip per token on large vocabularies).
   • Added unit test in .

2. Lazy Host KV Materialization on GPU Prefill:

   • Removed eager synchronous D2H KV cache copies from , relying on and on-demand if a CPU fallback or prompt cache export occurs.

3. Platform Repack Safety:

   • Guarded in with to ensure Linux x86 hosts do not attempt ARM repacking.
   • Ensured execution planner explicitly disables CPU repack in when CUDA resident decode is active.

Verification & Benchmarks

• Test Suite: Ran on an NVIDIA L4 GPU (), verifying and .
• Live Serving: Verified with TinyLlama Q8_0 on NVIDIA L4 GPU handling live streaming chat completions () with temperature and min-p sampling.