Skip to content

feat(server): add transactional managed cache lineage - #1414

Open
Changyong Kwak (samkwak188) wants to merge 17 commits into
qualcomm:mainfrom
samkwak188:feat/managed-cache-lineage
Open

feat(server): add transactional managed cache lineage#1414
Changyong Kwak (samkwak188) wants to merge 17 commits into
qualcomm:mainfrom
samkwak188:feat/managed-cache-lineage

Conversation

@samkwak188

@samkwak188 Changyong Kwak (samkwak188) commented Aug 31, 2026

Copy link
Copy Markdown

Summary

Adds an opt-in, transaction-safe managed cache protocol to the OpenAI-compatible chat server and reconciles it with current Qualcomm main.

  • Ordinary requests retain Qualcomm's automatic exact-extension behavior.
  • Managed requests bind reuse to a random session, parent revision, exact transcript extension, resolved model identity, reusable prior completion, and loaded-model generation.
  • Automatic, managed, and raw legacy modes invalidate one another explicitly. The added automatic-session validity bit forces a cold reset after another mode mutates the shared handle.
  • Only EOS-complete managed generations are reusable. Length, stop-sequence, callback, and unknown stops commit the logical revision as non-reusable, reset physical state immediately, and make the next exact extension report reset / previous_not_reusable.
  • Cancellation, disconnect, generation failure, branch, session switch, stale parent, completion traffic, and logits traffic abort or invalidate managed state before later reuse.
  • The final blocking response or terminal streaming chunk returns protocol header GenieX-Cache-Protocol: 2 and the five-field cache record.

Protocol version 2 accepts scalar text messages with system, user, and assistant roles. It rejects managed VLM input, native tool messages, separated reasoning, speculative decoding, and system-only warm-up before generation. Session IDs are lineage labels, not authentication, authorization, or tenant isolation.

QAIRT dependency

Managed reuse on QAIRT requires qualcomm/geniex-qairt-plugin#50. That correction renders the complete transcript, reconciles the exact token prefix against retained KV, rewinds divergent suffixes, rebuilds sampler history, and retains terminal EOG in KV without returning it as visible output.

The current gitlink pins b3b85ab16c5106a7048ca2f24a3c9e4a8089cba7. The temporary fork URL is required only until PR 50 lands; it must be changed back to the Qualcomm submodule URL before this PR merges.

Upstream reconciliation

Current PR head d9c2c2fd7772a662614c9b9ae1be6a23add8de65 contains Qualcomm main commit 2929a65788da939e2a02d5652624b2ff50a24adb. It preserves upstream automatic reset, VLM media replay, current native tool-call structures, and the existing single-handle request serialization.

A security review found and fixed three documentation mismatches: ordinary mode is automatic rather than unconditional reset, the wire protocol is version 2, and the identity hashes model-manager-resolved model/tokenizer paths rather than every runtime-specific bundle file.

Verification

  • DCO passes at the current PR head
  • QAIRT clean Windows ARM64 build: 517 of 517 targets
  • QAIRT CTest: 223 of 223 passed
  • All local GenieX Go tests passed
  • go vet ./cli/server/... passed
  • Fork PR Check passed all 26 jobs at code commit 6af9fee645de67dda649e487785d633b97b98ab4: https://github.com/samkwak188/GenieX/actions/runs/33668959554
  • Linux ARM64 managed-handler and keep-alive go test -race gate passed
  • Windows ARM64 CLI, SDK, Go, Python, and SDK integration jobs passed
  • Exact-range two-reviewer security scan covered all 18 changed source files with zero reportable findings
  • Windows ARM64 QAIRT protocol-2 smoke exercised first request, exact extension, branch, session switch, stale parent, non-reusable length completion, recovery, and client disconnect
  • Paired production-path task gate: 30 completed tasks, 18 managed hits, 18 hits with lower prompt-token work, 30/30 exact-marker successes under managed mode versus 19/30 under physical reset, zero managed regressions, and zero uncontrolled same-outcome differences

The artifact CI ran at 6af9fee6; the current PR tree adds only the reviewed documentation alignment after that code commit. The on-device run used Qualcomm Qwen 3 0.6B as a protocol smoke model. It does not claim a latency improvement or substitute for the planned Llama 3.1 8B study.

Remaining merge gate

  • Merge QAIRT PR 50, return .gitmodules to the Qualcomm URL, and advance the gitlink to the merged upstream commit
  • Run upstream-owned CI after that final dependency update

Closes #1413

Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Preserve transactional cache commit and abort semantics while adopting upstream multi-tool parsing, reasoning-aware streaming, and current SDK changes. Update compatibility tests and remove the stale single-tool documentation claim.
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Signed-off-by: Sam Kwak <samisme2005@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add transaction-safe lineage validation for retained chat caches

1 participant