Skip to content

perf(task-board): delete PR cache keys concurrently on invalidate - #6981

Open
pedrofrxncx wants to merge 1 commit into
mainfrom
perf/pr-cache-invalidate-concurrent-w3
Open

perf(task-board): delete PR cache keys concurrently on invalidate#6981
pedrofrxncx wants to merge 1 commit into
mainfrom
perf/pr-cache-invalidate-concurrent-w3

Conversation

@pedrofrxncx

@pedrofrxncx pedrofrxncx commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Source: reduction/optimization found while hunting the mcp-tool-latency-profiling focus area (relates to #2995 "DB latency amplifies significantly across MCP tool calls") — read pr-cache.ts (added by #6966, the JetStream KV cache fronting task-board PR reads/cards).

The problem: JetStreamKVPrCache.invalidate(namespace) lists every KV key under a namespace, then deletes them one at a time in a for await loop — each kv.delete() is its own NATS round-trip, awaited sequentially. invalidatePrCards(organizationId) and invalidatePrReadsForConnection(connectionId) are both awaited synchronously from task-board tool handlers (e.g. TASK_BOARD_ITEM_PR_LINK, merge-pr) right before they return. An org or connection with many cached PR entries pays N sequential round-trips inline on that tool call — the busier the board, the slower every PR-link/merge call gets, which is exactly the tool-call latency amplification pattern #2995 describes.

The fix: collect the matched keys, then Promise.all the deletes. They're independent, unrelated keys with no ordering requirement, and the existing per-key .catch(() => {}) (best-effort, TTL is the backstop) is preserved — behavior is unchanged, just concurrent instead of serial.

Verify: cd apps/api && bun test src/tools/task-board/pr-cache.test.ts — added a test asserting invalidate still drops every key under a namespace (not just the first), which would have caught an incorrect early-return/short-circuit in a naive rewrite.

Checks run locally: bun run fmt, bunx tsc --noEmit (apps/api), the targeted test above, and bunx oxlint on both changed files — all clean. Full CI runs the rest.


Summary by cubic

Speeds up task-board PR cache invalidation by deleting matched keys concurrently instead of one at a time, cutting the serial NATS round-trips that slow PR-link and merge tool calls on busy boards (the latency pattern in #2995). Behavior is unchanged — deletes are independent and best-effort with TTL as the backstop.

  • Switches invalidate() from a sequential for await loop to Promise.all over collected keys.
  • Adds a test asserting all keys under a namespace are dropped, not just the first.

Written for commit 254a341. Summary will update on new commits.

Review in cubic

invalidate() collected each namespace's KV keys then awaited kv.delete() one at a time in a for-await loop. invalidatePrCards()/invalidatePrReadsForConnection() are awaited synchronously from PR link/merge tool handlers, so an org (or connection) with many cached PR entries pays one full NATS round-trip per key, serially, before the tool call returns.

Deletes target independent, unrelated keys, so there's no ordering requirement — run them concurrently with Promise.all instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant