fix: absolute TTL, coarse ttl resolution, and jitter for caches

[ferhatelmas]

What kind of change does this PR introduce?

Bug fix

What is the current behavior?

Caches with long ttl have very low ttl resolution (1h vs 1ms). This creates timer arms with update age on get with hot keys. Additionally, update age on get isn't safe because staleness is unbounded for example from a dropped notification.

What is the new behavior?

Increase ttl resolution to 30s but make it overridable.
Drop update age on get, there will slightly more backend checks but with single flight, it's protected.
Add jitter so that it doesn't cause refresh storm.

Additional context

Related to #1316
After this, tenant pools can be a LRU cache only capped with capacity, no TTL and close cache improvements.

[Copilot]

Pull request overview

This PR adjusts internal caching behavior to avoid “sliding TTL” staleness for hot keys by moving to absolute TTL semantics, improving TTL clock resolution defaults, and adding TTL jitter to reduce coordinated refresh storms across tenants/caches.

Changes:

• Add default TTL clock resolution (30s) for LRU caches that specify a TTL, while still allowing explicit overrides.
• Introduce per-set TTL jitter (default ratio 0.1) in the LRU cache wrapper and apply it to key tenant-facing caches (tenant config, JWKS, S3 credentials, pgvector metric cache).
• Update/add tests to validate absolute TTL behavior and jitter behavior (including a tenant hot-read regression test).

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/internal/cache/lru.ts	Adds default TTL resolution logic and per-set TTL jitter support in the LRU wrapper.
src/internal/cache/lru.test.ts	Adds unit coverage for default TTL resolution, absolute TTL behavior, and jitter validation.
src/internal/cache/monitoring.test.ts	Adjusts tests to explicitly set ttlResolution: 0 where precise expiry timing is needed.
src/internal/database/tenant.ts	Switches tenant config cache to absolute TTL semantics and applies default jitter ratio.
src/internal/auth/jwks/manager.ts	Switches JWKS cache to absolute TTL semantics and applies default jitter ratio.
src/storage/protocols/s3/credentials/manager.ts	Switches S3 credentials cache to absolute TTL semantics and applies default jitter ratio.
src/storage/protocols/vector/adapter/pgvector/metric-cache.ts	Adds jitter configuration and removes hot-read TTL renewal behavior.
src/storage/protocols/vector/adapter/pgvector/metric-cache.test.ts	Updates/extends tests for absolute TTL and jitter behavior.
src/test/tenant.test.ts	Adds a regression test ensuring hot reads don’t renew tenant config TTL.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[coveralls]

Coverage Report for CI Build 31680233008

Coverage increased (+19.5%) to 80.957%

Details

• Coverage increased (+19.5%) from the base build.
• Patch coverage: 15 of 15 lines across 1 file are fully covered (100%).
• 18 coverage regressions across 2 files.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

18 previously-covered lines in 2 files lost coverage.

File	Lines Losing Coverage	Coverage
src/internal/http/agent.ts	17	32.56%
src/http/routes/s3/index.ts	1	86.09%

---

Coverage Stats

Relevant Lines:	12891
Covered Lines:	10854
Line Coverage:	84.2%
Relevant Branches:	7683
Covered Branches:	5802
Branch Coverage:	75.52%
Branches in Coverage %:	Yes
Coverage Strength:	522.94 hits per line

---

💛 - Coveralls

[ferhatelmas]

@claude review always

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (2)

src/storage/protocols/vector/adapter/pgvector/metric-cache.test.ts:43

• Math.random() never returns 1 (range is [0, 1)), so mocking it to 1 tests an impossible edge case. Using a value just under 1 keeps the same intent (max jitter) while matching real behavior.

    random.mockReturnValue(1)
    cache.set('full-jitter', 'euclidean')
    random.mockReturnValue(0)
    cache.set('no-jitter', 'cosine')

src/internal/cache/lru.test.ts:186

• Math.random() never returns 1 (range is [0, 1)), so mocking it to 1 tests an impossible edge case. Use a value just under 1 to simulate worst-case jitter while staying within the real domain.

    random.mockReturnValue(1)
    cache.set('full-jitter', { bytes: 1 })
    random.mockReturnValue(0)
    cache.set('no-jitter', { bytes: 1 })