Skip to content

fix(swr): log failed and skipped background refreshes at WARNING (LAB-7456) - #446

Merged
27Bslash6 merged 3 commits into
mainfrom
lab-7456-refresh-failure-warnings
Oct 2, 2026
Merged

27Bslash6 merged 3 commits into
mainfrom
lab-7456-refresh-failure-warnings

Conversation

@27Bslash6

@27Bslash6 27Bslash6 commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Closes LAB-7456

Problem

When a background refresh raised, or never ran, cachekit logged one DEBUG line. The caller keeps the cached value until it expires and must never see a revalidation failure, so at default log levels nothing showed that refresh-ahead was broken. When a call's arguments cannot be deep-copied (a lock, an open connection), the refresh is skipped on every hit: refresh-ahead can never run for that call, and nothing said so.

Change

Every failed or skipped background value refresh now logs a WARNING that carries the function's digest, the redacted key digest and the exception type (never its message). It fires at most once per function per minute for each kind of problem, and carries the count since the last one; occurrences in between stay at DEBUG. A sample line:

SWR revalidation failed in function <redacted:275c5caba10e4535> (5 since the last warning); callers keep the cached value until it expires. Latest key <redacted:62c04945d7954e1e>: RuntimeError
Mode Failed Skipped: arguments not deep-copyable Could not start
L1-only refresh-ahead (backend=None) L1-only SWR refresh failed (async and sync) L1-only SWR refresh skipped L1-only SWR refresh could not be started (sync thread; this path logged nothing before)
Backed stale_ttl revalidation SWR revalidation failed (async and sync) SWR revalidation skipped SWR revalidation could not be scheduled

Each kind has its own window, so frequent failures cannot hide a permanent skip on the same function.

The function appears as the <redacted:…> digest of its module.qualname, not in plain text, because a function created at runtime can carry caller data in __qualname__. redact_cache_key("app.sources.fetch") gives the digest to match a line to a function. For the same reason the L1-only sync refresh thread now has a static name (cachekit-swr-refresh, like the backed cachekit-swr-revalidate). Before, it was named after __name__, and that WARNING is emitted from that thread.

The throttle is the one the Key tracking failed WARNING already used, extracted into a fork-safe _WarnThrottle that both now share. A forked child's first claim replaces a lock a dead parent thread may hold, and drops the parent's window and count (owner-PID check, as elsewhere in the wrapper).

Unchanged: a refresh_ttl_on_get TTL-extension failure still logs at DEBUG. That extension is optional, and the entry still expires on its own TTL. Refreshes skipped at slot capacity also stay quiet, because they are transient and a later hit retries them.

Tests

  • Each path above emits a WARNING with the redacted key and exception type, and no raw namespace or exception text.
  • A function whose __qualname__ and __name__ carry caller data keeps them out of both the message and the record's threadName, and the message carries redact_cache_key("<module>.<qualname>").
  • Five failures in one window produce one WARNING (1 since the last warning) and four DEBUG lines; once the window elapses, the next WARNING says 5 since the last warning.
  • A failure WARNING does not suppress a skip WARNING on the same function.
  • A forked child neither blocks on a throttle lock it inherited held nor inherits the parent's window and count. The existing key-tracking fork test still passes against the shared class.
  • The TTL-extension failure is logged at DEBUG only.
  • Mutation checks: removing the fork reset, sharing one throttle across kinds, or reverting a site to DEBUG each fail a test.
  • tests/unit (3823 passed), tests/critical, doctests, --markdown-docs, tests/docs, ruff and basedpyright all pass locally. tests/unit/test_log_redaction_architecture.py passes.

Docs

docs/configuration.md (L1-only mode and stale_ttl sections) and docs/features/l1-invalidation.md (refresh failure and its troubleshooting entry) now describe the WARNING, how to match its function digest, and the deep-copy skip. SECURITY.md records that refresh WARNINGs name the function by digest and that SWR threads have static names. The stale_ttl section no longer says a failed recompute is "silent".

…-7456)

A background refresh that raised, or never ran because the call's arguments
cannot be deep-copied or its thread or task could not start, logged one DEBUG
line. The caller keeps getting the cached value until it expires and must
never see the failure, so at default log levels nothing showed that the
refresh was broken. For arguments that cannot be copied, refresh-ahead can
never run for that call at all.

These now log a WARNING naming the function, the redacted key digest and the
exception type: at most one per function per minute for each kind (failed,
skipped, could not start), carrying the count since the last one, with DEBUG
in between. This covers L1-only refresh-ahead (async and sync) and backed
stale_ttl revalidation. The throttle is the one "Key tracking failed" already
used, extracted into a fork-safe _WarnThrottle that both now share.

A refresh_ttl_on_get TTL extension failure still logs at DEBUG: it is
optional, and the entry still expires on its own TTL.
@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 35 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: cachekit-io/cachekit-py/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 0a10bd86-529e-4408-93bc-82b35c3235e7

📥 Commits

Reviewing files that changed from the base of the PR and between e3ffdcf and b5f031f.

📒 Files selected for processing (8)
  • SECURITY.md
  • docs/configuration.md
  • docs/features/l1-invalidation.md
  • src/cachekit/decorators/wrapper.py
  • tests/unit/test_async_resource_management.py
  • tests/unit/test_key_registry.py
  • tests/unit/test_l1_only_swr.py
  • tests/unit/test_swr_decorator.py
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kodus-27b

kodus-27b Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

  • Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
  • Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
  • Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
  • Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
  • API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
  • Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
  • Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
  • Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
  • Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
  • Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).
Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

kodus-27b[bot]
kodus-27b Bot previously approved these changes Oct 2, 2026
@codecov

codecov Bot commented Oct 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.55556% with 2 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/cachekit/decorators/wrapper.py 95.55% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

A function created at runtime can carry caller data in __qualname__ or
__name__. The refresh WARNING printed the qualname verbatim. The L1-only
sync refresh thread was also named after __name__, and that WARNING is
emitted from that thread, so a log format with %(threadName)s leaked the
name too.

The WARNING now names the function by the <redacted:...> digest of its
module.qualname: the same correlation id the key uses, and computable with
redact_cache_key() to match a line to a function. The L1-only refresh
thread gets a static name, like the backed revalidation thread.
@kodus-27b

kodus-27b Bot commented Oct 2, 2026

Copy link
Copy Markdown

Code Review Completed! 🔥

The code review was successfully completed based on your current configurations.

Kody Guide: Usage and Configuration
Interacting with Kody
  • Request a Review: Ask Kody to review your PR manually by adding a comment with the `@kody start-review` command at the root of your PR.

  • Provide Feedback: Help Kody learn and improve by reacting to its comments with a 👍 for helpful suggestions or a 👎 if improvements are needed.

Providing Context (Files & MCPs)

Add these hints in your PR description (or a comment) to unlock deeper checks:

  • Ticket / Acceptance Criteria: `Refs: ABC-123` (Linear/Jira/Asana/ClickUp/Trello) or a direct ticket link.
  • Bugfix Validation: a Sentry/Datadog/Bugsnag event link (or paste the stack trace/error message).
  • Endpoint Risk: mention the route (e.g., `POST /api/payments`) or controller/action name.
  • Attach a repo file as context: use an explicit marker like `@file:docs/guide.mdx#L10-L50` (replace with your real path).
  • API Contract Docs: include `@file:openapi.yaml` or `@file:swagger.json` when changing routes/schemas.
  • Definition of Done / Standards: include `@file:DOD.md` or `@file:CONTRIBUTING.md` if your repo has them.
  • Design System Source of Truth: include `@file:ui/index.ts` (replace with your DS entrypoint path).
  • Feature Flags: include the flag key/name and `@file:flags.ts` / `@file:config.json` (and optionally the PostHog flag name).
  • Edge/CDN Rules: link the Cloudflare rule/zone or describe the intended redirect/header behavior.
  • Attach an MCP tool output: use `@mcp<provider|tool>` (replace with an installed MCP provider + tool, e.g., `@mcp<sentry|events.search>`).
Current Kody Configuration
Review Options

The following review options are enabled or disabled:

Options Enabled
Bug ✅
Performance ✅
Security ✅
Business Logic ✅

Access your configuration settings here.

@27Bslash6
27Bslash6 merged commit 1ba3312 into main Oct 2, 2026
36 checks passed
@27Bslash6
27Bslash6 deleted the lab-7456-refresh-failure-warnings branch October 2, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant