Skip to content

Sync L2 hit never backfills L1; sync L2-hit stats stringify the whole value #164

Description

@27Bslash6

Summary

Two sync-vs-async asymmetries on the L2 hit path:

  1. No L1 backfill on sync L2 hit. The async wrapper re-warms L1 on every L2 hit; the sync wrapper does not. After L1 eviction/restart, sync decorators re-pay the full L2 read + deserialize on every call until the next miss-store.
  2. Stats stringify the whole object. size_bytes = len(str(cached_result[1]).encode('utf-8')) renders the entire deserialized value (e.g. str(df) on a whole DataFrame) just for a stats number — twice, once unconditionally.

Evidence

  • Backfill: only _l1_cache.put in the sync wrapper is the miss-store at decorators/wrapper.py:797; async backfills at :1015,:1096,:1122. Sync L2-hit block :700-757 has none.
  • Stringify: decorators/wrapper.py:726 (unconditional) and :739; sync L1 already uses len(l1_bytes) correctly at :669.

Impact

Throughput cliff for sync large-object reads + a payload-proportional throwaway allocation/CPU on every sync L2 hit.

Fix

Backfill L1 on sync L2 hits (mirror async); reuse the known serialized length for stats instead of str(value).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    performancePerformance improvementspythonPython library

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions