Context
The per-MB Schedule is currently persisted as a single SCALE-encoded blob under Key::MbSchedule(mb_hash), a full copy per computed MB. On testnet the schedule grew large enough that these copies bloat the database severely.
A hot fix was applied to stop the bleeding (see branch gsobol/ethexe/db-cleanup-hot-fix):
ComputeSubService::cleanup_computed_mb walks the parent chain after each computed head and prunes schedules deeper than SAFE_DEPTH = 100 blocks via unsafe Database::remove_mb_schedule.
DatabaseIterator::iter_mb had to start tolerating a missing schedule for a computed MB, weakening the integrity invariant that all post-execution rows exist once mb_meta.computed is set.
This is explicitly an unsafe, temporary solution: it hard-deletes consensus data by depth heuristic, makes historical replay/RPC of schedules older than SAFE_DEPTH impossible, and pokes a hole in DB integrity verification.
Proposal
Rework MB schedule storage into a merkle tree (CAS-backed, with structural sharing between consecutive MBs — same approach as program states):
- The per-MB row under
Key::MbSchedule becomes just the root hash; unchanged subtrees are shared between MBs instead of being copied wholesale.
- Old roots become garbage automatically once nothing references them, enabling safe, reference-aware GC instead of depth-based hard deletion.
- DB integrity verification can check the schedule tree by hash without requiring a full per-MB copy.
Tasks
Context
The per-MB
Scheduleis currently persisted as a single SCALE-encoded blob underKey::MbSchedule(mb_hash), a full copy per computed MB. On testnet the schedule grew large enough that these copies bloat the database severely.A hot fix was applied to stop the bleeding (see branch
gsobol/ethexe/db-cleanup-hot-fix):ComputeSubService::cleanup_computed_mbwalks the parent chain after each computed head and prunes schedules deeper thanSAFE_DEPTH = 100blocks viaunsafe Database::remove_mb_schedule.DatabaseIterator::iter_mbhad to start tolerating a missing schedule for a computed MB, weakening the integrity invariant that all post-execution rows exist oncemb_meta.computedis set.This is explicitly an unsafe, temporary solution: it hard-deletes consensus data by depth heuristic, makes historical replay/RPC of schedules older than
SAFE_DEPTHimpossible, and pokes a hole in DB integrity verification.Proposal
Rework MB schedule storage into a merkle tree (CAS-backed, with structural sharing between consecutive MBs — same approach as program states):
Key::MbSchedulebecomes just the root hash; unchanged subtrees are shared between MBs instead of being copied wholesale.Tasks
HashOf<T>plumbing)set_mb_schedule/mb_scheduleto root-hash storage + DB migrationunsafe Database::remove_mb_schedule,Database::contains_mb_schedule, andComputeSubService::cleanup_computed_mbDatabaseIterator::iter_mb(computed MB ⇒ schedule present)TODOmarkers referencing this issue