Skip to content

fix(compact): preserve base-tier adjacency when source node is dirty - #346

Open
temporaryfix wants to merge 1 commit into
GrafeoDB:mainfrom
temporaryfix:fix/overlay-edge-property-lookup
Open

fix(compact): preserve base-tier adjacency when source node is dirty#346
temporaryfix wants to merge 1 commit into
GrafeoDB:mainfrom
temporaryfix:fix/overlay-edge-property-lookup

Conversation

@temporaryfix

@temporaryfix temporaryfix commented May 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #345.

Summary

LayeredStore::edges_from and neighbors short-circuited the base layer whenever the source node was marked dirty. ensure_in_overlay (the only path that flags a node dirty in this context) copies labels and properties into the overlay but never copies adjacency, so a snapshot-tier edge whose endpoint was later touched by any overlay write — a property update, or merely being the endpoint of an unrelated new edge — silently vanished from reads.

In GQL this surfaces as property-anchored edge patterns such as MATCH (a {id: $x})-[:T]->(b {id: $y}) returning zero rows once any overlay write has occurred, because the planner may walk from either anchor and one of them is now dirty.

See #345 for the full root-cause analysis, minimal reproduction, and observed impact.

Fix

Drop the is_node_dirty guard in both methods. Safety of the change:

  • Per-edge deletions are still authoritative via deleted_from_base_edges.
  • Promoted edges (whose properties were modified, so they live at the same EdgeId in base and overlay) are collapsed by the existing dedup_by_key(eid) pass at the end of edges_from.
  • delete_node_edges walks base.load().edges_from(...) directly and was never affected by the guard.
  • As a side benefit, sum_of(edges_from(n).len()) is now consistent with edge_count() for dirty source nodes (it wasn't before).

The minor cost: traversals from a dirty source node now read both base and overlay adjacency rather than just overlay. That's the necessary cost of returning the correct answer.

Regression coverage

  • New test_base_edge_visible_from_promoted_endpoint — promotes both endpoints via unrelated overlay writes and asserts the original base edge remains reachable from both directions, via both edges_from and neighbors.
  • New post_compact_property_anchored_edge_survives_unrelated_overlay_write — GQL integration test that mirrors the production repro for both promotion directions.
  • Strengthened test_edges_from_dirty_source_merges_layers and test_neighbors_from_promoted_node_with_new_overlay_edges — the existing tests asserted only the new overlay edge and explicitly documented the bug in their comments. They now assert the merge behavior their names promise.

All four new/strengthened tests fail on the pre-fix code (confirmed by running them against main before applying the fix).

Test plan

  • cargo test -p grafeo-core --features "compact-store lpg" — 2017 + 27 + 5 + 61 tests pass.
  • cargo test -p grafeo-engine --features "compact-store lpg gql" --test post_compact_overlay_visibility — 6/6 pass (5 existing + 1 new).
  • cargo test -p grafeo-engine --features "compact-store lpg gql" across post_compact_index, post_compact_numeric_content, compact_store_integration, compact_roundtrip_proptest, compact_store_tiered_spill, snapshot, database_api, backward_compatibility, mvcc_integration, versioning_coverage, temporal_properties, savepoint_undo, label_rollback, set_property_rollback, grafeo_file, golden_format, schema_wal_replay, wal_recovery — all pass.
  • Pre-existing compile errors in query_correctness.rs and merge_rollback.rs (both call removed/renamed methods execute_sql/execute_cypher) are reproducible on clean main and are unrelated to this change.

🤖 Generated with Claude Code


Summary by cubic

Preserves base-tier adjacency when a node becomes dirty so existing snapshot edges remain visible. Restores correct results for property-anchored edge queries in GQL after overlay writes to either endpoint.

  • Bug Fixes
    • Always read base adjacency in neighbors and edges_from, even when the source node is dirty; merge with overlay, respect per-edge deletions, and dedup by EdgeId.
    • Fixes double-anchored patterns like MATCH (a {id: ...})-[:T]->(b {id: ...}) returning zero rows after unrelated overlay writes.
    • Adds regression coverage (unit + GQL tests) for promoted endpoints in both directions.
    • Minor trade-off: traversals from dirty nodes now read both base and overlay.

Written for commit 70c9b92. Summary will update on new commits.

`LayeredStore::edges_from` and `neighbors` short-circuited the base
layer whenever the source node was marked dirty. `ensure_in_overlay`
(the only path that flags a node dirty in this context) copies labels
and properties into the overlay but never copies adjacency, so a
snapshot-tier edge whose endpoint was later touched by any overlay
write — a property update, or merely being the endpoint of an
unrelated new edge — silently vanished from reads.

In GQL this surfaces as property-anchored edge patterns such as
`MATCH (a {id: $x})-[:T]->(b {id: $y})` returning zero rows once any
overlay write has occurred, because the planner may walk from either
anchor and one of them is now dirty.

Drop the `is_node_dirty` guard in both methods. Per-edge deletions
remain authoritative via `deleted_from_base_edges`, and promoted edges
(those whose properties were modified, so they live at the same
`EdgeId` in base and overlay) are still collapsed by the existing
`dedup_by_key(eid)` pass at the end of `edges_from`.

Regression coverage:
  * `test_base_edge_visible_from_promoted_endpoint` — new unit test
    that promotes both endpoints via unrelated overlay writes and
    asserts the original base edge is reachable from both directions.
  * `post_compact_property_anchored_edge_survives_unrelated_overlay_write`
    — new GQL integration test mirroring the production repro.
  * `test_edges_from_dirty_source_merges_layers` and
    `test_neighbors_from_promoted_node_with_new_overlay_edges` —
    strengthened to assert that base-tier adjacency survives
    promotion, not just that the new overlay edge is visible.

All four new/strengthened tests fail on the pre-fix code.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@temporaryfix
temporaryfix requested a review from StevenBtw as a code owner May 15, 2026 13:30

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

@codecov

codecov Bot commented May 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@temporaryfix

Copy link
Copy Markdown
Contributor Author

Would you like me to target the release branch once you have triaged this?

temporaryfix added a commit to temporaryfix/grafeo that referenced this pull request May 15, 2026
@codspeed-hq

codspeed-hq Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 70 untouched benchmarks


Comparing temporaryfix:fix/overlay-edge-property-lookup (70c9b92) with main (4ebae02)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Property-anchored edge lookups drop snapshot-tier edges after any overlay write

1 participant