Fix Cascades planner treating IS NOT DISTINCT FROM as unusable for index scans - #4598
Conversation
| # Regression test for a Cascades planner bug: a keyset-pagination OR-range on trailing index | ||
| # columns, combined with a genuinely unbound "IS NOT DISTINCT FROM" parameter on the index's | ||
| # LEADING key column, could fail to plan at all (UnableToPlanException) rather than producing a | ||
| # (possibly suboptimal) plan. | ||
| # | ||
| # The scoped_item table models a common multi-tenant shape: rows are partitioned by an optional | ||
| # "scope_id" (e.g. a tenant/zone), and callers query within one scope (or "scope-wide", across | ||
| # all scopes) using a null-safe equality check ("scope_id IS NOT DISTINCT FROM row's scope"), | ||
| # because the scope value itself may be null (a "default"/unscoped row). | ||
| # | ||
| # The scoped_item_scope_ts_idx index is rooted at (scope.scope_id, ts, id) - scope_id is the | ||
| # LEADING key column, not the trailing one. A full/unbounded scan of this index is therefore | ||
| # ordered by (scope_id, ts, id), NOT by (ts, id) alone - so it can only satisfy | ||
| # "ORDER BY ts DESC, id DESC" (which never mentions scope_id) if scope_id first resolves to a | ||
| # single equality value. That resolution requires recognizing "IS NOT DISTINCT FROM" as an | ||
| # equality-shaped range boundary. Without that, neither a bounded-range plan nor the "just scan | ||
| # everything" fallback is available (the fallback's natural order doesn't match ORDER BY either) - | ||
| # hence no plan at all. | ||
| # | ||
| # Critically, the "IS NOT DISTINCT FROM" comparison must be a genuinely unbound parameter, not a | ||
| # literal: a NULL-valued bind parameter gets rewritten to a plain "IS NULL" comparison before the | ||
| # planner ever sees a "NOT_DISTINCT_FROM" node at all (that's always been a valid equality-shaped | ||
| # comparison), so it never exercises the bug. A non-null bound value stays a genuine | ||
| # NOT_DISTINCT_FROM comparison against a real parameter reference. | ||
| # | ||
| # supported_version is pinned to the current (in-development) version: this fix hasn't shipped in | ||
| # any released version yet, so running this test against an older external server build would | ||
| # correctly (from that older build's own perspective) fail - that's not a regression to catch. |
There was a problem hiding this comment.
Can you shorten this AI please?
normen662
left a comment
There was a problem hiding this comment.
The PRB seems to be failing
📊 Metrics Diff Analysis ReportSummary
ℹ️ About this analysisThis automated analysis compares query planner metrics between the base branch and this PR. It categorizes changes into:
The last category in particular may indicate planner regressions that should be investigated. New QueriesCount of new queries by file:
Plan and Metrics ChangedThese queries experienced both plan and metrics changes. This generally indicates that there was some planner change Total: 5 queries Statistical Summary (Plan and Metrics Changed)
There were no queries with significant regressions detected. Minor Changes (Plan and Metrics Changed)In addition, there were 5 queries with minor changes. |
Summary
ScanComparisons.getComparisonTypeandRangeConstraintsclassifiedNOT_DISTINCT_FROMasComparisonType.NONEinstead ofEQUALITY, so it could never be folded into an index scan's bounds — it always fell back to a residual FILTER on top of a full/covering scan.NOT_DISTINCT_FROMcomparison sits on the leading key column ofan index whose ordering doesn't otherwise match the query's ORDER BY, the planner has no way to resolve that
leading column to a bound value — and the "just scan everything" fallback's natural order doesn't satisfy the
requested order either. With no viable candidate plan, the Cascades planner throws UnableToPlanException
("Cascades planner could not plan query").
NOT_DISTINCT_FROMasComparisonType.EQUALITYinScanComparisons, and add it toRangeConstraints'sequality-range handling (Range.singleton(...))
ComparisonTypes. IS_DISTINCT_FROM(its negation) is explicitly left classified as NONE — likeNOT_EQUALS, it can't be expressed as a contiguous range, so it correctly remains a residual filter.Test
scoped-keyset-pagination.yamsql: end-to-end regression test reproducing the actual UnableToPlanException scenario — a keyset-pagination OR-range on trailing index columns combined with agenuinely unbound IS NOT DISTINCT FROM parameter on the index's leading key column. fails with UnableToPlanException pre-fix, passes with the correct union-of-covering-scans plan post-fix.