Skip to content

Support an in-call LIMIT clause for ARRAY_AGG() - #4600

Merged
robert-brunel merged 4 commits into
mainfrom
apple/robert-brunel/array_agg-limit-2
Sep 11, 2026
Merged

Support an in-call LIMIT clause for ARRAY_AGG()#4600
robert-brunel merged 4 commits into
mainfrom
apple/robert-brunel/array_agg-limit-2

Conversation

@robert-brunel

Copy link
Copy Markdown
Contributor

ARRAY_AGG(«expr» LIMIT «n») restricts the number of elements collected into the array per group. Rows past the limit are neither converted nor collected by the accumulator, although they still have to be consumed in order to determine the group boundary. The LIMIT clause must come last, after the {RESPECT|IGNORE} NULLS and in-call ORDER BY clauses.

LIMIT can be used as a guard to impose a bound on both the arrays held in memory and the partial state serialized into a continuation. In conjunction with an in-call ORDER BY clause (which is anticipated by the grammar but not yet supported), LIMIT is also a convenient way to produce “top N” lists.

Regarding the grammar, note that the new aggregateLimitClause (which is admitted for ARRAY_AGG() only) is deliberately narrower than the existing (but unsupported) limitClause of a SELECT query: It takes no OFFSET, and «n» must be a literal, since the limit gets baked into the plan rather than bound per execution.

Resolves #4599.

@robert-brunel robert-brunel self-assigned this Sep 9, 2026
@robert-brunel
robert-brunel requested a review from hatyo September 9, 2026 14:21
@robert-brunel robert-brunel added the enhancement New feature or request label Sep 9, 2026
@robert-brunel
robert-brunel added this pull request to stack #4601 September 9, 2026 14:22
@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/array_agg-limit-2 branch from 3d1e277 to 3314ce2 Compare September 9, 2026 14:42
Comment thread yaml-tests/src/test/resources/array-agg-tests.yamsql
@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/array_agg-limit-2 branch from 3314ce2 to f99e800 Compare September 10, 2026 13:55
Base automatically changed from apple/robert-brunel/array_agg-limit-1 to main September 10, 2026 16:41
@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/array_agg-limit-2 branch from f99e800 to e4d5660 Compare September 10, 2026 16:45

@pengpeng-lu pengpeng-lu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@hatyo hatyo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

`ARRAY_AGG(«expr» LIMIT «n»)` restricts the number of elements collected into the array per group. Rows past the limit are neither converted nor collected by the accumulator, although they still have to be consumed in order to determine the group boundary. The `LIMIT` clause must come last, after the `{RESPECT|IGNORE} NULLS` and in-call `ORDER BY` clauses.

`LIMIT` can be used as a guard to impose a bound on both the arrays held in memory and the partial state serialized into a continuation. In conjunction with an in-call `ORDER BY` clause (which is anticipated by the grammar but not yet supported), `LIMIT` is also a convenient way to produce “top N” lists.

Regarding the grammar, note that the new `aggregateLimitClause` (which is admitted for `ARRAY_AGG()` only) is deliberately narrower than the existing (but unsupported) `limitClause` of a `SELECT` query: It takes no `OFFSET`, and `«n»` must be a literal, since the limit gets baked into the plan rather than bound per execution.

Resolves #4599.
@robert-brunel
robert-brunel force-pushed the apple/robert-brunel/array_agg-limit-2 branch from e4d5660 to a8eeefd Compare September 11, 2026 12:04
@github-actions

Copy link
Copy Markdown

📊 Metrics Diff Analysis Report

Summary

  • New queries: 1
  • Dropped queries: 0
  • Plan changed + metrics changed: 0
  • Plan unchanged + metrics changed: 0
ℹ️ About this analysis

This automated analysis compares query planner metrics between the base branch and this PR. It categorizes changes into:

  • New queries: Queries added in this PR
  • Dropped queries: Queries removed in this PR. These should be reviewed to ensure we are not losing coverage.
  • Plan changed + metrics changed: The query plan has changed along with planner metrics.
  • Metrics only changed: Same plan but different metrics

The last category in particular may indicate planner regressions that should be investigated.

New Queries

Count of new queries by file:

  • yaml-tests/src/test/resources/array-agg-tests.metrics.yaml: 1

@robert-brunel robert-brunel added the Run mixed-mode Label to add to Pull Requests to have it run mixed mode tests label Sep 11, 2026
@robert-brunel
robert-brunel merged commit 4c41cce into main Sep 11, 2026
28 checks passed
@robert-brunel
robert-brunel deleted the apple/robert-brunel/array_agg-limit-2 branch September 11, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Run mixed-mode Label to add to Pull Requests to have it run mixed mode tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support an in-call LIMIT clause for ARRAY_AGG()

3 participants