Skip to content

Build large test payloads lazily#670

Merged
eerxuan merged 3 commits into
documentdb:mainfrom
danielfrankcom:pr/lazy-payloads
Jul 9, 2026
Merged

Build large test payloads lazily#670
eerxuan merged 3 commits into
documentdb:mainfrom
danielfrankcom:pr/lazy-payloads

Conversation

@danielfrankcom

Copy link
Copy Markdown
Collaborator

This change builds large test payloads lazily to avoid GitHub runner resource limits.

Size limit and large collection tests embedded multi-megabyte strings, byte buffers, and document lists directly in their parametrized data. pytest keeps every parameter for the whole session and each xdist worker holds its own copy, so this pinned ~2.8 GiB per worker and pushed the replica-set CI job into swap space.

  • Added lazy(build), which defers construction and is typed as the built value so field types are unchanged.
  • The framework materializes lazy values where it consumes them, so each value is built while its test runs and freed after.
  • Per-worker collection memory should drop from ~2.8 GiB to ~0.5 GiB.
  • A collection-time guardrail fails any test whose parametrized data exceeds 1 MB, so there are no future regressions.

Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
@danielfrankcom danielfrankcom requested a review from a team as a code owner July 7, 2026 23:31
@danielfrankcom

Copy link
Copy Markdown
Collaborator Author

Based on the logs for this CI run and an earlier run in #645, this change seems to significantly improve the situation:

Metric This run (lazy) Before the fix
Peak host memory 10,012 / 15,988 MB (63%) 15,236 / 15,989 MB (95%)
Peak swap 0 MB (never touched swap) ~3,071 MB

@documentdb-triage-tool documentdb-triage-tool Bot added compatibility test Compatibility test related enhancement New feature or request labels Jul 7, 2026
@documentdb-triage-tool

Copy link
Copy Markdown

🤖 Auto-triaged by documentdb-triage-tool.

Applied: compatibility test, enhancement
Project fields suggested: Component test-coverage · Priority P1 · Effort XL · Status Needs Review
Confidence: 0.85 (mixed)

Reasoning

component from path globs (test-coverage, test-framework, ci); effort from diff stats (498+218 LOC, 49 files); LLM: Fixes a CI memory exhaustion issue (~2.8 GiB per worker pushing jobs into swap) by lazily building large test payloads, which is an active blocker for reliable CI execution.

If a label is wrong, remove it manually and ping @patty-chow so the rules can be tuned. The bot will not re-label items that already have component labels.

@eerxuan eerxuan merged commit 5521433 into documentdb:main Jul 9, 2026
20 checks passed
SarthakDalmia1 pushed a commit to SarthakDalmia1/functional-tests that referenced this pull request Jul 10, 2026
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
Signed-off-by: Sarthak Dalmia <sadalmia@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compatibility test Compatibility test related enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants