Skip to content

Add vectorSearch stage tests#649

Open
danielfrankcom wants to merge 4 commits into
documentdb:mainfrom
danielfrankcom:pr/vectorSearch
Open

Add vectorSearch stage tests#649
danielfrankcom wants to merge 4 commits into
documentdb:mainfrom
danielfrankcom:pr/vectorSearch

Conversation

@danielfrankcom

Copy link
Copy Markdown
Collaborator

This change adds tests for the vectorSearch aggregation stage.

The mongot sidecar change here overlaps with #638 and #645. All 3 contain the same commit that should merge cleanly regardless of which is merged first.

I defined a number of pytest fixtures to define the necessary collections for these tests, as an index of this type is far more expensive to set up than the ones we use elsewhere (on the order of 30-60 secs). Using a module/package scoped fixture allows us to amortize this cost across the test cases rather than taking the hit each time.

Closes #376

Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
@danielfrankcom danielfrankcom requested a review from a team as a code owner June 26, 2026 23:01
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
Signed-off-by: Daniel Frankcom <frankcom@amazon.com>
@documentdb-triage-tool

Copy link
Copy Markdown

🤖 Auto-triaged by documentdb-triage-tool.

Applied: compatibility test, enhancement
Project fields suggested: Component test-coverage · Priority P2 · Effort XL · Status Needs Review
Confidence: 0.30 (deterministic)

Reasoning

component from path globs (test-coverage, test-framework); effort from diff stats (5982+16 LOC, 33 files); LLM failed: Invalid response body while trying to fetch https://api.anthropic.com/v1/messages: Premature close

If a label is wrong, remove it manually and ping @patty-chow so the rules can be tuned. The bot will not re-label items that already have component labels.

@documentdb-triage-tool documentdb-triage-tool Bot added compatibility test Compatibility test related enhancement New feature or request labels Jun 26, 2026
@ramgokul-14

Copy link
Copy Markdown
  • Question - Pervasive error_code=UNKNOWN_ERROR. 100+ error cases across ~14 files assert the generic sentinel instead of a specific code. This is a compatibility suite; if DocumentDB returns a different code than MongoDB for the same bad input, UNKNOWN_ERROR masks the divergence (REVIEW.md §8, TEST_COVERAGE §6). The PR does use specific codes where it can (VECTOR_SEARCH_LIMIT_NOT_NUMBER_ERROR, EXPRESSION_NOT_OBJECT_ERROR, NOT_FIRST_STAGE_ERROR, etc.), and the comments assert mongot collapses many errors into one opaque code — which may make this defensible. Required action: confirm whether UNKNOWN_ERROR matches a real stable numeric code or means "any error," and add a one-line rationale comment where the executor genuinely can't differentiate. Where a real code exists, capture it in error_codes.py.

  • Index-READY polling loop copy-pasted 5–6× (conftest.py, test_smoke_vectorSearch.py, test_stages_position_vectorSearch.py, test_vectorSearch_nested.py, test_vectorSearch_stored_source.py). Extract to a wait_for_search_index_ready(coll) helper in utils/vectorSearch_common.py (REVIEW.md §6). One layer of abstraction is explicitly allowed by TEST_FORMAT.

  • Missing documentation updates (REVIEW.md §9). New feature category + new dev-environment dependency: TEST_COVERAGE.md has no rules for search-stage patterns (index polling, mongot dependency, search error surfaces); CONTRIBUTING.md/README.md don't mention the new mongot sidecar that mongo-replset now pulls; FOLDER_STRUCTURE feature tree isn't updated for stages/vectorSearch/.

  • searchSyncUser created, never dropped (engine_registry.py). This is infrastructure provisioning (idempotent, once per target — not per-test), so it's defensible, but REVIEW.md §7 calls out user cleanup explicitly

  • Fixed dev credential duplicated in three files (dev/compose.yaml, dev/mongot.yml, engine_registry.py). Not a security issue (documented local-dev secret, umask 077), but a drift risk — add cross-reference comments (mongot.yml lacks one).

  • nit: test_smoke_vectorSearch.py runs a function-scoped 120s index build for a "smoke" test and lacks a # Property [...] comment; consider reusing the package fixture.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compatibility test Compatibility test related enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add compatibility test for $vectorSearch

2 participants