Skip to content

[analytics-engine] Fix flaky rust test: concurrent_shard_warmup_does_not_corrupt#22327

Open
LantaoJin wants to merge 1 commit into
opensearch-project:mainfrom
LantaoJin:bugfix/flaky_concurrent_shard_warmup_does_not_corrupt
Open

[analytics-engine] Fix flaky rust test: concurrent_shard_warmup_does_not_corrupt#22327
LantaoJin wants to merge 1 commit into
opensearch-project:mainfrom
LantaoJin:bugfix/flaky_concurrent_shard_warmup_does_not_corrupt

Conversation

@LantaoJin

@LantaoJin LantaoJin commented Jun 26, 2026

Copy link
Copy Markdown
Member

Description

FoyerCache is configured disk-only (.memory(1) — in-memory tier is 1 byte). put_metadatainsert() returns immediately, but the actual write goes through Foyer's async storage flusher. This test fires 5 concurrent put_metadata tasks (the most write-concurrent test in the file), then issues get() immediately — so under heavy load an entry can still be queued in the write buffer when read, intermittently tripping assert!(cached.is_some()).

Fix: Added cache.wait_for_flush().await after the warmup tasks join and before the verification reads. That method exists for exactly this purpose — its documented post-condition is "all previously put() entries are on SSD and findable via get()." The production cache path is unaffected; this only makes the test's read-after-write deterministic.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@LantaoJin LantaoJin requested a review from a team as a code owner June 26, 2026 07:38
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

(Review updated until commit 265b417)

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

Copy link
Copy Markdown
Contributor

❌ Gradle check result for fae79fe: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…not_corrupt

Signed-off-by: Lantao Jin <ltjin@amazon.com>
@LantaoJin LantaoJin force-pushed the bugfix/flaky_concurrent_shard_warmup_does_not_corrupt branch from fae79fe to 265b417 Compare June 30, 2026 01:38
@github-actions

Copy link
Copy Markdown
Contributor

Persistent review updated to latest commit 265b417

@github-actions

Copy link
Copy Markdown
Contributor

✅ Gradle check result for 265b417: SUCCESS

@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.37%. Comparing base (319d78a) to head (265b417).

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #22327      +/-   ##
============================================
- Coverage     73.40%   73.37%   -0.03%     
+ Complexity    76075    76032      -43     
============================================
  Files          6076     6076              
  Lines        345500   345500              
  Branches      49732    49732              
============================================
- Hits         253608   253523      -85     
- Misses        71707    71755      +48     
- Partials      20185    20222      +37     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LantaoJin

LantaoJin commented Jun 30, 2026

Copy link
Copy Markdown
Member Author

@mch2 @sandeshkr419

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant