[maccoss]

Summary

• The first-pass grid search kept the strict maximum of the inner-CV passing counts, but neighboring C values usually pass within noise of each other. On 3-file Stellar, C = 0.1, 1 and 10 passed 5,025, 5,037 and 5,002 targets. The pick flips on tiny input changes, and it matters:
  • a weakly regularized C = 1 fit splits weight between correlated spectral features (apex-scan xcorr / median_polish_cosine against the multi-scan sg_weighted_cosine);
  • the first pass scores the same either way, but the second pass, which reuses the frozen first-pass model on reconciled peaks, scores much worse;
  • two Stellar libraries that differed only by 1e-4 GPU/CPU rounding gave 21,176 and 28,309 experiment precursors at the same entrapment FDP.
• grid_search_c now keeps the smallest C whose count is within PercolatorConfig::c_selection_tolerance (default 0.01) of the best, through a new svm::select_c. Tolerance 0 is the old strict maximum, with the first C in grid order winning a tie.
• In C# (osprey: Fixed first-pass SVM C selection being decided by noise ProteoWizard/pwiz#4703), the same rule:
  • took those two libraries to 30,316 and 30,485;
  • moved the regression datasets' experiment precursors by +16% (Stellar), +1% (Stellar library decoys), +6% (Stellar generated decoys with entrapment) and 0% (Astral). On the entrapment leg, the measured FDP stayed within one standard error of 1%.
• Cross-impl: this mirrors C# PercolatorTrainer.SelectC, which shipped the rule first. As with Sample one run per precursor for first-pass SVM training #66, there is no opt-out here; C# keeps OSPREY_SVM_C_TOLERANCE only for A/B work.

Companion to ProteoWizard/pwiz#4703.

Test plan

• cargo fmt --check, cargo clippy --workspace -- -D warnings, cargo test --workspace
• test_select_c_tolerance runs the same cases as C# FdrTest.TestSvmCSelectionTolerance, including the real Stellar inner-CV sweep
• test_grid_search_c_first_tie_wins still pins the strict-maximum tiebreak at tolerance 0
• End-to-end cross-impl on 3-file Stellar against C# at its default tolerance (Compare-EndToEnd-Crossimpl.ps1 -CsSvmCTolerance '', C# from the pwiz .NET 10 port branch with the matching change):
  • this branch alone: the first pass is identical (1,448,698 first-pass sidecar records at 1e-9), precursors 31,720 on both sides, blib content identical. The only remaining differences are second-pass experiment q-values and 208 extra Rust protein groups, which are what Apply the best-of-runs experiment-q floor before anything reads or persists it #68 fixes.
  • this branch with Apply the best-of-runs experiment-q floor before anything reads or persists it #68 merged locally: OVERALL PASS, bit-parity at 1e-9 end to end (Stage 7 protein FDR, blib, and all 999,212 second-pass sidecar records).
• The same on 3-file Astral, with Apply the best-of-runs experiment-q floor before anything reads or persists it #68 merged locally: OVERALL PASS, bit-parity at 1e-9 end to end (precursors 117,236 on both sides, Stage 7 protein FDR, blib, both passes' FDR sidecars)

Co-Authored-By: Claude noreply@anthropic.com

[Copilot]

Copilot review overview

🔵 Needs a closer look

Cache and sidecar validity do not account for the changed C-selection policy, allowing stale artifacts to bypass the new behavior.

Review effort: Lite
Findings: None

What changed in this PR

Updates Percolator’s SVM hyperparameter selection to prefer the strongest regularization within 1% of the best inner-CV result.

Changes:

• Adds configurable tolerance-based select_c logic.
• Integrates the 1% default into Percolator.
• Adds tests and documentation.

File	Summary
docs/​11-testing.md	Documents the new regression test.
docs/​07-fdr-control.md	Documents C-selection behavior and configuration.
crates/​osprey-ml/​src/​svm.rs	Implements tolerant C selection and tests.
crates/​osprey-fdr/​src/​percolator.rs	Adds and forwards the selection tolerance.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.