fix(scoring): gate valid merged PRs on source token score#1397
Closed
Dexterity104 wants to merge 1 commit into
Closed
fix(scoring): gate valid merged PRs on source token score#1397Dexterity104 wants to merge 1 commit into
Dexterity104 wants to merge 1 commit into
Conversation
Collaborator
|
The eligibility gate this patches no longer exists on test. check_eligibility now counts merged PRs directly against min_valid_merged_prs and no longer compares token_score against min_token_score_for_base_score, so the SOURCE-vs-aggregate mismatch from #1396 is already gone and this diff conflicts against the current path. Closing as out of date. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The OSS scoring path measured one threshold constant against two different quantities. The base-score gate in
calculate_base_score_for_pr_fileskeyed off the SOURCE-only token score, while the eligibility "valid merged PR" gate incheck_eligibilitykeyed offpr.token_score, the all-category aggregate (SOURCE + TEST). As a result a merged PR could earn zero source base score yet still count towardmin_valid_merged_prs, the gate that unlocks all scoring for a repository. A near-threshold source PR plus a small test file was enough to flip a PR from "not valid" to "valid".This change makes validity match the base-score gate:
BaseScoreResultnow returnssource_token_score(already computed inside the helper, so no extra work).ScoredPRstoressource_token_scorealongside the aggregatetoken_score, set duringscore_pr.check_eligibilitycounts a merged PR as valid only when itssource_token_scoreclears the threshold.check_eligibilitydocstring now describe the actual behavior.The aggregate
token_scoreis still kept for reporting rollups, the storage adapter, and the issue discovery cache, so both fields remain in use. The issue discovery path is intentionally untouched: it gates on a separate constant (min_token_score_for_valid_issue) compared consistently against the aggregate, so it has no internal mismatch.Related Issues
Fixes #1396
Type of Change
Testing
Added
test_validity_gate_keys_off_source_not_aggregate_token_score, which confirms a repo of merged PRs that pass the aggregate threshold but fail the SOURCE threshold is ineligible. Updated the_mergedtest helper to set the gating field, extended the base-score helper test to assertsource_token_scoreis surfaced, and updated theBaseScoreResultstubs intest_scoring.py. The eligibility gate logic was verified directly: SOURCE-fail rows produce an ineligible result and SOURCE-pass rows produce an eligible one.Checklist