Skip to content

json2junit: preserve repeated test runs - #697

Merged
George Adams (gdams) merged 3 commits into
mainfrom
dev/gadams/json2junit-duplicate-tests
Nov 16, 2026
Merged

George Adams (gdams) merged 3 commits into
mainfrom
dev/gadams/json2junit-duplicate-tests

Conversation

@gdams

Copy link
Copy Markdown
Member

Summary

Fix the duplicate run entry for package crypto/rand test TestAllocations conversion error exposed by microsoft/go#2727.

Go permits internal and external test packages to define tests with the same name, but emits the same package and test names for both in go test -json. Repeated runs with -count can produce the same sequence. The converter previously rejected the second run even when the first had already finished.

  • Keep completed test cases separately from active runs, preserving each run's outcome, duration, and failure/skip output.
  • Give subsequent runs unique JUnit names (TestAllocations [2], etc.), including when package prefixes are enabled or a generated name would otherwise collide.
  • Keep rejecting overlapping duplicate runs: their JSON events do not identify which instance produced each event.
  • Document the behavior and add regression coverage. Existing golden outputs remain unchanged.

This fixes test-result conversion only; the separate system-crypto allocation failure in microsoft/go#2727 still needs its own fix.

Validation

  • Confirmed the regression test failed with the original duplicate-run error before changing the converter.
  • go test -race ./json2junit ./cmd/json2junit -count=1
  • go vet ./json2junit ./cmd/json2junit
  • Built ./cmd/json2junit and converted real go test -json -count=2 output from internal/external tests with identical names; verified all four runs became distinct JUnit test cases.
  • git diff --check

Allow completed test names to run again within a package, as happens with internal and external tests and go test -count. Preserve each result and assign unique JUnit names while retaining errors for overlapping duplicate runs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@gdams
George Adams (gdams) requested a review from a team as a code owner November 16, 2026 22:11
Copilot AI balanced review requested due to automatic review settings November 16, 2026 22:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Restarting the suffix search for every repeated test makes conversion quadratic, creating excessive overhead for large go test -count streams.

1 open issue
What changed in this PR

This PR updates json2junit to preserve sequential runs with identical test names, addressing conversion failures from internal/external test packages and go test -count. Overlapping duplicate runs remain rejected.

Changes:

  • Retain each run’s outcome, duration, and failure or skip output separately.
  • Assign unique JUnit names, accounting for package prefixes and naming collisions.
  • Document repeated-run support and add regression coverage.
File Description
json2junit/​testdata/​TestConverter/​repeatednames/​repeatednames.xml Expected JUnit output for repeated names and collisions.
json2junit/​testdata/​inputs/​good/​repeatednames.jsonl Input fixture exercising repeated names and suffix collisions.
json2junit/​json2junit.go Separate active-run tracking from retained cases and assign unique names.
json2junit/​json2junit_test.go Verify repeated outcomes, durations, output, and package-prefixed names.

Review effort: Balanced

Comment thread json2junit/json2junit.go Outdated
Comment on lines +246 to +248
for n := 2; suite.testNames[name]; n++ {
name = fmt.Sprintf("%s [%d]", baseName, n)
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in a8c7f2f. The converter now remembers the next suffix per base JUnit name while retaining the used-name check for literal-name collisions. Added collision tests with and without package prefixes and benchmarks at 1,000 and 4,000 repeated runs. On a local one-iteration benchmark, 4,000 runs dropped from about 712 ms / 381 MB allocated to 8.6 ms / 3.8 MB allocated. Race-enabled converter tests and go vet pass.

Remember the next suffix for each base JUnit name instead of rescanning all previous runs. Keep used-name checks for collisions with literal test names, and cover collisions and repeated-run scaling with tests and benchmarks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@gdams George Adams (gdams) added the release-on-merge Triggers an automated release when this PR is merged label Nov 16, 2026
@gdams
George Adams (gdams) enabled auto-merge (squash) November 16, 2026 23:33
Comment thread json2junit/json2junit.go Outdated

testCases map[string]*junitTestCase `xml:"-"` // Map from test name to *junitTestCase.
testCases map[string]*junitTestCase `xml:"-"` // Active test cases, keyed by Go test name.
testNames map[string]bool `xml:"-"` // JUnit names already used in this suite.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Misuse of map[string]bool vs. map[string]struct{}?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Changed testNames to map[string]struct{} since this is only a membership set. Verified with go test -race ./json2junit ./cmd/json2junit -count=1 and go vet ./json2junit ./cmd/json2junit; pushed in a362ac8.

Comment thread json2junit/json2junit.go Resolved
Use struct{} for the used-name set, which only needs membership checks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@gdams
George Adams (gdams) merged commit 97443a0 into main Nov 16, 2026
15 checks passed
@gdams
George Adams (gdams) deleted the dev/gadams/json2junit-duplicate-tests branch November 16, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-on-merge Triggers an automated release when this PR is merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants