Skip to content

Fix CanReadArrayOfAnySize timeout on checked coreclr by skipping the 2 GB case on non-release runtimes#126006

Open
Copilot wants to merge 7 commits intomainfrom
copilot/fix-timeout-edge-case-tests
Open

Fix CanReadArrayOfAnySize timeout on checked coreclr by skipping the 2 GB case on non-release runtimes#126006
Copilot wants to merge 7 commits intomainfrom
copilot/fix-timeout-edge-case-tests

Conversation

Copy link
Contributor

Copilot AI commented Mar 24, 2026

EdgeCaseTests.CanReadArrayOfAnySize repeatedly times out (~14+ min) on the checked coreclr windows x64 Release CI leg. The 2 GB array case (Array.MaxLength = 2,147,483,591) was previously gated by #if RELEASE && NET, enabling it for Release library builds. However, checked coreclr's extra assertions/validation make processing a 2 GB array prohibitively slow.

Description

Instead of skipping all test cases on non-release runtimes, the fix adds a targeted in-method skip guard for only the 2 GB case. The #if RELEASE && NET preprocessor guard and [ConditionalTheory] with platform conditions are replaced with [ConditionalTheory] (no parameters, to allow SkipTestException to be treated as a skip rather than a failure) and a runtime check inside the method body:

[ConditionalTheory]
[InlineData(100)]
[InlineData(64_001)]
[InlineData(127_000)]
[InlineData(2147483591)] // Array.MaxLength
public void CanReadArrayOfAnySize(int length)
{
    if (length == 2147483591 && (!PlatformDetection.Is64BitProcess || !PlatformDetection.IsReleaseRuntime || !PlatformDetection.IsNetCore))
    {
        throw new SkipTestException("It would take too much time to execute.");
    }
    // ...
}

The smaller cases (100, 64_001, 127_000) continue to run on all platforms and runtime configurations. The 2 GB case is skipped only when:

  • The process is not 64-bit, or
  • The runtime is not a release runtime (!IsReleaseRuntime, which covers both checked and debug coreclr of any bitness), or
  • The runtime is not .NET Core (i.e. running on .NET Framework)

!PlatformDetection.IsReleaseRuntime is used instead of SlowRuntimeTimeoutModifier != 1 because SlowRuntimeTimeoutModifier returns 1 for checked (non-release, non-debug) builds, meaning it would not correctly skip the 2 GB test case on checked runtimes.

Using [ConditionalTheory] without parameters (instead of plain [Theory]) ensures that SkipTestException is correctly treated as a skip rather than a test failure.

Original prompt

This section details on the original issue you should resolve

<issue_title>System.Formats.Nrbf.Tests.EdgeCaseTests.CanReadArrayOfAnySize times out on checked coreclr</issue_title>
<issue_description>## Build Information
Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1348315
Leg Name: Libraries Test Run checked coreclr windows x64 Release

Error Message

{
    "ErrorMessage": ["CanReadArrayOfAnySize", "timed out"],
    "BuildRetry": false,
    "ExcludeConsoleLog": false
}

Description

System.Formats.Nrbf.Tests.EdgeCaseTests.CanReadArrayOfAnySize times out after 14+ minutes on the checked coreclr windows x64 Release leg. The test runs as a long-running test until the Helix work item is killed.

Helix log: https://helix.dot.net/api/2019-06-17/jobs/c6faf158-7c28-4acb-bec6-ddcc415ba5e5/workitems/System.Formats.Nrbf.Tests/console

Previous occurrence: #110285 (closed December 2024)

Pull request where observed: #125961 (codeflow update, unrelated to the failure)

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1348315
Error message validated: [CanReadArrayOfAnySize timed out]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 3/23/2026 11:03:41 PM UTC

Report

Build Definition Test Pull Request
1348938 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125881
1348882 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125174
1348844 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125129
1348773 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125983
1348315 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125961
1348754 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125981
1348752 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125083
1348706 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125439
1348612 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125973
1348585 dotnet/runtime System.Formats.Nrbf.Tests.WorkItemExecution #125835
1348578 dotnet/runtime [System.Formats.Nrbf.Tests.WorkItemExecution](https://dev.azure.co...

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

…aseRuntime condition

Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/185c955f-9d8a-4286-94bf-f0af9fc6b65f
Copilot AI changed the title [WIP] Fix timeout issue in CanReadArrayOfAnySize test Fix CanReadArrayOfAnySize timeout on checked coreclr by skipping on non-release runtimes Mar 24, 2026
Copilot AI requested a review from danmoseley March 24, 2026 02:19
@danmoseley danmoseley marked this pull request as ready for review March 24, 2026 02:29
Copilot AI review requested due to automatic review settings March 24, 2026 02:29
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates a resource-intensive System.Formats.Nrbf test to avoid timing out when executed on non-Release runtimes (notably Checked CoreCLR), where additional runtime validation makes the existing workload too slow.

Changes:

  • Add PlatformDetection.IsReleaseRuntime to the ConditionalTheory gating EdgeCaseTests.CanReadArrayOfAnySize, skipping it on Checked/Debug runtimes.

Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot apply my suggestion, but make sure it builds and tests are passing before you push the commit

@adamsitnik adamsitnik added the test-enhancement Improvements of test source code label Mar 25, 2026
…f ConditionalTheory

Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com>
Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/127eb19e-8537-496a-aeb8-6a23a106f8cd
auto-merge was automatically disabled March 25, 2026 14:19

Head branch was pushed to by a user without write access

Copy link
Contributor Author

Copilot AI commented Mar 25, 2026

@copilot apply my suggestion, but make sure it builds and tests are passing before you push the commit

Applied in 6a4f571 — build confirmed clean (0 warnings, 0 errors).

Copilot AI changed the title Fix CanReadArrayOfAnySize timeout on checked coreclr by skipping on non-release runtimes Fix CanReadArrayOfAnySize timeout on checked coreclr by skipping the 2 GB case on slow runtimes Mar 25, 2026
Copilot AI requested a review from adamsitnik March 25, 2026 14:22
Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @danmoseley !

[Theory] does not handle SkipTestException - it treats the throw as a
test failure. [ConditionalTheory] wires up a custom test invoker that
catches SkipTestException and reports the test as skipped.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

🤖 Copilot Code Review — PR #126006

Holistic Assessment

Motivation: The PR addresses a real, recurring CI timeout — CanReadArrayOfAnySize with the 2 GB Array.MaxLength case takes 14+ minutes on the checked coreclr windows x64 Release leg, killing the Helix work item. The original #if RELEASE && NET compile-time guard was ineffective because it checks the library build configuration (always Release), not the runtime configuration. The fix is well-motivated.

Approach: Replacing the compile-time guard with a runtime skip is the right approach, and using [ConditionalTheory] (no parameters) to enable SkipTestException handling is correct. However, the specific runtime condition has a gap that prevents it from fixing the reported issue.

Summary: ❌ Needs Changes. The skip condition uses PlatformDetection.SlowRuntimeTimeoutModifier != 1, but SlowRuntimeTimeoutModifier returns 1 for Checked runtimes on x64 — the exact configuration that's timing out. The 2 GB test case will still run and timeout on checked coreclr x64.


Detailed Findings

❌ Skip condition does not fire on checked coreclr x64 — the reported CI leg

The condition at line 71:

if (length == 2147483591 && (!PlatformDetection.Is64BitProcess || PlatformDetection.SlowRuntimeTimeoutModifier != 1 || !PlatformDetection.IsNetCore))

On checked coreclr windows x64 (the failing CI leg):

Sub-condition Value Reason
!Is64BitProcess false It's x64
SlowRuntimeTimeoutModifier != 1 false See trace below
!IsNetCore false It's .NET Core

Result: true && (false \|\| false \|\| false)false — test is NOT skipped, timeout persists.

SlowRuntimeTimeoutModifier trace for Checked runtime on x64:

if (IsReleaseRuntime)        // false — it's Checked
    return 1;
if (IsRiscV64Process)        // false — it's x64
    return IsDebugRuntime ? 10 : 2;
else
    return IsDebugRuntime ? 5 : 1;  // IsDebugRuntime is false (Checked ≠ Debug) → returns 1

The three runtime modes (Debug, Checked, Release) are mutually exclusive via AssemblyConfigurationEquals. A Checked runtime is neither Debug nor Release, so SlowRuntimeTimeoutModifier falls through to the else branch and returns 1 — the same as Release.

Note: SlowRuntimeTimeoutModifier's own doc comment says "For use as needed on tests that time out when run on a Debug or Checked runtime", but the implementation doesn't account for Checked on non-RISC-V. That appears to be a pre-existing gap in SlowRuntimeTimeoutModifier itself.

Suggested fix — revert to @adamsitnik's first suggestion which correctly handled this:

if (length == 2147483591 && (!PlatformDetection.Is64BitProcess || !PlatformDetection.IsReleaseRuntime || !PlatformDetection.IsNetCore))

Or, if SlowRuntimeTimeoutModifier is preferred, fix it first to return > 1 for Checked runtimes (e.g., return IsDebugRuntime ? 5 : IsCheckedRuntime ? 2 : 1;).

(This finding was independently confirmed by both Claude Opus 4.6 and GPT-5.4.)

[ConditionalTheory] without parameters — Correct

Changing from [ConditionalTheory(typeof(PlatformDetection), nameof(PlatformDetection.Is64BitProcess))] to bare [ConditionalTheory] is correct:

  • The old attribute skipped all test cases (100, 64K, 127K) on 32-bit, which was unnecessarily restrictive — those small arrays work fine on 32-bit.
  • The 32-bit guard is now applied only to the 2 GB case via the in-method check.
  • [ConditionalTheory] (vs plain [Theory]) is required for SkipTestException to be reported as a skip rather than a failure, as @danmoseley correctly identified.

✅ Smaller test cases now run on all platforms — Good improvement

The smaller InlineData cases (100, 64_001, 127_000) are no longer gated by #if or 64-bit conditions, expanding test coverage.

Note

This review was generated by Copilot (Claude Opus 4.6), with a parallel review from GPT-5.4.

Generated by Code Review for issue #126006 ·

@danmoseley
Copy link
Member

@copilot address feedback

@danmoseley
Copy link
Member

@copilot it needs to skip on all checked runtimes whatever bitness. See text above

… of any bitness

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/558ab2d2-fd9c-4914-a0aa-7032325fc90c

Co-authored-by: danmoseley <6385855+danmoseley@users.noreply.github.com>
auto-merge was automatically disabled March 26, 2026 15:37

Head branch was pushed to by a user without write access

Copy link
Contributor Author

Copilot AI commented Mar 26, 2026

@copilot it needs to skip on all checked runtimes whatever bitness. See text above

Fixed in ff355fb. SlowRuntimeTimeoutModifier returns 1 for checked builds (it only returns >1 for debug), so it didn't catch the timeout case. Switched to !PlatformDetection.IsReleaseRuntime which correctly skips on checked and debug runtimes of any bitness.

Copilot AI changed the title Fix CanReadArrayOfAnySize timeout on checked coreclr by skipping the 2 GB case on slow runtimes Fix CanReadArrayOfAnySize timeout on checked coreclr by skipping the 2 GB case on non-release runtimes Mar 26, 2026
@danmoseley danmoseley enabled auto-merge (squash) March 26, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.Formats.Nrbf test-enhancement Improvements of test source code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

System.Formats.Nrbf.Tests.EdgeCaseTests.CanReadArrayOfAnySize times out on checked coreclr

4 participants