Track Windows Lyrical CI crash from upstream class_loader concurrent unload race #63

Summary

Track an intermittent native crash in the Windows ROS Lyrical / Fast DDS CI job, consistent with the upstream class_loader concurrent library unload bug fixed by ros/class_loader#234.

Temporary triage policy: CI failures matching the signature below are a known upstream dependency issue and may be ignored until the dependency is updated. This applies only to this specific crash signature, not to other failures in the same job or test.

Observed failure

• Workflow: test, PR Lazily materialize ROS graph collection snapshots #62, commit 8872b0787f2d258bc64070d1564c4911739e4617.
• Run: https://github.com/noelex/rclnet/actions/runs/36442337713
• Failed job: https://github.com/noelex/rclnet/actions/runs/36442337713/job/108995745604
• Environment: Windows Server 2022, ROS Lyrical, rmw_fastrtps_cpp, .NET 8.
• Step: Test (fastrtps).
• Crash time: 2026-09-28 15:25:09 UTC.
• Active test, confirmed by the blame sequence file: Rcl.NET.Tests.ShutdownLifecycleTests.RetainedChildrenKeepEachPhysicalContextsLoggingReference.
• The test retains one native node in each of two contexts and closes both contexts concurrently using Task.WhenAll(first.DisposeAsync().AsTask(), second.DisposeAsync().AsTask()).

The test host terminates with:

Fatal error. System.AccessViolationException:
Attempted to read or write protected memory.

at Rcl.Interop.RclCommon.rcl_shutdown(rcl_context_t*)
at Rcl.SafeHandles.SafeContextHandle.Shutdown()
at Rcl.RclContext+<>c__DisplayClass77_0.<Run>b__2()
at Rcl.RclContext+<>c__DisplayClass77_0.<Run>g__Attempt|0(System.Action)
at Rcl.RclContext.Run()

The .NET 8 run was aborted after 285 passed tests and 1 skipped test. The later .NET 9 and .NET 10 runs in the same job succeeded. The final MSB4181 is a consequence of the test-host crash, not the primary failure. All 11 other matrix jobs passed.

Dump analysis

Artifact: windows-lyrical-test-diagnostics (the workflow retains diagnostics for only 7 days).

Files include:

• Sequence_73e9349496674b9eb6f545aba1bbb11d.xml
• testhost.exe_7768_1790609109_crashdump.dmp

The dump and matching DLLs from the configured Windows ROS archive show:

• The faulting thread is unloading buffer backend plugins through rosidl_buffer_backend_registry, class_loader, and rcpputils::SharedLibrary::unload_library().
• The immediate failure is inside rcutils_unload_shared_library, at the indirect call to lib->allocator.deallocate(lib->library_path, lib->allocator.state).
• The saved fault context has RIP = 0 and RAX = 0; the return address is rcutils.dll + 0xde06, immediately after call rax at offset 0xde04.
• The referenced rcutils_shared_library_t is entirely zeroed, including its deallocator function pointer. This is consistent with overlapping unload operations on shared library state.

This strongly matches the upstream race, but the fix has not yet been validated against this CI failure. There is no evidence tying the crash to PR #62's graph snapshot changes.

Upstream status and affected dependency

• Upstream fix: Make loadLibrary()/unloadLibrary() safe against concurrent load and unload ros/class_loader#234 (merged September 2, 2026).
• Released in class_loader 3.0.2 on September 4, 2026: https://github.com/ros/class_loader/blob/3.0.2/CHANGELOG.rst
• Current Windows Lyrical archive in .github/workflows/test.yml: ros2-lyrical-2026-08-07-windows-AMD64.zip from https://github.com/ros2/ros2/releases/tag/release-lyrical-20260807
• As of September 29, 2026, that is still the latest official Lyrical release archive and predates the fix.

This is not inherently Windows-only: the upstream report reproduces the race on Linux/Jazzy. Ubuntu passing this run does not establish immunity; its installed class_loader version was not verified.

Follow-up when an upstream Windows package is available

• Check the next official Windows Lyrical archive and confirm it contains class_loader 3.0.2 or the equivalent backported fix; a newer archive date alone is insufficient.
• Update the Windows Lyrical archive URL in .github/workflows/test.yml and ensure the ROS installation cache does not reuse the old package.
• Re-run the Windows Lyrical / Fast DDS tests across .NET 8, 9, and 10, including repeated runs of the concurrent-context shutdown test.
• Remove the temporary known-failure exception and close this issue after verification.

Until then, matching failures can be left without a code change or repeated reruns solely to obtain a green result. Different crash locations, assertion failures, and other CI errors still require investigation. Keep the test enabled so the eventual dependency update can be validated.