[jeremydmiller]

Closes #5409.

Adds src/Testing/DurabilityChaosRig, the out-of-process crash rig that found #5399, #5408 and #5410. A driver runs a producer and three to five workers, injects faults from one seed, then checks a ledger for lost, duplicated and stuck messages. The faults are SIGKILL, SIGTERM, SIGSTOP past the stale node timeout, and docker pause of the database or the broker. Like KafkaPerfRig, it is in neither .slnx solution and not in CI, so LocalGate and VerifyLocalGateCoverage are unaffected.

• README: covers the roles, every CHAOS_* knob, the containers each mode needs, what CLEAN means, how to investigate a run that is not clean, and the gotchas.
• CLAUDE.md: a short section next to the Performance Rig section.
• The obsolete CHAOS_PROCESS_ID_NEWID workaround was already gone, since Default envelope ids no longer collide between processes on one machine #5399 fixed the cause.

Smoke runs (2026-11-07, this branch)

Mode	Seed	Faults injected	Sent	Result	Slowest message
PostgreSQL queue + Marten, calm baseline	1	none, 30 s	5,780	CLEAN	185 s
PostgreSQL queue + Marten	31	2 dbdown, 3 pause, 2 kill, 90 s	14,660	CLEAN	489 s
RabbitMQ + Marten	32	8 kill, 3 term, 1 dbdown, 1 brokerdown, 90 s	15,180	CLEAN	20 s
SQL Server queue + EF Core	33	5 kill, 3 term, 2 pause, 90 s	11,040	1 dead letter	n/a

Two of these are real findings, already being fixed:

• PostgreSQL latency: the long tail on both PostgreSQL runs is Investigate the slow drain after PostgreSQL outages in the durability chaos rig (~8 min settle, ~470 s worst message) #5413. After a full batch the listener waits a whole polling interval, because the check was count > MaximumMessagesToReceive, which a pop can never satisfy. The same bug is in the MySQL and Oracle listeners.
• SQL Server dead letter: a Work handler's commit was chosen as a deadlock victim, and the message was dead-lettered. The system_health deadlock graph shows the scheduled-message poll's select holding shared page locks while handlers mark rows Handled. That is SQL Server: the scheduled-message poll is a deadlock victim under normal multi-node load #5412.

🤖 Generated with Claude Code

https://claude.ai/code/session_013nKJrqsJw3j23Xd46q37ob