[jeremydmiller]

The bug

EnvelopeIdGeneration.NewId (the default) and NewIdSqlServerOrder used the static MassTransit NewId. It builds an id from the machine's network identity, a tick, and a sequence number each process counts on its own, and Wolverine registered no process id provider. So two Wolverine processes on one machine draw identical envelope ids: in a two-process probe on one machine, 9,233 of 200,000 ids were shared. With a process id provider, none were.

In a shared inbox, under load, a collision shows up three ways. All of them look like durability bugs:

• 23505 on pkey_wolverine_incoming_envelopes_id when a handler cascades. The handler's transaction fails and the message is dead-lettered. On RabbitMQ it goes to the broker's dead-letter queue, so it is invisible in the database.
• Discarding a duplicate delivery of envelope X ... which the inbox already holds for a different message, which inbox deduplication then drops silently.
• The PostgreSQL queue pop rolls back its whole batch, which stalls the queue until the conflicting handled row expires.

This could account for some of the hard-to-reproduce data-loss reports, for example #4162 (RabbitMQ, SIGKILL, durable inbox, no inbox row), but that is not confirmed against any specific report.

How it was found

An out-of-process chaos rig. A producer commits a ledger row together with its Work message through the Marten outbox. Three to five workers in separate processes handle Work, which can cascade to a durable local queue, schedule a follow-up, and fail on its first attempt so it gets a scheduled retry. The driver SIGKILLs or SIGTERMs a random process every few seconds and restarts it, lets the cluster settle, then checks for messages that were lost, processed twice, or left stuck in the inbox, outbox or dead letters. The rig is not part of this PR.

Run	Messages	Kills / graceful stops	Before	With this change (same seed)
PostgreSQL queue	~18.7k	15 / 6	4 lost, 549 duplicate-key errors	clean, 0 duplicate-key errors
RabbitMQ	~22.2k	17 / 3	10 lost (2 in the native DLQ, 6 discarded as duplicates)	clean

A longer run with the same workaround applied in the rig (5 workers, 5 minutes, 48 SIGKILLs, 52 graceful stops, about 98k messages over both transports) was also clean.

The fix

EnvelopeIds is Wolverine's own NewIdGenerator. It uses NewId's own tick and worker id providers, plus CurrentProcessIdProvider. It deliberately does not call NewId.SetProcessIdProvider(), which is global and would change the ids MassTransit or application code in the same process draw from NewId. Ordering is unchanged: ids are still time ordered for .NET and PostgreSQL (NewId), and for SQL Server (NewIdSqlServerOrder).

Inside containers the app is usually PID 1 everywhere, so the process id does not separate instances that share a network identity. Those normally have distinct MACs anyway, and GuidV7 remains the documented choice for them.

Tests

• the_default_ids_differ_between_two_processes_drawing_them_at_the_same_moment: two generators at a fixed tick overlap without process ids (the bug, reproduced deterministically) and do not overlap with distinct process ids.
• default_ids_are_unique_across_threads.
• The tests that pinned the generator by reflecting on typeof(NewId) now point at EnvelopeIds: CoreTests, SqlServerTests and PolecatTests.

Run locally: the CoreTests id-generation classes (19), the SqlServerTests (7) and PolecatTests (2, net10.0) id-order tests, and dotnet build wolverine.slnx -c Release -f net9.0. Not run locally: the full CoreTests suite.

🤖 Generated with Claude Code