Problem
In packages/jobs/src/webhook-sweeper.ts, consecutiveFailures only increments when the full delivery retry chain is permanently exhausted (not on intermediate retries). A successful mid-run delivery resets the counter to 0. This means:
- Configured threshold: 5
- Real effective threshold: 5 permanently exhausted chains (not 5 simple failures)
- A flaky endpoint that fails 4× then succeeds 1× then fails 4× again never trips the breaker
Also: consecutiveFailures is reset to 0 on ANY success in a sweep window, even if 4 other deliveries in the same window permanently failed.
Fix
Increment on every delivery failure (not just exhausted chain) and only reset on a fully-clean sweep window:
// On each permanent failure:
consecutiveFailures += 1;
// On success — only reset if NO failures in this sweep:
if (allSucceeded) consecutiveFailures = 0;
// Otherwise: leave counter as-is
Severity
MEDIUM — circuit breaker provides false safety; persistently flaky endpoints never auto-disable.
Problem
In
packages/jobs/src/webhook-sweeper.ts,consecutiveFailuresonly increments when the full delivery retry chain is permanently exhausted (not on intermediate retries). A successful mid-run delivery resets the counter to 0. This means:Also:
consecutiveFailuresis reset to 0 on ANY success in a sweep window, even if 4 other deliveries in the same window permanently failed.Fix
Increment on every delivery failure (not just exhausted chain) and only reset on a fully-clean sweep window:
Severity
MEDIUM — circuit breaker provides false safety; persistently flaky endpoints never auto-disable.