Skip to content

feat: implement Kafka retry queue with hybrid retry strategy#63

Open
bashkirian wants to merge 1 commit into
mainfrom
feat/62-kafka-retry-queue
Open

feat: implement Kafka retry queue with hybrid retry strategy#63
bashkirian wants to merge 1 commit into
mainfrom
feat/62-kafka-retry-queue

Conversation

@bashkirian

Copy link
Copy Markdown
Owner

Add Kafka-based retry queue for payout orchestration with a hybrid retry strategy inspired by Hyperswitch:

  • Provider-level retry: quick retry same provider (short delays)
  • Global retry: fresh provider selection when all providers fail
  • Tiered backoff with jitter to prevent thundering herd

Infrastructure:

  • Add Redpanda to docker-compose
  • Create payout-retry-provider, payout-retry-global, payout-retry-dlq topics

Core changes:

  • Add kafka-go dependency
  • Create internal/queue package with Kafka publisher/consumer
  • Create internal/worker package with retry worker
  • Add retrying state and retry count tracking to payouts
  • Add migration for retry tracking columns

Configuration:

  • Tiered delays: provider [1s, 5s, 15s], global [30s, 2m, 5m, 15m, 30m]
  • Random jitter up to 20% to prevent thundering herd

Closes #62

Add Kafka-based retry queue for payout orchestration with a hybrid
retry strategy inspired by Hyperswitch:

- Provider-level retry: quick retry same provider (short delays)
- Global retry: fresh provider selection when all providers fail
- Tiered backoff with jitter to prevent thundering herd

Infrastructure:
- Add Redpanda to docker-compose
- Create payout-retry-provider, payout-retry-global, payout-retry-dlq topics

Core changes:
- Add kafka-go dependency
- Create internal/queue package with Kafka publisher/consumer
- Create internal/worker package with retry worker
- Add retrying state and retry count tracking to payouts
- Add migration for retry tracking columns

Configuration:
- Tiered delays: provider [1s, 5s, 15s], global [30s, 2m, 5m, 15m, 30m]
- Random jitter up to 20% to prevent thundering herd

Closes #62
@bashkirian bashkirian force-pushed the feat/62-kafka-retry-queue branch from 61bb27d to 8095940 Compare June 4, 2026 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: implement Kafka retry queue with hybrid retry strategy

1 participant