Skip to content

[ROSAENG-62134] ocm-agent fleet mode: restoreNotificationStatus causes infinite retry storm on 429 rate limit #319

Description

@samanthajayasinghe

Summary

When the OCM service log API returns HTTP 429 (rate limit exceeded), ocm-agent fleet mode enters an infinite retry loop. The restoreNotificationStatus() function rolls back lastTransitionTime after every failed send, causing canSendNotification() to return true on every reconciliation cycle (~15 min). This generates ~24 wasted API requests per hour, per affected cluster, indefinitely.

Root Cause

The issue is in pkg/handlers/webhookrhobsreceiver.go in the processAlert function (line 415-423). The code flow:

updateNotificationStatus() sets lastTransitionTime = now and increments FiringNotificationSentCount (written to K8s MFNR)

sendNotification() POSTs to OCM API → receives 429

restoreNotificationStatus() rolls back both lastTransitionTime and FiringNotificationSentCount to their pre-update values

On the next reconciliation (~15 min later), canSendNotification() reads the rolled-back lastTransitionTime, finds it is older than ResendWait (24h), and returns true

Steps 1-4 repeat indefinitely

The rollback was designed as a workaround for the optimistic update pattern (status is updated before the send, so it must be rolled back on failure). However, it does not distinguish between transient errors (where retry is appropriate) and rate-limit errors (where retry makes the problem worse).

Impact

API waste: Each 15-min cycle fires 1 initial POST + 2 SDK retries per cluster = 3 requests. With 2 clusters affected, that is 6 requests/cycle or ~24 requests/hour, all returning 429.

Envoy/Limitador pressure: The fleet service account's request-rate quota (enforced by the Envoy sidecar via Limitador at limitador.app-sre-rate-limiting.svc:8081) is consumed by these wasted retries, potentially starving legitimate service log sends for other clusters on the same management cluster.

Silent service log loss: The service log is never created — the customer never gets notified — but the system keeps trying without success.

Observed Behavior

Error in ocm-agent-fleet logs:

error="can't post service log: status is 429, identifier is '8', code is 'OCM-CA-8' and operation identifier is '...': Exceeds rate limit request"

Proposed Fix

Two changes:

  1. pkg/ocm/ocm.go — Return typed RateLimitError on 429

Add a RateLimitError type and return it from SendServiceLog when the response status is 429. This allows callers to distinguish rate limiting from other failures.

  1. pkg/handlers/webhookrhobsreceiver.go — Rate-limit-aware backoff

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pr-openAn open PR already addresses this issue

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions