Skip to content

Narrow idle-poll heartbeat scope via X-Processing-Service-Name header #1264

Description

@mihow

Follow-up to PR #1261 commit 44d4c322, which added a heartbeat dispatch on GET /api/v2/jobs/?ids_only=1 polls so idle workers no longer age out of PROCESSING_SERVICE_LAST_SEEN_MAX between jobs.

Current scope (too coarse)

_mark_async_services_seen_for_project(project_id) marks every async ProcessingService attached to the polling project as seen. If multiple workers share a project but only one is actually polling, all of them show as live.

Proposal

Reuse AMI_ANTENNA_SERVICE_NAME — the ADC env var already used at registration-time to get_or_create a ProcessingService row by name — as a heartbeat identity.

Antenna side

  • Accept header X-Processing-Service-Name on JobViewSet.list() when ?ids_only=1 is set.
  • If present, look up ProcessingService.objects.async_services().filter(name=<header>, projects=<project>).first() and mark only that row seen. Cache key becomes heartbeat:list:service:<service_id> so one service's poll does not suppress another's.
  • If absent or no matching row, fall back to the current project-wide heartbeat. Backward-compatible with older ADC clients.

ADC side (ami-data-companion)

  • Thread the service name through get_http_sessionget_jobsPOST /result/ as X-Processing-Service-Name, using the same get_full_service_name(service_name) form ({name} ({hostname})) that registration already sends.
  • Granularity is per-host (hostname in the service name), which matches the per-box ProcessingService row pattern we're using in practice.

Why now

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    PSv2Async & distributed ML backend (PSv2): job state, NATS dispatch, result handling. Umbrella #515.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions