Add GET /jobs/next action; deprecate list(ids_only=1) claim-semantics

[mihow]

Context

PR #1261 bundled a set of async_api concurrency fixes. One of those — the
worker fairness rework — reshaped GET /jobs/?ids_only=1 into a
claim-next-job endpoint: random ordering, limit=1 default via a paginator
override, and a heartbeat side-effect on every call. Reviewers (see
PR #1261) flagged that the list() endpoint has quietly become a pop()
queue with three hacks bolted on.

The right shape is a dedicated action.

Goal

Introduce a dedicated GET /api/v2/jobs/next/ action that:

1. Returns the next claim for a worker: a single job's id (or the
   {id, pipeline_slug} pair the worker currently expects).
2. Accepts the same filter params workers use today
   (pipeline__slug__in, dispatch_mode, incomplete_only).
3. Runs the heartbeat on every call (same Redis-throttled helpers already
   in ami/jobs/views.py).
4. Uses random ordering + limit=1 internally — no paginator override.

Refactor plan:

• Extract the shared work into a helper (e.g. _get_next_ml_job(request))
  called by both the new next() action and the existing
  list(ids_only=True) path.
• Mark list(ids_only=True) as deprecated in the OpenAPI schema and in
  comments, but keep it working until every running ADC worker has
  migrated.
• Once ADC is migrated (companion issue in ami-data-companion), remove:
  • the order_by(\"?\") override in get_queryset()
  • the paginator override in JobViewSet.paginator
  • the ids_only=1 branch of list() — the heartbeat moves to next()

Each of the three removals is marked `⚠️ TEMPORARY HACK — remove by
2026-04-24` in code. That deadline is aspirational, not hard — treat as
"start this as soon as #1261 lands."

Related

• PR fix(jobs): fixes for concurrent ML processing jobs #1261 — the PR that introduced the hacks
• Narrow idle-poll heartbeat scope via X-Processing-Service-Name header #1264 — scoping the heartbeat to the specific calling service (needs feat: API key auth and identity for processing services #1194)
• feat: API key auth and identity for processing services #1194 — client-ID / application-token auth

Acceptance

• GET /jobs/next/ action exists and returns a single claim
• ADC trapdata/antenna/client.py::get_jobs migrated to /next
• All three ⚠️ TEMPORARY HACK markers in ami/jobs/views.py removed
• list(ids_only=True) either deleted or documented as the old contract

[mihow]

Ensure we add caching to this. It gets polled quite often