Implement dynamic max_concurrent_requests for LLM inference calls

[hynky1999]

Problem

LLM inference blocks currently use a static max_concurrent_requests value. That makes examples and production jobs choose between being conservative for rate limits or aggressive for throughput. It also pushes users to tune provider-specific concurrency manually, even when the best value depends on runtime behavior: provider throttling, latency, token volume, request failures, and model-specific limits.

This came up while running LeRobot subtask annotation with Gemini. The example can survive high concurrency, but hard-coding a fixed value in examples/docs is noisy and brittle.

Proposed Direction

Add dynamic concurrency control for LLM calls so inference blocks can adjust request concurrency during a run.

Potential behavior:

• Start from a configurable initial concurrency.
• Increase concurrency while requests are succeeding and latency/error rates are healthy.
• Decrease concurrency on provider throttling, retryable 429/503 errors, high failure rates, or excessive latency.
• Keep a hard upper bound for safety.
• Emit metrics for current concurrency, throttles, retries, and backoff decisions.
• Work across providers through common inference transport/error metadata.

API Sketch

Possible options:

mdr.robotics.subtask_annotation(
    max_concurrent_requests="auto",
)

or:

mdr.inference.ConcurrencyPolicy(
    initial=16,
    max=256,
    min=1,
    target_error_rate=0.01,
)

Acceptance Criteria

• Inference blocks can opt into dynamic request concurrency.
• Static max_concurrent_requests=int remains supported.
• Provider throttling/retryable errors reduce concurrency without failing the whole job immediately.
• Healthy sustained success can increase concurrency up to the configured cap.
• Runtime metrics expose the selected concurrency and backoff behavior.
• Tests cover concurrency increase/decrease behavior with simulated provider responses.

Notes

This should be implemented in the shared inference execution path rather than only in subtask_annotation, so it benefits all LLM/VLM blocks.