Add a per-sample wall-clock cap for live fuzzing (marathon reps starve slots + collapse diversity)

[Ella-Hoeppner]

Summary

Live-fuzzing samples have no per-sample wall-clock cap. The only limits are CYCLE_CAP_SECONDS = 1800 (per fuzz cycle) and message_limit (per experiment config, currently 100). An agent that keeps calling start_fuzzing(900s) cycle after cycle without concluding can run for hours on a single harness — at ~2–3 messages per 900s cycle, 100 messages × 15 min ≈ many hours. These "marathon" reps tie up concurrency slots, stall sample throughput, and (worse) collapse harness/entry-point diversity, which matters more than raw fuzz-time for surfacing real bugs.

Evidence (observed during the 2026-06-18 overnight fuzzing-linked run)

With 7 concurrent shards, completed-sample count flatlined (56 for >1 hour) while all containers stayed at ~100% CPU. Running reps at that moment:

207 min: haiku/libpng        136 min: haiku/libpng         91 min: opus/zlib
 82 min: sonnet/libpng        82 min: haiku/zlib           81 min: haiku/libjpeg-turbo
 65 min: sonnet/libxml2

i.e. 2–3 reps running 1.5–3.5h each, corroborated by container uptimes (one up 3h). They are genuinely fuzzing (not hung) — just never concluding. Net effect: fuzz-hours/tokens keep accruing (the efficiency-metric denominator is fine), but a 3.5h session spends all its budget on ONE libpng harness instead of spreading across entry points.

Why it matters

• Throughput / slot starvation: a few marathon reps monopolize the concurrency pool, so other (model, target) cells advance slowly and the sweep stays imbalanced longer.
• Diversity: for vuln-finding, N short samples (N harnesses, N entry points) generally beat 1 very long sample (1 harness). The marathon pattern trades breadth for depth on an arbitrary harness the agent happened to land on.
• Predictability: sample wall-time is currently unbounded, making run-time estimation and balanced sweeps hard.

Suggested approach (not prescriptive)

• Add a per-sample wall-clock budget (e.g. 45–60 min) enforced in the solver loop in tasks/fuzzing.py — track elapsed since sample start and stop entering new fuzz cycles once exceeded (let the in-flight cycle finish, then go to triage/finish). Make it a constant near CYCLE_CAP_SECONDS and/or a task param so sweeps can tune it.
• Alternatively/additionally, cap the number of fuzz cycles per sample, or have start_fuzzing decrement a per-sample time budget so the sum of cycle durations can't exceed the cap.
• Consider surfacing the cap in the experiment TOML (defaults) alongside message_limit and fuzz_duration.
• Keep CYCLE_CAP_SECONDS (per-cycle) as-is; this is a separate whole-sample bound.

Code pointers

• scaffold/src/parser_security_eval/tasks/fuzzing.py — CYCLE_CAP_SECONDS (per-cycle cap), the solver loop, and start_fuzzing tool; the agent loop has no aggregate time bound.
• Experiment config knobs (message_limit, fuzz_duration) live in the per-shard TOML / experiments/ configs.

Caveat / timing

This changes the sample-length distribution, so it's a methodology change — coordinate it with a clean run boundary (don't mix capped and uncapped samples in one baseline slice). Deferred intentionally; not urgent.