Context
Core research question: "What is the incidence of vulns per unit walltime of fuzzing in targeted parsers?" This experiment establishes the baseline by running single-agent live fuzzing on each Tier 1 target.
Experiment design
- Targets: libpng, libjpeg-turbo, libxml2, zlib
- Agent: single agent per run (no swarm)
- Models: 2-3 frontier models (e.g., Claude Sonnet 4.6, Claude Opus 4.6, GPT-4o)
- Duration: multiple time budgets to see discovery curves:
- 30-min cycles (lit-review validated: PBFuzz 25.6x speedup, HGFuzzer 24.8x speedup)
- Continuous 2-hour run (control/comparison)
- Note: the lit review strongly suggests short cycles outperform continuous runs — this experiment should directly test and quantify this for our targets
- Engine: libFuzzer (default), with AFL++ comparison runs
- Replicates: 3+ runs per configuration for statistical significance
- Metrics:
- Unique crashes found vs. time
- Coverage vs. time
- Crashes per CPU-hour
- Harness quality (compiles on first try? iterations needed?)
- Controls:
- Existing oss-fuzz harness (no agent) as baseline
- Random harness mutations as lower baseline
Additional condition from lit review
Add a cycle-length comparison condition motivated by PBFuzz/HGFuzzer/RandLuzz findings:
- Condition X: 30-min cycles with LLM synthesis step between each (recommended architecture)
- Condition Y: continuous 2-hour run with no mid-run LLM interaction
This directly tests the key lit review finding that short cycles + LLM reflection converge faster. Expected outcome: Condition X finds first crash sooner and achieves higher coverage at the 2-hour mark.
Deliverables
- Results table: vulns/CPU-hour by target × model × engine × cycle-condition
- Coverage growth curves per target (time-to-first-crash highlighted)
- Cycle-length comparison: 30-min cycles vs continuous run
- Comparison to non-AI fuzzing baseline
- Written analysis in
docs/ or experiments/
Dependencies
Blocks
Context
Core research question: "What is the incidence of vulns per unit walltime of fuzzing in targeted parsers?" This experiment establishes the baseline by running single-agent live fuzzing on each Tier 1 target.
Experiment design
Additional condition from lit review
Add a cycle-length comparison condition motivated by PBFuzz/HGFuzzer/RandLuzz findings:
This directly tests the key lit review finding that short cycles + LLM reflection converge faster. Expected outcome: Condition X finds first crash sooner and achieves higher coverage at the 2-hour mark.
Deliverables
docs/orexperiments/Dependencies
Blocks