Add a reproducible scan benchmark & fixture harness for accuracy and latency

[DysektAI]

Problem

RatScanner currently has no quantitative baseline for scan latency or OCR/template-match accuracy. Decisions like "should we replace Tesseract with a lighter/faster OCR engine?" cannot be answered because the real overhead is unmeasured.

There is already an optional local harness, tests/RatScanner.Tests/ScanPipelineImageHarnessTests.cs, that runs real screenshots from the repo root, but it:

• Has no timing instrumentation.
• Is not documented for contributors.
• Cannot be run in CI because it depends on untracked local image files.
• Does not give users a way to package up a scan failure for reporting.

User reports of scan failures (wrong item, low confidence, missed marker, etc.) are also hard to reproduce across different display scale, brightness, contrast, overlays, and language settings.

Proposed solution (MVP)

1. Instrument the scan pipeline with Stopwatch

   • Inspection.OCR and MatchItem in src/ScanEngine/Processing/Inspection.cs.
   • Inventory grid parse and Icon template matching in src/ScanEngine/Processing/.
   • Output per-stage elapsed time to logs and/or a benchmark JSON report.

2. Create a ScanBenchmark runner

   • Console or xUnit test that loads fixture PNGs from a configurable directory (e.g. RATSCANNER_FIXTURES env var or tests/fixtures/).
   • Runs NameScanScreen, NameScan, and IconScan equivalents against each fixture using the same config/crop math as RatScannerMain.
   • Writes a JSON report per fixture: expected vs. detected item, confidence, per-stage milliseconds, and active config (resolution, scale, language).
   • Supports an expected manifest so it can be used for regression assertions.

3. Add an in-app "Export scan diagnostics" action

   • Save the exact Bitmap from GetScreenshot plus a sidecar JSON (scan type, click position, config, result, confidences, display bounds, DPI) into RatConfig.Paths.Debug.
   • Users can attach the bundle when reporting scan issues, and the benchmark can replay it.

4. Document fixture contribution guidelines

   • Prefer the in-game screenshot key to avoid overlays and preserve native resolution.
   • For name scans, include the full screen with an inspect window open.
   • For icon scans, include the inventory/stash and annotate expected item/position.
   • Keep fixtures small; do not commit large game-screenshot fixtures to the main repo (see tests/AGENTS.md).

5. Keep it optional in CI

   • Skip when no fixture directory is present, mirroring the existing Assert.SkipUnless behavior in ScanPipelineImageHarnessTests.

Acceptance criteria

• Running the benchmark on a fixture set produces a JSON report with per-stage timings and detected items/confidences.
• A user can export a scan diagnostics bundle from the app and the benchmark can replay it.
• Docs explain how to capture and contribute fixtures without bloating the repo.

Out of scope

• Broad in-app analytics/telemetry.
• Public hosting of copyrighted game screenshots in the main repo.

Related context

• tests/RatScanner.Tests/ScanPipelineImageHarnessTests.cs
• src/App/RatScannerMain.cs (NameScan, IconScan, GetScreenshot)
• src/ScanEngine/Processing/Inspection.cs
• src/ScanEngine/Processing/Icon.cs
• src/ScanEngine/Processing/Inventory.cs
• tests/AGENTS.md

[DysektAI]

Implemented across the TarkovTracker-org forks by tarkovtracker-org/RatEye#1 and #22.

RatEye now provides the versioned .ratdiag.json replay contract, per-stage processing timings, benchmark runner, JSON reports, expected-result assertions, and fixture contribution documentation. RatScanner exports the latest captured bitmap and a compatible replay sidecar containing scan type, detected IDs/confidences, RatEye configuration, display/DPI context, cursor position, and stage timings. Optional real-image tests continue to skip when local fixtures are absent.

A public screenshot corpus remains intentionally external/local because the issue explicitly excludes committing large copyrighted fixture sets. That is not a missing implementation requirement. Closing as completed.