Epic: Robust, fidelity-preserving transpilation engine

[Tokarzewski]

Tracking issue for the push to a robust engine whose output stays faithful to the source's architecture (same modules/classes/functions/control-flow), idiom-mapping only at the leaf level — not a refactored beast.

The architecture (HIR→MIR→LIR, refuse-don't-guess, verified+cached LLM holes, stdlib_maps flywheel) is in place. The gap is empirical: ~100% on curated algorithms vs ~46% on real C++ (compile-only). Close it via measurement → fidelity guarantees → a verification/repair loop → ground-truth typing → a compounding data flywheel.

Phase 0 — Measure

• Failure taxonomy: bucket every verify-gate failure #42 Failure taxonomy in the verify-gate (unblocks prioritization of everything below)

Phase 1 — Fidelity core (the 'graph of leaves, faithful skeleton')

• HIR→LIR provenance map (fidelity backbone) #43 HIR→LIR provenance map (backbone)
• Add --fidelity {structural|idiomatic} dial #44 --fidelity {structural|idiomatic} dial
• Structural-fidelity verifier (skeleton isomorphism check) #45 Structural-fidelity verifier (depends on HIR→LIR provenance map (fidelity backbone) #43)
• Semantic contracts on IR nodes (beyond types) #46 Semantic contracts on IR nodes

Phase 2 — Robustness loop

• Verification-driven repair loop (escalating tiers) #47 Verification-driven repair loop (highest ROI)
• Behavioral equivalence: source I/O traces + generated inputs #48 Behavioral equivalence: source traces + generated inputs (engine + fuzz inputs + pass-rate metric + repair-loop signal done for Python→Python/Rust; other source langs & compiled targets deferred)

Phase 3 — Hard source languages (ground truth, not guessing)

• Python trace-driven typing for static targets #49 Python trace-driven typing
• C++ ground-truth types via clang AST / compile_commands.json #50 C++ types via clang AST / compile_commands.json

Phase 4 — Compounding flywheel

• Close the data flywheel: promote verified repairs → stdlib_maps + SFT #51 Promote verified repairs → stdlib_maps + SFT corpus (engine side complete & wired into the live loop; next-fine-tune step gated on cloud training, feat: monetize transpilers as the best 'source → verified Mojo' inference API #55/north-star: bootstrap the verified Mojo ecosystem — make real codebases migratable to Mojo #56)

Principles: non-uniform granularity (faithful skeleton, idiomatic leaves) · analyze globally, translate locally · contracts carry semantics · provenance map = checkable fidelity invariant · translate fine, verify coarse · never weaken refuse-don't-guess.

[Tokarzewski]

Status — fidelity engine epic, this session.

Phase 0–3 remain done; Phase 2 #48 (behavioral equivalence) advanced with a
concrete pure-Python slice (PR #73): classify_divergence now names the root
cause of a behavioral divergence, starting with the C-family integer-semantics
gap (Python floored %/// vs Rust/C/Zig/Mojo truncated — -1 % 2 → 1 vs -1).
BehavioralReport.divergence_class surfaces it in summary() and the
run_mismatch repair signal, giving #47's loop a sharper hint. 6 new pytest
cases; full suite green.

This divergence class was independently surfaced by the new #57 dataset
generator's Rust runner on gcd/lcm/fast_pow — now a first-class label.

Still deferred (need toolchains absent here): non-Python source oracles and
compiled targets beyond Rust (C/Go/Zig/Mojo/Fortran) via their _fmt harnesses;
the Runner protocol is the extension point. The compile-rate-vs-behavioral-rate
gap on real C++ (#52's empirical goal) needs the full verify toolchain to move
further.