Skip to content

docs(gsdc): record where the remaining gap to taroz is (Kaggle probes) - #666

Merged
rsasaki0109 merged 1 commit into
developfrom
claude/confident-clarke-k37g2b
Oct 27, 2026
Merged

rsasaki0109 merged 1 commit into
developfrom
claude/confident-clarke-k37g2b

Conversation

@rsasaki0109

Copy link
Copy Markdown
Owner

Summary

  • What changed? The readback record (docs/use_cases/records/gsdc2023_official_readback_20261024.json) gains a taroz_kaggle_probes section, and the goal doc gains a short section on it. The section records:
    • the Kaggle score of the archived taroz test output;
    • four phone-group swaps;
    • 40 single-drive swaps on top of the official recipe file 57678818.
  • Why? The remaining gap to the taroz post-deadline score (Private 0.024 m, Public 0.020 m) could not be located on train. Train with the current recipes is level with taroz or better.

Scope

  • Single feature / single operational improvement / single user-visible value
  • Adds or tightens one regression, sign-off, dogfood, or artifact-schema check
  • No unrelated refactor mixed in
  • No docs-only noise mixed into solver/protocol work. Docs only.
  • Protocol ingest, parser work, runtime knobs, solver behavior, and docs are split unless this PR explicitly needs the boundary crossing

External Reference

  • Source repo used for comparison: taroz/gsdc2023. Its archived test result_gnss_imu.mat files were assembled as submission.m does.
  • What was adopted here: nothing. The probes are diagnostic, and the recipe is unchanged.
  • What was intentionally not adopted: taroz rows in any submission the recipe pins.

Behavior Boundary

  • Runtime default changed? No.
  • Env-gated or config-gated? No.
  • Artifact/schema changed? Yes. The readback record gains a taroz_kaggle_probes section.
  • Dataset or credentials required? No.

Validation

  • python3 tests/test_cli_tools.py
  • python3 tests/test_benchmark_scripts.py
  • python3 tests/test_packaging.py
  • python3 tests/test_python_bindings.py -v
  • python3 tests/test_ros2_node.py
  • ctest --test-dir build --output-on-failure

Focused checks run:

python3 tests/test_gnss_reproduce.py   # OK (the record still loads)
git diff --check

Sign-off / Benchmark Impact

  • Added or updated regression/sign-off: none.
  • Affected dataset(s): GSDC 2023-2024 test (Kaggle readback only).
  • Findings:
    • Taroz archive: Public 0.796 / Private 0.782 m, the README score exactly (57683971).
    • Private split size: about 20 drives. A 2-epoch A205U misalignment (about 470 m) scored Private 24.117 m.
    • Private: most of the gap is on lax-n Pixel5 (-0.010), sjc-we1 Pixel7Pro (-0.007) and sjc-r A505U (-0.006). Native is better on 5 Private drives.
    • Public: mtv-m Mi8 (-0.018) and the ebf-xx start (-0.017) dominate. Native is better on ebf-x Mi8 (+0.014).
    • Where it shows: taroz wins through slowly varying 0.1-0.4 m differences, often at drive starts. Train shows no such start deficit (first 5 minutes 0.506 vs 0.516 m median error).
  • Expected metric impact: none.

Notes

  • Risk: none (docs only).
  • Follow-up PRs: possibly the ebf-xx start (Public) and the lax-n start (Private).
  • Explicit non-goals: using taroz output in the recipe.

Generated by Claude Code

The archived taroz test output, assembled as submission.m does,
reproduces Public 0.796 / Private 0.782 m (57683971). Swapping single
drives and phone groups of 57678818 for the taroz rows puts most of the
Private gap on lax-n Pixel5, sjc-we1 Pixel7Pro and sjc-r A505U, and the
Public gap on mtv-m Mi8 and the ebf-xx start; train shows no matching
start deficit. Diagnostic only; the recipe is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NWJ6qRF465rAzkFgQJvz7P
@rsasaki0109
rsasaki0109 merged commit 2b5580e into develop Oct 27, 2026
11 checks passed
@rsasaki0109
rsasaki0109 deleted the claude/confident-clarke-k37g2b branch October 27, 2026 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants