Skip to content

Fix loading saved evaluation results and script exit codes - #24

Merged
nushakrishnan merged 3 commits into
cvg:mainfrom
ishantamrakar:fix-eval-tooling
Oct 7, 2026
Merged

nushakrishnan merged 3 commits into
cvg:mainfrom
ishantamrakar:fix-eval-tooling

Conversation

@ishantamrakar

Copy link
Copy Markdown
Contributor

For items 1 and 2 of #23; item 3 is left open there.

  • SparseEvalResult.load_from_npy rebuilds ControlPointSummary objects from the dicts written by save_as_npy, so the metric functions work on a reloaded result.
  • evaluate_wrt_control_points.py, evaluate_wrt_pgt.py and evaluate_wrt_mps.py exit with status 1 when run() returns False.

Tested on sequence_1_19 from the demo, with the pinned pycolmap/pyceres on Linux:

  • Scores and recalls printed by the scripts are unchanged (CP score 49.8639 for the demo OpenVINS estimate, 69.9432 for the pGT; evaluate_wrt_pgt.py recalls unchanged).
  • A result saved by evaluate_wrt_control_points.py and reloaded with load_from_npy gives the same score and recall as the in-memory result.
  • Graceful failures now exit 1 (an estimate with too few control points for the initial alignment; a missing --sparse_eval_result); successful runs exit 0.
  • evaluate_wrt_mps.py has the same change but was not run, as I don't have MPS data for the demo.
  • format.sh with ruff 0.12.7 reports no changes.

@nushakrishnan
nushakrishnan merged commit 238b6ca into cvg:main Oct 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants