Feat/3d refine fixed points - #17
Merged
Merged
Conversation
…ection GUI Imperfect camera calibration means no single 3D point reprojects exactly onto all views, so Edit 3D now supports finalizing per-view points. There is always one internal 3D point per keypoint; a drag re-estimates it by a constrained DLT over the fixed views' locked pixels plus the dragged cursor, and non-fixed views follow its reprojection. So how a point moves in one view when another is dragged depends on which views are fixed. - Dragging a joint pins the dragged view at the drop pixel (a finalized constraint), so it stays exactly where dropped instead of snapping to the reprojection; the live mid-drag re-solve follows the mouse without pinning. - Right-click toggles a view's "fixed" flag directly (e.g. to un-pin). - Fixed points render with a bold yellow ring. The corrections.h5 sidecar gains a pose2d_corrections/fixed (V,T,P) boolean mask (schema v2, backward compatible: v1 files load with it all-False). results.h5 is still never touched. The three outputs of a refined frame are the estimated 3D points, the corrected per-view 2D, and the fixed mask. Adds EditorState refinement tests, sidecar round-trip/v1-compat tests, and headless widget tests covering the pin-on-release gesture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`deeperfly gui` now starts a local FastAPI + uvicorn server and opens a browser editor instead of a PySide6 desktop window. The front-end is plain ES modules served straight from deeperfly/gui/web (no bundler, no build step); the server exposes a small REST + WebSocket API around a Qt-free Session that wraps the existing EditorState corrections logic. This runs headless and can be reached from another machine (default-bound to localhost; tunnel with `ssh -L` for remote use). - FastAPI + uvicorn move from the optional `gui` extra into the core deps (tiny next to torch/jax, and `gui` is a first-class subcommand); the test group swaps PySide6 for httpx to drive the API/WebSocket in-process. `doctor` now checks for FastAPI + uvicorn. - New `gui` options: --host, --port, --no-browser, --keep-alive. Binding a non-loopback host warns (the editor is unauthenticated). By default the server stops a few seconds after the last tab closes; a refresh reconnects. - Edit 3D is now the default mode when a result has 3D, falling back to Edit 2D otherwise. - Removes the old view.py/window.py Qt widgets and their tests; adds test_gui_server.py covering the API, WebSocket edits, and shutdown behavior. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ection GUI A per-view 2D keypoint in Edit 3D can now be flagged "obscured" (invisible): the camera genuinely cannot see it, so it is dropped from triangulation and its dot just follows the reprojection. Marking a view invisible re-solves the 3D point from the remaining visible views, so a bad/occluded observation stops dragging the estimate off. Invisible is mutually exclusive with fixed -- a point is at most one of normal / fixed / obscured. Interaction: select a joint, then use the new point-status widget (shows "<point> · <camera>" and a Normal/Fixed/Obscured switch) or the l (fix) / o (obscure) keys. Dragging an obscured point un-obscures it back to normal. - corrections.py: new pts2d_invisible mask + set_invisible(); sidecar schema v3 (older sidecars load with it all-False); set_pts2d/clear_2d clear the flag. - state.py: toggle_invisible() + _resolve_3d_from_visible() (fixed views still define the point when >=2 are fixed; otherwise triangulate every non-obscured view's displayed 2D); apply_3d_edit un-obscures the dragged view. - server.py: invisible mask in the points payload + toggle_invisible dispatch. - web: status widget, l/o shortcuts, ghosted (dashed magenta) rendering, and the drag-to-un-obscure flow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.