Make Move drag atomic with an explicit start position and guaranteed release

[zengfanfan]

Summary

Move(drag=True) currently starts dragging from the process-wide current cursor position. That preserves the existing API, but it makes reliable drag automation hard because selecting the start point and performing the drag require separate tool calls.

I would like to contribute a backward-compatible deterministic drag path for Move (or a separate Drag tool if maintainers prefer) that can perform one atomic gesture with an explicit start point, bounded duration/intermediate movement, guaranteed mouse release after a successful press, and optional fail-closed foreground assertions.

Current behavior

The current Move tool accepts loc, label, and drag. When drag=True, it delegates to desktop drag logic using the current cursor position as the source. In the current code path:

• Move(loc=..., drag=False) can position the cursor first.
• Move(loc=..., drag=True) then drags from wherever the cursor is at that moment.
• The low-level drag path performs mouse-down/move/up, but release is not enforced with a finally path after a successful press.

This is usable for simple flows, but it is race-prone for agents because the start position is global state between two separate tool calls.

Generic reproduction

A safe WinForms pointer-event fixture was used to record pointer events. The public Move contract was exercised through the registered Move tool with the real Desktop object:

1. Move the pointer to a safe start point with Move(loc=[278,371], drag=False).
2. Drag to a safe end point with Move(loc=[678,371], drag=True).
3. Inspect the fixture's pointer event log.

Observed event evidence from the fixture:

first=down(109,114)@2026-07-10T12:49:05.1726152Z | intermediate_moves=25 | final=up(509,114)@2026-07-10T12:49:05.5316388Z

This confirms the existing drag can produce down/move/up events in a generic app. The reliability gap is the public contract: the source point is implicit global cursor state rather than an explicit argument in the same tool call.

Expected behavior

A caller should be able to request a deterministic drag in one tool call:

• explicit start and end coordinates;
• bounded/effective duration with intermediate mouse moves;
• exact final coordinate attempt;
• mouse release guaranteed after a successful press, including movement exceptions;
• optional foreground title/process checks that fail before pressing the mouse button.

Proposed backward-compatible API

One possible shape is to extend Move only when drag=True:

Move(
    loc=[end_x, end_y],
    drag=True,
    from_loc=[start_x, start_y],          # optional; omitted keeps legacy current-cursor behavior
    duration=0.35,                        # optional bounded duration for intermediate movement
    expected_window_title="...",         # optional case-insensitive title substring guard
    expected_process="notepad.exe",      # optional exact process basename guard
)

Compatibility rules:

• Existing Move(loc, label, drag) behavior remains valid.
• from_loc accepts the same list / JSON-stringified-list coercion style as loc.
• New deterministic-drag-only parameters should fail clearly when drag is false instead of being silently ignored.
• If both foreground guards are supplied, both must match.
• Guard mismatch should fail before any mouse-down event.

If maintainers prefer a separate Drag tool, I can implement that shape instead. I am opening this issue first because the public API choice should be agreed before code is submitted.

Safety requirement

After mouse-down succeeds, the implementation should attempt mouse-up in a finally path even when intermediate movement fails. If mouse-up itself fails, the tool should not report false success.

Alternatives considered

• Keep using Move twice: simple, but the source point remains mutable global state between tool calls.
• Add public MouseDown / MouseUp tools: more flexible, but exposes long-lived button state and is easier for agents to misuse.
• Add only duration to current drag: improves smoothness but does not solve explicit-start reliability.
• Put target checks in the caller: still allows foreground drift between observation and input.

Compatibility impact

This can be implemented without changing existing callers. Existing Move(..., drag=True) would continue to drag from the current cursor position. The new parameters would only opt into deterministic behavior.

Duplicate search

I searched open and closed issues/PRs for drag, mouse, Move, duration, explicit start, stuck button, foreground guard, Unity, and pointer events. Related but not equivalent items found:

• fixed drag tool #45: fixed drag tool
• Add coordinate_system parameter for DPI-aware coordinate conversion #81: coordinate_system / DPI coordinate conversion
• Click/Move/Type coordinates misaligned on high-DPI displays due to physical vs logical pixel mismatch #86: Click/Move/Type high-DPI coordinate mismatch
• feat: add 9 advanced automation tools with DPI coordinate system support #93: advanced automation tools with DPI coordinate system support

I did not find an existing issue or PR for an atomic explicit-start drag with guaranteed release and optional foreground guards.

Environment used for initial validation

• Windows-MCP checkout: upstream/main at 132c53707ad2aad79e401c45a87a58b481e0c3fe when tested
• Local package version observed: 0.8.2
• Python: 3.13.2
• UV: 0.11.26
• Platform: Windows

Baseline repository gates before any drag changes had unrelated existing failures in my checkout:

• uv run ruff check .: existing lint failures outside the proposed change
• uv run pytest: 318 passed, 3 failed in tests/test_iserror_compliance.py imports unrelated to drag

I will keep any PR focused on this input/desktop/UIA behavior and will not include private consumer paths, local planning documents, or unrelated fixes.