feat: dispatch automation bundles through SDK conversation runtimes

[neubig]

Overview

This PR adds conversation-scoped execution for automations while keeping agent policy separate from execution mechanics.

Today, local automation bundles run through global Agent Server file/bash APIs and may receive a host-level session key. That contract cannot safely target a per-conversation Docker runtime, and it does not let an automation select an Agent Server agent profile. This change introduces an explicit execution scope and a conversation-aware backend so agent work can run with profile-selected tools and secrets in either a local or Docker-backed conversation workspace.

Existing automations remain unchanged: execution_scope defaults to "run".

Behavior

Scope	Intended use	Execution contract
run	Existing lightweight scripts and scanners	Uses the existing local or cloud run backend.
conversation	Delegated agent work	Creates or resumes an Agent Server conversation using agent_profile_id, then executes the bundle through that conversation's workspace APIs.

For conversation-scoped runs, the Agent Server decides whether the workspace is local or Docker-backed. Bundle code sees the same environment in either case.

What changes

• Adds execution_scope: "run" | "conversation" and optional agent_profile_id to automation definitions and API responses.
• Snapshots both fields onto each queued run so later definition edits affect only future runs.
• Adds migration 025 for the new definition/run columns, with "run" as the backwards-compatible default.
• Adds ConversationBackend, which wraps the existing local Agent Server backend and owns only conversation lifecycle, runtime credentials, verification, and cleanup.
• Routes bundle upload, synchronous commands, detached command startup, and command verification through SDK workspace APIs that support a conversation runtime ID.
• Persists the actual conversation ID on the run. Subject-based automations reuse the deterministic subject conversation; other runs use their run ID.
• Extends Git sync serialization/import and capability/preset routes for the new fields.
• Makes the watchdog observe started commands, allowing process exit to complete callback-less runs before their timeout.
• Releases Docker conversation runtimes after completion, timeout, or cancellation while preserving local conversations and subject-owned conversations according to existing keep-alive behavior.

Lifecycle

1. Queueing snapshots the automation's execution scope and agent profile onto AutomationRun.
2. The dispatcher selects the existing backend for run scope or wraps the local Agent Server backend with ConversationBackend for conversation scope.
3. ConversationBackend creates/resumes the conversation and discovers whether its runtime is local or Docker.
4. The bundle is uploaded and started through AsyncRemoteWorkspace using the conversation runtime ID.
5. The watchdog checks the detached command, records completion/failure, and releases owned Docker runtime resources.

Security model

• Conversation execution is currently supported only with a configured local Agent Server.
• An agent profile and a direct model selection are mutually exclusive because the profile owns its model configuration.
• Local conversation workers use the configured local Agent Server credential.
• Docker workers receive only a runtime-scoped session key; they do not receive the host Agent Server key or the shared Automation callback key.
• Because restricted conversation workers cannot authenticate completion callbacks, the watchdog is their primary completion path.

Compatibility

• Existing definitions default to execution_scope: "run"; their dispatch behavior is unchanged.
• Existing local and cloud run backends and scheduler behavior are preserved.
• Conversation scope requires agent_profile_id and is rejected outside local Agent Server mode.
• The database migration uses cross-database SQLAlchemy types for PostgreSQL and SQLite.

Validation

Current-head CI is green for:

• unit tests
• backend lint/type checks
• image build
• PR title and description validation
• PR artifact validation

The added tests cover conversation backend behavior, local/Docker credential scoping, dispatch and command verification, watchdog completion and cleanup, cancellation, API persistence, migration behavior, and Git sync round-tripping.

Review focus and pre-merge gates

The architectural review should focus on conversation lifecycle ownership, credential boundaries, watchdog completion, and backwards compatibility for run-scoped automations.

The following items must be resolved before merge:

• Replace the temporary openhands-sdk git pin (4a008f6) and openhands-workspace==1.46.0 compatibility pin with released SDK/workspace versions. The SDK work comes from software-agent-sdk #5046 and #5081; Docker conversation runtimes also require software-agent-sdk #3403.
• Restore the package version in pyproject.toml to the repository's current 1.12.1 value.
• Ensure preset creation and validation consistently couple agent_profile_id with execution_scope: "conversation", so a selected profile cannot be stored and silently ignored by run-scoped dispatch.

Related work

Closes #448. Closes #450. Implements #273.

This is the base of stack #475: #449 → #467 → #468. PR #466 is a parallel child that also depends directly on this PR.

---

This PR description was rewritten by OpenHands AI on behalf of the PR author.

Automated review used the wrong decision (APPROVED instead of COMMENT) and is dismissed. Findings are reposted as a comment.

Automated review used the wrong decision (APPROVED instead of COMMENT) and is dismissed. Findings are reposted as a comment.

Automated review used the wrong decision (APPROVED instead of COMMENT) and is dismissed. Findings are reposted as a comment.

Automated review used the wrong decision (APPROVED instead of COMMENT) and is dismissed. Findings are reposted as a comment.

Automated review used the wrong decision (APPROVED instead of COMMENT) and is dismissed. Findings are reposted as a comment.

[all-hands-bot]

This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.

Summary

This PR introduces conversation-scoped execution for automations, cleanly separating execution_scope ("run" vs "conversation") from agent profile selection. The ConversationBackend wraps LocalAgentServerBackend and adds conversation lifecycle management (creation, runtime credential provisioning, verification, cleanup). The security model is sound — Docker conversation runs receive only the runtime session key, not the host API key, and callback URLs are correctly omitted for conversation-scoped runs when local_api_key is set.

The migration is cross-database compatible with generic SQLAlchemy types and correct upgrade/downgrade ordering. Tests are comprehensive, exercising real SDK transport paths through the sdk_http_transport fixture rather than mock-only assertions.

Findings

1. Preset endpoints accept agent_profile_id but don't set execution_scope

Both POST /v1/preset/prompt and POST /v1/preset/plugin accept agent_profile_id and pass it to the Automation constructor, but neither sets execution_scope. The model defaults to "run", so get_backend() returns a LocalAgentServerBackend (not ConversationBackend), and the profile is silently ignored at dispatch time. The /v1 endpoint correctly sets execution_scope=body.execution_scope. This inconsistency means a user creating a preset with an agent profile will get run-scoped execution and the profile will never be used. See inline comment on preset_router.py:558.

2. validate_agent_profile_selection doesn't validate execution_scope

The validation function checks local mode and model exclusivity, but not that agent_profile_id implies execution_scope="conversation". A user can create an automation via /v1 with agent_profile_id set and execution_scope="run" — the profile is stored but never used at dispatch. Consider either rejecting this combination or documenting that agent profiles require conversation scope. See inline comment on model_profiles.py:55.

3. Git dependency pin (acknowledged)

pyproject.toml replaces openhands-sdk==1.46.0 with a git URL pin at commit 4a008f6a5b7cf38f8fcdf4e11d4a874950cff764. The PR description explicitly calls this out as temporary and says to replace it with a released SDK before merge. This is a first-party package so the 7-day rule doesn't apply, but the pin should be replaced before this ships.

Positive observations

• The ConversationBackend design is clean: conversation ID derivation from subject_key is consistent between the backend property and the dispatcher override, so there's no mismatch risk.
• The watchdog staleness query correctly includes conversation-scoped runs with bash_command_id so they get polled for completion (since they don't use callbacks).
• Docker conversation runs correctly receive only the runtime session key via get_runtime_session_key(), not the host API key.
• The sdk_http_transport test fixture is a well-designed approach to testing SDK transport without borrowing a caller's HTTP client.
• Git sync correctly serializes and validates execution_scope and agent_profile_id, including profile-model exclusivity on import.

Risk Assessment

🟡 MEDIUM — The git dependency pin (unreleased SDK commit) and the preset endpoint design gap are the main risk factors. The security model, migration, and test coverage are solid. The feature is explicitly local-mode only and the base of a stacked PR series.

Verdict

✅ Worth merging — Core logic is sound, security model is correct, and tests are comprehensive. Address the preset execution_scope gap and replace the git pin before or shortly after merge.

---

Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it's merge-ready.

Was this review helpful? React with 👍 or 👎 to give feedback.

[all-hands-bot]

The Automation constructor here sets agent_profile_id but not execution_scope, which defaults to "run". At dispatch time, get_backend() checks run.execution_scope == "conversation" — so a preset created with an agent_profile_id will get a LocalAgentServerBackend, not a ConversationBackend, and the profile will never be used. The /v1 endpoint correctly sets execution_scope=body.execution_scope. Consider setting execution_scope="conversation" when agent_profile_id is provided, or documenting that preset endpoints don't yet support conversation-scoped execution.

[all-hands-bot]

The validation checks local mode and model exclusivity, but not that agent_profile_id implies execution_scope="conversation". A user can create an automation via /v1 with agent_profile_id set and execution_scope="run" (the default), and the profile will be stored but silently ignored at dispatch time. Consider adding a validation that agent_profile_id requires execution_scope="conversation", or at minimum documenting the coupling.

[all-hands-bot]

This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.

Summary

This PR adds conversation-scoped execution to the automation service: definitions can opt into execution_scope: "conversation" with an agent_profile_id, and the new ConversationBackend wraps the existing LocalAgentServerBackend to create/resume an Agent Server conversation, provision runtime credentials (local or Docker), and release the runtime on completion. The watchdog gains polling for callback-less conversation runs, and the dispatcher suppresses callback URLs for restricted backends. The core architecture is sound -- the backend delegation pattern is clean, the conversation ID derivation is correct, and the migration is cross-database compatible.

Blocking Issues

1. Version regression in pyproject.toml

The PR downgrades version from 1.12.1 to 1.12.0 in pyproject.toml, while openhands/automation/__init__.py and openhands/automation/app.py remain at 1.12.1. Per the AGENTS.md release procedure, release-please keeps all four version locations in sync. This mismatch will cause release-please to see 1.12.0 as the current version and generate an incorrect release PR. The pyproject.toml version should match the other files (1.12.1) or be left untouched.

2. Unresolved SDK dependency pin + workspace downgrade

openhands-workspace is downgraded from 1.48.0 to 1.46.0, and openhands-sdk is changed from a PyPI pin (==1.48.0) to a git+URL pin at an immutable integration commit. The PR description acknowledges both are temporary and must be replaced with released SDK versions before merge. This is fine for a stacked PR under active development, but must not merge to main with the git pin or the workspace downgrade in place -- CI that runs uv sync --frozen against the release PR will pull the git dependency, and downstream consumers will be broken.

Non-blocking Observations

Watchdog polls every conversation-scoped RUNNING run on every tick. The mark_stale_runs query now includes (execution_scope == "conversation" AND bash_command_id IS NOT NULL) as an alternative to timeout_at < now. This is the only way to detect completion for callback-less runs, so the design is correct, but at scale it means every watchdog cycle fetches and verifies all active conversation runs regardless of timeout. If conversation-scoped automations become high-volume, consider adding a last_verified_at column or polling interval to throttle verification frequency.

Update endpoint allows execution_scope: "conversation" without agent_profile_id. The validation in validate_agent_profile_selection returns early when agent_profile_id is None, so a PATCH setting execution_scope to "conversation" without a profile passes validation but fails at dispatch time with a clear error. This is acceptable -- the error surfaces early and clearly -- but could be caught at update time for a better user experience.

Risk Assessment

MEDIUM - The core code changes are well-structured and tested, but the version regression and unresolved SDK dependency pin are merge-blockers that would break the release pipeline and downstream consumers.

Verdict

Needs rework - Fix the pyproject.toml version regression and replace the SDK git pin + workspace downgrade with released versions before merging. The conversation execution implementation itself is solid.

---

Improve this review? If any feedback above seems incorrect or irrelevant to this repository, you can teach the reviewer to do better:

1. Add a .agents/skills/custom-codereview-guide.md file to your branch (or edit it if one already exists) with the /codereview trigger and the context the reviewer is missing (e.g., "Security concerns about X do not apply here because Y"). See the customization docs for the required frontmatter format.
2. Re-request a review - the reviewer reads guidelines from the PR branch, so your changes take effect immediately.
3. When your PR is merged, the guideline file goes through normal code review by repository maintainers.

Resolve with AI? Install the iterate skill in your agent and run /iterate to automatically drive this PR through CI, review, and QA until it is merge-ready.

Was this review helpful? React with thumbs_up or thumbs_down to give feedback.

[all-hands-bot]

Blocking: version regression. This downgrades version from 1.12.1 to 1.12.0, while __init__.py and app.py still have 1.12.1. Per the AGENTS.md release procedure, release-please keeps all four version locations in sync -- this mismatch will produce an incorrect release. Restore 1.12.1 here.

[all-hands-bot]

Blocking: dependency downgrade. openhands-workspace goes from 1.48.0 to 1.46.0. Combined with the git+URL pin for openhands-sdk on line 29, these are acknowledged as temporary in the PR description but must be replaced with released PyPI versions before merge to main.

[neubig]

Superseded by the minimal profile-selection PR #479. Conversation reuse now uses the existing Automation KV store plus the Software Agent SDK from the shared Extensions helper (OpenHands/extensions#592), so this PR's execution backend, callback, and subject-turn machinery is no longer needed.