Skip to content

feat(desktop): add App Dock sidecar tools - #52164

Closed
gustavomhss wants to merge 121 commits into
anomalyco:devfrom
gustavomhss:feat/app-dock-mcp
Closed

gustavomhss wants to merge 121 commits into
anomalyco:devfrom
gustavomhss:feat/app-dock-mcp

Conversation

@gustavomhss

@gustavomhss gustavomhss commented Sep 29, 2026 •

Copy link
Copy Markdown

Context

Adds desktop-only App Dock tools for sidecar LLM browser control.

What

  • Add App Dock RPC bridge and desktop plugin tools.
  • Require scoped dock permission for every operation.
  • Bind bridge to explicit main-window owner.
  • Stabilize snapshot reads across navigation.

Tested

  • packages/opencode: bun typecheck
  • packages/opencode: bun test src/plugin/app-dock.test.ts --rerun-each=3
  • packages/desktop: bun run test:app-dock:tools
  • packages/desktop: bun run test:app-dock:rpc
  • Mutation probe: restoring maxText || 1500 makes RPC integration fail on explicit maxText: 0.

Risk / Rollback

Large feature diff: 63 files, including generated App Dock corpus docs. Draft pending split/review decision. Revert commits 941082b through afdc50d to roll back.

Gustavo Schneiter and others added 30 commits September 3, 2026 21:15
Adds a Tasks tab to the session side panel (review surface) showing live background work for the current session: subagent cards joined by the task tool's metadata.sessionId (child session id), foreground shell cards, needs-input state from pending permissions, nested (+N) counts, elapsed timers, click-to-open transcript, per-task stop, and auto-open on 0->N. Derives everything from synced reactive stores (no fetching/polling).
Parity test requires every English key present in all locales. Real translations land via the locale sync flow; until then English source stands in (established precedent: 128 identical-EN keys in de.ts).
feat(app): tasks panel for background subagents and shell tools
New optional 'model' param as 'providerID/modelID' (e.g. 'openrouter/deepseek/deepseek-chat'). Resolution: param > agent model > parent model. Provider part selects credentials ambiently (OAuth subscriptions, API keys, custom provider aliases for extra keys). Malformed values fail fast. Docs in task.txt. Tests: forward + reject + unchanged defaults.
When the session carries task rules with model patterns (pattern containing '/'), the resolved provider/model is added to the permission ask patterns and metadata. Otherwise behavior is unchanged (no extra prompt). Denied models surface the session ruleset so the caller can retry with a listed model.
DialogSubagentModels mirrors DialogManageModels (Dialog + searchable grouped List + per-model Switch, live-apply). Reads effective state from session permission rules, appends allow/deny task rules via session.update (server merges, last match wins; first constraint writes the star-slash-star deny baseline). Opened via tasks.models command. English fallback across locales; real translations land via locale sync.
Adds the subagent model picker next to the session model and reasoning selectors: session model, reasoning level (when present), subagent models. New subagentModelsControl slot in session-ui PromptInput; app-side control mirrors the model button styling and opens DialogSubagentModels.
Adds three optional lifecycle hooks to the plugin system, mirroring
Claude Code's PreToolUse/PostToolUse/SessionStart/SessionEnd/Stop
surface for the cases OpenCode was missing.

Events:
- session.start: fires when a session is created, before any tool
  calls. Plugin may tag the session via output.metadata (reserved
  keys blocked).
- stop: fires at end of every agent turn. If the plugin sets
  output.continue = true and reason === 'completed', the loop
  re-iterates with the same context. Honored only on 'completed' to
  prevent infinite loops on aborted or errored turns.
- session.end: fires before session deletion, hard-capped at 5s.
  output.cleanup = false defers forced cleanup within the ceiling.

Implementation:
- packages/plugin/src/index.ts: adds three optional fields to the
  Hooks interface (purely additive, no breaking change to existing
  plugins).
- packages/opencode/src/session/session.ts: emits session.start
  inside createNext; emits session.end inside remove with timeout
  and error isolation. Plugin.node is wrapped in a lazy Proxy to
  avoid a TDZ error from the session<->plugin circular import.
- packages/opencode/src/session/prompt.ts: emits stop inside the
  inner 'result === stop' branch where the runLoop is about to
  break. Honors output.continue by returning 'continue' instead of
  'break' so the while loop re-iterates.

Tests (6 new, all passing in 4.4s):
- packages/opencode/test/plugin/trigger-lifecycle.test.ts: 5 tests
  using a real plugin file loaded via opencode.json. Verifies that
  session.start, session.end, and stop receive the documented input
  shapes; that session.end output.cleanup and stop output.continue
  mutations propagate; and that stop output defaults to
  continue=false when the plugin does not touch it.
- packages/opencode/test/session/lifecycle-hooks.test.ts: 1 E2E test
  that creates a session via Session.Service.create and verifies
  the loaded plugin's session.start hook fires with the correct
  sessionID, cwd, and timestamp.

Docs:
- packages/docs/plugins.mdx: full reference for all three hooks
  (input/output tables, error-handling contract, end-to-end
  telemetry example). Added to docs.json navigation under a new
  'Plugins' group.

Error handling: every emit is wrapped in Effect.catch that logs and
continues; a throwing plugin never breaks session creation, deletion,
or the agent loop. The 5s session.end timeout is enforced via
Effect.timeout.

Co-authored-by: gustavoschneiter <gustavoschneiter@users.noreply.github.com>
* feat(core): multiple labeled API keys per provider

* feat(providers): resolve labeled API keys at runtime in v1

* fix(providers): address cold review findings

- resolve getProvider through ensureProvider so keys added post-boot
  don't race a missing provider entry (crash on first request)
- re-read credential state in ensureProvider to pick up remove/update
- fail loudly in v2 runner when a virtual credential is missing instead
  of silently routing to a different active key
- respect provider.disabled for virtual provider availability
- redact API keys from the public provider list response
- honor disabled/enabled provider sets in the post-boot catalog fallback
- route vars/model loaders to the base provider for virtual ids

* fix(app): persist labeled keys via v2 credential route

connect.key was overridden to write legacy auth.json, dropping the label
and never creating a credential row. Try the v2 integration connect.key
(which stores a labeled credential) first, falling back to the legacy
auth write only when the server lacks the v2 route.

* test(app): cover v2-first connect.key with legacy fallback

* fix(providers): address second cold review round

- server-compat connect.key falls back to legacy auth only on missing-route
  (404/405), re-throwing real credential failures (400/401/500) instead of
  silently writing legacy auth on top of a broken key
- getProvider now fails typed with NoProvidersError; update the existing
  provider test that asserted the old undefined-return, and hoist the
  getProvider error channel in the Service interface so consumers handle
  the typed failure
- fix parseVirtualID usage in vars/model loader lookup (baseID, not base)
  and type the credential id lookup

---------

Co-authored-by: Gustavo Schneiter <gmhschn@gmail.com>
Composer button and dialog now work on drafts: toggles accumulate in a draft-scoped pending set, flushed into session permission rules right after session creation and before the first prompt is sent (sequential, no race). Button visible in both V2 and legacy composers; sessionID threaded via props.
…tion

describeTask appends the session's allowed provider/model patterns (same last-match-wins semantics as enforcement) so the model sees exactly what it may pass as task model, plus a pointer to the parameter. No session rules means no section (unchanged behavior).
Child cards are now keyed exclusively by child session id (children() authoritative, task tool parts enrich or fall back to orphans) — fixes duplicate agent cards. childStats reads the promise-client message shape (type: assistant, model: {id, providerID}) alongside the legacy flat fields — restores per-card model + tokens. Headlines strip the ' (@agent subagent)' suffix.
feat: subagent model allowlist + tasks panel polish
Co-authored-by: Gustavo Schneiter <gmhschn@gmail.com>
* refactor(opencode): lazy-load CLI commands to cut startup cost

The entrypoint statically imported every command module (server, session,
sdk, config schemas) at process start, costing tens of seconds and ~200MB
RSS even for fast paths like --version. Register commands through a
delegating module that only loads the real implementation when yargs
parses an invocation that exercises it.

Measured: --version instructions 10.4B -> 5.6B (-46%), peak RSS
224MB -> 132MB (-41%), user CPU 3.4s -> 1.9s (-44%).

Also decouple UI.cancelled-error from the effect import so the ui module
loads without pulling in the full effect runtime.

* perf(core): skip durable event log for message/part updates in local mode

updateMessage/updatePart published full snapshot events to the durable
event table on every change, duplicating the complete payload per update.
A single 23MB message re-rendered 90x wrote ~2GB of redundant JSON to the
event log (the user's opencode.db grew to 7GB, 6.4GB of it the event
table). The only consumer of these durable rows is the experimental
workspaces sync (off by default); the local UI/SSE/projection read the
projected message/part tables.

Add PublishOptions.persist (default true = unchanged). updateMessage/
updatePart pass persist: experimentalWorkspaces so local installs stop
appending to the event log while still projecting and notifying. With
workspaces enabled the behavior is byte-identical to before. The bridge
only emits sync envelopes for events carrying a durable marker, so local
events are never observed by cross-instance sync.

Measured: write-path E2E (90x growing part up to 24MB) 10.2s -> 5.5s
(-46%); publish micro-bench -92%; durable bytes per turn 100% eliminated.

Add db compact command as an optional one-shot tool to reclaim rows
already written (guarded against experimentalWorkspaces, which requires
contiguous seq for sync replay).

* refactor(opencode): type lazyCommand without any

* test(core): drop timing-based persist benchmark

The publish timing bench was flaky under machine load and duplicated the
behavioral coverage already provided by event-persist-gate. The measured
speedup (write-path E2E -46%, publish micro-bench -92%) is captured in the
perf commit message instead of as a timing assertion.

* docs(core): explain persist:false seq placeholder and commit hook contract

Reviewers need the invariant spelled out: the local-only publish path
supplies an inert seq placeholder to projectors (none read it) and skips
commit hooks by design (no caller combines them). Adds the rationale for
gating message/part persistence on experimentalWorkspaces in session.ts.

* feat(opencode): add native HuGR composer tools

---------

Co-authored-by: Gustavo Schneiter <gmhschn@gmail.com>
Adds 'opencode debug read --params <json>' to invoke the legacy read tool directly without an LLM. Emits a machine-readable envelope: params, latency ms, title, metadata (incl. telemetry source/size/lines_read/saved) and output. Failures come back as ok:false with the error. Runs under Effect.scoped so LSP deps resolve.
bench-read.sh runs N invocations of 'opencode debug read' across whole/symbol/tail modes against a target file and emits CSV (mode,runs,avg,p50,p95,size,lines_read,saved,source). Supports BENCH_SYMBOL override and --json for full envelopes. Bash-only (no %f printf; stats via python).
…ark harness

findSymbolByIndent and tail line-count now share a path+mtime+size memo scan cache, fixing two defects the benchmark exposed: (1) symbols past line 2000 fell back to an unbounded scan every call even warm; (2) tail's negative-offset line count re-read the full file uncached (~1s on 15k-line files). Scan cache is bounded at 200k lines. debug read now accepts a params ARRAY (warm in-process runs), --meta-only (envelope without output payload for benchmark-safe huge reads), and emits envelopes with write-flush. Ship bench-corpus.sh (3 sizes x 2 densities), bench-hardcore.sh (cold+warm matrix over whole/symbol/tail/depth), bench-report.py (latency, output-bytes deltas, resolution source, slice economics).
…view

Two hardening deltas from the cross-market comparison (Claude/Codex/Grok), benchmark-validated against the corpus:

1. Unresolvable symbol now DEGRADES to a normal file read with a <system-note> instead of failing — the model never pays a second round-trip on a probe name. Symbol resolution contract unchanged on the happy path (LSP → indentation → render); failure path switched from hard error to graceful continuation.

2. Token budget (25k est, ~2 bytes/token conservative bound) cuts dense content earlier than the 50KB byte cap with a 'PARTIAL view' footer + next offset, so token-dense reads never dump a context bomb (Grok parity). Code ASCII fires the gate co-incident with the byte cap; tests updated: symbol-not-found now asserts degradation; new PARTIAL-view truncation test.

Benchmark (v2, same corpus): no regression — symbol slice economics identical (15,005/15,009 lines saved large-dense; -97..-99% output), resolution accuracy unchanged, cold→warm unchanged.
…g + metaprompt

Extends the read tool with the async-resolve capabilities recommended after the cross-market comparison + RL adoption research:

- search='<fragment>' — find a symbol by name fragment and read ONLY that range, auto-resolving via LSP → indentation heuristic → degrade, so the model can request a precise slice without knowing the exact symbol name and without paying a second round-trip on a miss. Shares the resolveRange path with symbol; source reported.
- expand: sparse=true numbering (anchor 1 + every 10th, Grok-style) opt-in to cut numbering tokens on huge reads; default unchanged.
- read.txt metaprompt: instructs models to prefer symbol/search over whole-file reads when a name/fragment is known, since the tool degrades safely on a miss — nudging RILI(RL) adoption without changing the contract or adding decision friction.

Tests 45/45 (new: search-by-fragment, sparse anchors). Benchmark v3: search delivers -99% output on dense files (58,470 → 469 chars), consistent with symbol; no regression on whole/tail.
Gustavo Schneiter added 22 commits September 9, 2026 16:35
# Conflicts:
#	packages/opencode/src/session/compaction.ts
Remove status===running guard in PtyHttpApi.get handler so exited
PTY sessions with their exit code are returned by the API.

This enables the PTY test to observe exit 4 → status: exited, exitCode: 4.

Co-authored-by: opencode
- Add app-dock-rpc.ts: bridge between sidecar and Electron main process for dock_* ops
- Add app-dock-browser.ts: injected scripts (snapshot, click, type, scroll, navigate, etc.)
- Add app-dock-api.ts: TypeScript interface for AppDock
- Add app-dock-tools.test.ts, app-dock-rpc.test.ts, app-dock-youtube.test.ts, app-dock-live.test.ts
- Extend app-dock.ts with execute(), scroll(), hover(), drag(), clickAt(), scrollTo(), storage(), evaluate(), network(), pdf(), list(), closeAll()
- Register RPC bridge in ipc.ts with profile resolver and new IPC handlers (app-dock-list, etc.)
- Update server.ts to accept sidecarPath and forward messages for RPC
- Add preload API: appDockList, tab-selected event
- Add opencode plugin: app-dock.ts with 16 dock_* tools (open, read, click, type, navigate, go, list, close, scroll, wait, screenshot, keyboard, evaluate, storage, network, activate)
- Add corpus docs: invariants, requirements, goldens, cards for app-dock
Add AppDockPlugin to internalPlugins when flags.client === 'desktop'
so the sidecar loads dock_* tools when running in desktop context.
…Script, buildRetryScript)

- Fix typo: buildPDFSript -> buildPDFScript
- Fix buildFrameScript: cross-origin iframe access handling
- Fix buildRetryScript: use string concat to avoid TS parser bug with template literals
- Update CI workflow to remove non-existent app-dock-test-support.ts
- Convert module singletons to AppDockRPC class with explicit init/reset
- Add resetAppDockRPC() export for test isolation
- Fix handleDockRPC to use proper promise chain (no async IIFE leak)
- Remove debug logging from app-dock-tools.test.ts
- Add 60s watchdog in child process to prevent hangs
- Add retry logic for read after reload to handle namespace stabilization
- Increase wait times after reload (500ms) and before fullscreen read (300ms)
- Window focus is now more reliably handled with registerAppDockWindow() call
@github-actions github-actions Bot added the needs:compliance This means the issue will auto-close after 2 hours. label Sep 29, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

  • PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 2 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

@gustavomhss
gustavomhss marked this pull request as ready for review September 29, 2026 20:26
@gustavomhss

Copy link
Copy Markdown
Author

Opened against wrong upstream repository; closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs:compliance This means the issue will auto-close after 2 hours.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant