Skip to content

feat(screens): preview layouts through the existing skill - #144

Merged
Lancetnik merged 3 commits into
feat/a2ui-layout-sizingfrom
feat/a2ui-skill-preview
Oct 12, 2026
Merged

Lancetnik merged 3 commits into
feat/a2ui-layout-sizingfrom
feat/a2ui-skill-preview

Conversation

@Lancetnik

Copy link
Copy Markdown
Member

Why are these changes needed?

The assistant could author layouts but could not inspect their rendered appearance. Add one progressively disclosed screens/preview script returning an image and measured component bounds through AG2's typed tool result. It previews saved Screens/instances, Chat drafts and unsaved Card or Screen candidates without saving files, revising drafts or running sources.

Use the same compiled Svelte renderer in a passive preview entry. Collect page overflow, parent escape, horizontal scroll, browser errors and blocked resources. Chromium is selected explicitly per profile in Settings → General; an empty path disables preview. Rendering uses a temporary browser profile, blocks external resource requests, and cleans up browser/server processes. Screenshots are bounded to 4096px height and 8MB; fonts and remote images may differ because external resources are blocked.

Preview exists only in screens; card-author references that script. No global tool or duplicate preview script is registered.

Related issue number

Screen design feedback follow-up. Stacked on #143 in stack #131.

Checks

  • I ran ruff check ., ruff format --check ., and pytest -m "not integration" locally and they pass.
  • If I changed anything under web/, I rebuilt and committed the SPA bundle (npm --prefix web run build).
  • I've included any doc changes needed for this change.
  • I've added or updated tests corresponding to the changes (if relevant).

Validation: 2462 Python tests passed; mypy passed; 557 SPA tests passed; svelte-check reports no errors or warnings. Skill tests cover actual image results through both plugin modes, disabled skills, candidate references, draft edits and read-only behavior. API tests cover browser configuration. Live Chrome: desktop/narrow screenshot rendering and General setting save/reload, with console and network checks. Image tool results also survive WebSocket JSON and a real SQLite round trip.

AI assistance

Implementation and documentation were prepared with Codex assistance and reviewed against the actual diff, protocol results and browser behavior.

  • I understand the changes in this PR and can explain them in my own words.
  • I have verified that this description accurately reflects the actual diff.
  • If AI assistance was used, I reviewed, tested, and validated the generated code/text before submitting.

@Lancetnik
Lancetnik added this pull request to stack #131 October 10, 2026 17:17
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Branch coverage

Name Stmts Miss Branch BrPart Cover
src/assistant/a2ui.py 281 22 84 14 89%
src/assistant/a2ui_skill.py 125 9 36 7 90%
src/assistant/acp/approvals.py 58 1 8 2 95%
src/assistant/acp/chats.py 251 22 56 12 89%
src/assistant/acp/listeners.py 28 1 10 1 95%
src/assistant/acp/serve.py 66 16 4 2 74%
src/assistant/acp/serve_ws.py 103 0 24 3 98%
src/assistant/agent.py 272 31 78 8 88%
src/assistant/attachments.py 57 2 20 0 97%
src/assistant/builtin_tools.py 98 2 36 1 98%
src/assistant/card_author.py 59 16 18 2 64%
src/assistant/card_drafts.py 157 21 60 16 83%
src/assistant/card_instances.py 221 21 78 18 86%
src/assistant/card_sources/runtime.py 77 17 14 2 77%
src/assistant/card_sources/schema.py 91 13 44 13 81%
src/assistant/card_sources/service.py 235 22 102 23 87%
src/assistant/cards/__init__.py 336 30 152 21 90%
src/assistant/cards/layout.py 40 1 18 1 97%
src/assistant/channels/__init__.py 15 7 6 0 38%
src/assistant/channels/base.py 40 3 4 1 91%
src/assistant/channels/discord.py 136 73 44 1 42%
src/assistant/channels/formatting.py 121 2 32 3 97%
src/assistant/channels/router.py 544 15 186 14 96%
src/assistant/channels/slack.py 165 95 52 1 39%
src/assistant/channels/telegram.py 284 51 96 18 79%
src/assistant/cli.py 498 252 92 8 46%
src/assistant/codex_auth.py 241 67 54 5 71%
src/assistant/coding/bridge_client.py 44 7 4 1 83%
src/assistant/coding/bridge_server.py 115 30 22 4 74%
src/assistant/coding/diff.py 98 13 36 6 84%
src/assistant/coding/model_catalog.py 101 6 30 4 92%
src/assistant/coding/session.py 92 7 18 2 92%
src/assistant/config.py 204 22 58 7 89%
src/assistant/connections.py 264 18 78 14 91%
src/assistant/feedback.py 39 5 4 1 86%
src/assistant/filesearch.py 96 12 48 6 88%
src/assistant/folders.py 264 13 82 4 95%
src/assistant/gateway/app.py 470 180 124 19 58%
src/assistant/gateway/core.py 863 104 230 39 85%
src/assistant/gateway/openapi_schema.py 26 0 6 1 97%
src/assistant/gateway/profile_manager.py 410 43 118 23 87%
src/assistant/gateway/repair.py 66 3 26 1 96%
src/assistant/gateway/routes/card.py 122 14 28 6 87%
src/assistant/gateway/routes/card_instance.py 43 3 4 0 94%
src/assistant/gateway/routes/common.py 39 1 16 1 96%
src/assistant/gateway/routes/connection.py 167 2 42 2 98%
src/assistant/gateway/routes/file.py 194 4 72 6 96%
src/assistant/gateway/routes/folder.py 87 7 8 1 92%
src/assistant/gateway/routes/llm.py 252 17 54 6 92%
src/assistant/gateway/routes/permission.py 55 2 12 2 94%
src/assistant/gateway/routes/profile.py 73 4 8 0 95%
src/assistant/gateway/routes/secret.py 66 5 6 1 92%
src/assistant/gateway/routes/settings.py 201 18 26 8 89%
src/assistant/gateway/routes/skill.py 223 9 30 1 96%
src/assistant/gateway/routes/system.py 239 26 42 4 89%
src/assistant/gateway/routes/task.py 104 7 18 5 90%
src/assistant/gateway/stream_bridge.py 37 0 6 1 98%
src/assistant/gateway/tasks_service.py 407 50 130 17 86%
src/assistant/gateway/wire.py 27 4 6 1 85%
src/assistant/hitl/channel.py 21 2 4 2 84%
src/assistant/hitl/desktop.py 133 12 26 5 89%
src/assistant/hitl/gateway.py 22 4 2 1 79%
src/assistant/hitl/inquiry.py 168 6 36 5 95%
src/assistant/integrations/google_auth.py 136 67 24 0 49%
src/assistant/live_configs.py 116 9 40 13 86%
src/assistant/llm_configs.py 221 5 86 1 98%
src/assistant/memory.py 134 11 44 5 89%
src/assistant/middleware.py 81 2 12 2 96%
src/assistant/observability.py 69 5 10 0 94%
src/assistant/observers.py 97 0 24 1 99%
src/assistant/onboarding.py 64 2 26 0 98%
src/assistant/pairing.py 145 2 46 3 97%
src/assistant/peers.py 150 1 40 3 98%
src/assistant/permissions.py 300 21 106 8 93%
src/assistant/profiles.py 186 2 46 4 97%
src/assistant/provider_catalog.py 108 6 46 6 92%
src/assistant/resources.py 210 6 46 8 95%
src/assistant/scheduler_lock.py 33 0 4 1 97%
src/assistant/screen_skill.py 121 22 46 13 75%
src/assistant/screens.py 106 11 34 7 87%
src/assistant/secrets.py 262 4 78 2 98%
src/assistant/self_tools.py 78 21 18 4 70%
src/assistant/settings.py 186 9 54 7 93%
src/assistant/skills.py 55 3 14 0 96%
src/assistant/skills_install.py 167 19 56 12 86%
src/assistant/state_store.py 163 12 50 6 92%
src/assistant/storage.py 36 4 0 0 89%
src/assistant/system_tools.py 152 45 50 13 66%
src/assistant/tasks/scheduling.py 99 6 30 4 92%
src/assistant/tasks/store.py 168 19 56 7 88%
src/assistant/tasks/summary.py 38 1 2 1 95%
src/assistant/tools/__init__.py 88 5 38 2 94%
src/assistant/tools/_mcp_compat.py 25 3 2 1 85%
src/assistant/tools/approval.py 12 6 4 0 38%
src/assistant/tools/ask.py 14 6 2 0 50%
src/assistant/tools/coding.py 30 7 2 1 75%
src/assistant/tools/docker_sandbox.py 68 20 10 1 68%
src/assistant/tools/files.py 70 18 24 6 74%
src/assistant/tools/finance.py 117 44 48 1 61%
src/assistant/tools/google.py 186 132 36 2 28%
src/assistant/tools/image_gen.py 92 38 28 2 55%
src/assistant/tools/mcp.py 195 14 42 9 89%
src/assistant/tools/weather.py 119 19 34 6 82%
src/assistant/tools/web_fetch.py 30 4 10 3 82%
src/assistant/usage.py 74 7 14 0 85%
src/assistant/visual_preview.py 154 113 40 1 22%
src/assistant/voice.py 60 19 12 2 65%
src/assistant/voice_providers.py 72 26 4 0 63%
src/assistant/workspace.py 271 25 88 3 92%
TOTAL 16871 2293 4522 575 85%

49 files skipped due to complete coverage.

@Lancetnik
Lancetnik merged commit f072294 into main Oct 12, 2026
9 checks passed
@Lancetnik
Lancetnik deleted the feat/a2ui-skill-preview branch October 12, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants