Skip to content

feat(app): render image content parts in the message viewer#31

Open
cacoos wants to merge 1 commit into
raindrop-ai:mainfrom
cacoos:feat/render-image-parts
Open

feat(app): render image content parts in the message viewer#31
cacoos wants to merge 1 commit into
raindrop-ai:mainfrom
cacoos:feat/render-image-parts

Conversation

@cacoos

@cacoos cacoos commented Jun 15, 2026

Copy link
Copy Markdown

Problem

The Workshop message viewer is text-only. messageParsing.ts extracts just type:"text" blocks and discards every other content part, so image parts never render — even though they're present in the raw payload (visible via the Raw toggle).

This bites any agent that feeds the model a screenshot. Concretely, a readPage-style tool appends a user message like:

{ "role": "user", "content": [
  { "type": "text", "text": "This is a screenshot of the page at https://..." },
  { "type": "image", "image": "https://...cloudfront.../read-page-....jpg" }
]}

Workshop shows the text bubble but draws nothing for the image, so it looks like the screenshot was lost when it wasn't:
Screenshot 2026-06-15 at 11 43 01 AM

Fix

Parse image/file content parts into a new optional Message.images and render them as thumbnails (click to open full size).

  • messageParsing.tsextractImageSrc pulls a displayable <img> src out of the common shapes:

    • AI SDK { type: "image", image: <url | data-uri | base64> }
    • AI SDK / provider-normalized { type: "file", mediaType: "image/*", data: <url> } (the shape the AI SDK normalizes image parts to)
    • Anthropic { type: "image", source: { type: "base64" | "url", ... } }
    • bare base64 is wrapped into a data: URI using the block's media type

    Images are collected in both the raw-payload path (parseMessages) and the normalized-span path (messagesFromSpan). Image-only messages (no text) are now kept instead of filtered out.

  • MessageList.tsx — new reusable MessageImages component; MessageBubble renders images under the text and shows an "N images" preview when a message has no text.

  • ChatFlow.tsx — threads images through the user_msg item so appended screenshot messages render in the conversation flow (the surface where this was reported).

Non-image blocks (tool_use, tool_result, text) are unchanged. The new field is optional, so all existing Message consumers are unaffected.

Verification

tsc --noEmit is clean for the changed files. Parser verified against the four payload shapes above (AI SDK image, file/image/*, Anthropic base64 source, and the normalized-span path) — each yields the expected images array while text and non-image parts are preserved, and an image-only message is retained with content: "".

Notes

  • Thumbnails are capped at max-h-64 and wrapped in a link to the full-size source.
  • Data URIs and http(s) URLs are used directly; only bare base64 is wrapped with a media type.

The message parser dropped every non-text content block, so image/file
parts (e.g. the screenshot a readPage-style tool appends as a user
message) were invisible in the chat flow and message-list viewers even
though the data was present in the raw payload.

Extract a displayable img src from image/file blocks across the AI SDK
shapes ({type:'image',image}, {type:'file',mediaType:'image/*',data})
and Anthropic's {type:'image',source}, carry them on Message.images, and
render thumbnails (click to open full size) in both MessageList bubbles
and ChatFlow user messages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant