Skip to content

Add image support to ACP agent message handling#196

Open
Glavin001 wants to merge 2 commits intorivet-dev:mainfrom
Glavin001:claude/add-image-support-sandbox-MdKa0
Open

Add image support to ACP agent message handling#196
Glavin001 wants to merge 2 commits intorivet-dev:mainfrom
Glavin001:claude/add-image-support-sandbox-MdKa0

Conversation

@Glavin001
Copy link

Summary

This PR adds support for handling image content in ACP (Agent Communication Protocol) agent messages. Previously, the opencode adapter only handled text and thought chunks; now it properly processes image chunks by converting them to file parts with appropriate MIME types and URLs.

Key Changes

  • Image chunk handling: Added logic to detect and process image content type in agent message chunks, extracting base64 data, MIME type, and optional URI
  • Text finalization: Before emitting an image part, any accumulated text is finalized and persisted to ensure proper message ordering
  • File part creation: Images are converted to file parts with:
    • Proper MIME type detection
    • Support for both embedded base64 data (as data-URIs) and external URIs
    • Automatic filename generation based on image type
  • MIME type mapping: Added mime_extension() helper function to map MIME types to file extensions (png, jpg, gif, webp, svg)
  • Event persistence: Image parts are persisted to the session transcript before being emitted
  • Capability flags: Updated ACP agent capabilities to reflect support for images and error events in the session transcript schema

Implementation Details

  • Image chunks are handled in a new match branch within the existing agent_message_chunk / agent_thought_chunk handler
  • Text accumulation is preserved for non-image content, maintaining the existing streaming behavior
  • The implementation follows the same event emission and persistence patterns as existing text handling
  • Data-URI encoding is used as a fallback when no external URI is provided, allowing images to be embedded directly in the transcript

https://claude.ai/code/session_01Bjpf7tV6JPiVun7isaModz

Enable sending images to Claude Code via the Sandbox Agent. The
claude-code-acp adapter already supports images in ACP prompt content
blocks, but the Sandbox Agent was not advertising the capability and
the OpenCode compatibility layer was silently dropping image content
from session update notifications.

Changes:
- Set `images: true` in Claude agent capabilities (support.rs)
- Handle ACP image content blocks in translate_session_update, converting
  them to OpenCode-compatible file parts with data-URI or external URLs
- Update coverage matrix docs to show Claude images as supported (✓)

https://claude.ai/code/session_01Bjpf7tV6JPiVun7isaModz
…matrix

Audit of claude-code-acp adapter revealed two matrix inaccuracies:

- Error Events: was `-` (native but not converted), should be `✓`.
  JSON-RPC errors from Claude Code (RequestError.internalError, auth
  errors) are already translated to `session.error` SSE events by the
  opencode-adapter (line 3829-3840).

- File Attachments: was `-` (native but not converted), should be blank.
  Claude Code does not support file attachments natively at all --
  claude-code-acp explicitly ignores blob resources with
  "Ignore blob resources (unsupported)" comment.

Also set error_events: true in support.rs AgentCapabilities for Claude.

https://claude.ai/code/session_01Bjpf7tV6JPiVun7isaModz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants