Summary
Enrich the Voice Workspace canvas (the agent-controlled panel beside the voice orb) and polish the orb animation. Today the canvas supports only markdown and html panel types and overwrites itself on every update; the orb pulses directly off a raw amplitude value with no temporal smoothing. This adds Mermaid diagrams, image display, and client-side panel history, smooths and enlarges the orb, and adds a graceful transition when the canvas updates.
These are borrowed from a local prototype of the same voice-workspace concept that has a richer panel renderer and a smoother orb.
Context
Trinity already ships voice workspace mode (#699/#860): src/backend/services/gemini_voice.py declares four panel tools (show_markdown, update_panel, append_to_panel, clear_panel) and src/frontend/src/views/AgentWorkspace.vue renders the panel (markdown via DOMPurify, HTML via a sandboxed iframe with Chart.js bundled) on a 300 ms poll of GET /api/agents/{name}/voice/{session_id}/panel.
Gaps vs. the prototype:
- No diagrams —
mermaid is not a dependency and there is no show_diagram tool. For a "thinking whiteboard," flowcharts / sequence / mindmap / timeline diagrams are arguably higher-value than the Chart.js charts already supported.
- No images — the agent cannot show a generated chart, screenshot, or diagram asset.
- No history — the panel is overwritten on every update; once the agent redraws, the previous "whiteboard" is gone.
- Jerky orb —
renderFrame reads voice.amplitude raw each frame (no smoothing) and the core size 45 + energy*20 is computed inline (also unsmoothed), so the orb jumps; it also goes flat when silent (no idle "breathe").
- Abrupt canvas swaps — new panel content replaces old with no transition.
Scope
1. Mermaid diagrams (top borrow)
- Add a
show_diagram(diagram, title?) panel tool and a mermaid panel type.
- Render the diagram inside the sandboxed iframe (bundle
mermaid the same way Chart.js is injected), keeping SVG injection out of the parent DOM / DOMPurify path and consistent with the existing security model.
- Follow-up (optional, deferred): auto-render
```mermaid fenced blocks inside show_markdown — messier because markdown renders in the parent DOM via DOMPurify.
2. show_image(src, title?, caption?)
- Web URLs render directly; agent-workspace file paths are served through the existing file-download endpoint so the backend streams the bytes (no widening of the iframe sandbox, no arbitrary local-FS read). Validate the path is workspace-confined.
3. Client-side panel history navigation (frontend-only, no backend change)
- Keep a ring buffer of the last ~40 panel snapshots; add prev/next + a dropdown selector in the panel header. "Live" follows the latest; navigating back pins a snapshot until the user returns to live (or a brand-new update arrives).
4. Orb polish — less jerky, slightly larger, larger soft-glow amplitude
- Smooth
energy with a persistent value lerped toward the target using asymmetric attack/release (attack ~0.18, release ~0.10) instead of using the raw amplitude each frame.
- Smooth the core size with a persistent
coreSize lerped toward targetCoreSize rather than computing 45 + energy*20 inline.
- Add a low-amplitude idle "breathe" floor so the orb gently moves at rest.
- Enlarge: bump the core base (~45 → ~58) and/or the global spread/size (1.1 →
1.25); increase the energy→core-size coupling (*20 → ~*32) for a larger soft-glow swing.
5. Graceful canvas-update transition
- Cross-fade / subtle slide-in of new panel content, a brief "updated" flash on the header timestamp, and (optionally) a thin top loader bar. Respect
prefers-reduced-motion.
Acceptance Criteria
Technical Notes
- Backend:
src/backend/services/gemini_voice.py — _PANEL_TOOLS, _PANEL_TOOL_NAMES, _execute_panel_tool, WORKSPACE_PANEL_INSTRUCTIONS. Panel tools execute in-process (no agent container call), so new types are cheap to add.
- Frontend:
src/frontend/src/views/AgentWorkspace.vue — panel render branch (panelState.type), the iframe srcdoc builder (where Chart.js is already injected — add mermaid the same way), the poll handler (history ring buffer hook), and the orb renderFrame / drawCore loop (smoothing, size, glow).
- Frontend deps:
src/frontend/package.json already has chart.js, marked, dompurify — add mermaid.
- Endpoint contract for the panel is unchanged (
GET /api/agents/{name}/voice/{session_id}/panel in src/backend/routers/voice.py); items 3–5 are frontend-only.
- Keep the security posture: agent HTML/diagrams render only inside the opaque-origin sandboxed iframe; do not move agent-supplied markup into the parent DOM.
Out of Scope
- Server-push for panel updates (stays on the existing 300 ms poll).
- In-markdown
```mermaid auto-rendering (tracked as the optional follow-up above).
Summary
Enrich the Voice Workspace canvas (the agent-controlled panel beside the voice orb) and polish the orb animation. Today the canvas supports only
markdownandhtmlpanel types and overwrites itself on every update; the orb pulses directly off a raw amplitude value with no temporal smoothing. This adds Mermaid diagrams, image display, and client-side panel history, smooths and enlarges the orb, and adds a graceful transition when the canvas updates.These are borrowed from a local prototype of the same voice-workspace concept that has a richer panel renderer and a smoother orb.
Context
Trinity already ships voice workspace mode (#699/#860):
src/backend/services/gemini_voice.pydeclares four panel tools (show_markdown,update_panel,append_to_panel,clear_panel) andsrc/frontend/src/views/AgentWorkspace.vuerenders the panel (markdown via DOMPurify, HTML via a sandboxed iframe with Chart.js bundled) on a 300 ms poll ofGET /api/agents/{name}/voice/{session_id}/panel.Gaps vs. the prototype:
mermaidis not a dependency and there is noshow_diagramtool. For a "thinking whiteboard," flowcharts / sequence / mindmap / timeline diagrams are arguably higher-value than the Chart.js charts already supported.renderFramereadsvoice.amplituderaw each frame (no smoothing) and the core size45 + energy*20is computed inline (also unsmoothed), so the orb jumps; it also goes flat when silent (no idle "breathe").Scope
1. Mermaid diagrams (top borrow)
show_diagram(diagram, title?)panel tool and amermaidpanel type.mermaidthe same way Chart.js is injected), keeping SVG injection out of the parent DOM / DOMPurify path and consistent with the existing security model.```mermaidfenced blocks insideshow_markdown— messier because markdown renders in the parent DOM via DOMPurify.2.
show_image(src, title?, caption?)3. Client-side panel history navigation (frontend-only, no backend change)
4. Orb polish — less jerky, slightly larger, larger soft-glow amplitude
energywith a persistent value lerped toward the target using asymmetric attack/release (attack ~0.18, release ~0.10) instead of using the raw amplitude each frame.coreSizelerped towardtargetCoreSizerather than computing45 + energy*20inline.1.25); increase the energy→core-size coupling (*20→ ~*32) for a larger soft-glow swing.5. Graceful canvas-update transition
prefers-reduced-motion.Acceptance Criteria
show_diagram(diagram, title?)is declared as a panel tool, added to_PANEL_TOOL_NAMES, handled in_execute_panel_tool(setspanel_state.type = "mermaid"), and documented inWORKSPACE_PANEL_INSTRUCTIONS.mermaidis a frontend dependency and amermaid-type panel renders a valid diagram inside the sandboxed iframe; invalid syntax shows a contained error, not a broken panel.show_image(src, title?, caption?)displays a web-URL image and an agent-workspace file-path image (served via the existing download endpoint); paths outside the workspace are rejected.prefers-reduced-motion.Technical Notes
src/backend/services/gemini_voice.py—_PANEL_TOOLS,_PANEL_TOOL_NAMES,_execute_panel_tool,WORKSPACE_PANEL_INSTRUCTIONS. Panel tools execute in-process (no agent container call), so new types are cheap to add.src/frontend/src/views/AgentWorkspace.vue— panel render branch (panelState.type), the iframesrcdocbuilder (where Chart.js is already injected — addmermaidthe same way), the poll handler (history ring buffer hook), and the orbrenderFrame/drawCoreloop (smoothing, size, glow).src/frontend/package.jsonalready haschart.js,marked,dompurify— addmermaid.GET /api/agents/{name}/voice/{session_id}/panelinsrc/backend/routers/voice.py); items 3–5 are frontend-only.Out of Scope
```mermaidauto-rendering (tracked as the optional follow-up above).