Skip to content

Spec-compliant resource_link returns + synapse-app UI rebuild#1

Merged
mgoldsborough merged 24 commits intomainfrom
feat/spec-resource-links
Apr 17, 2026
Merged

Spec-compliant resource_link returns + synapse-app UI rebuild#1
mgoldsborough merged 24 commits intomainfrom
feat/spec-resource-links

Conversation

@mgoldsborough
Copy link
Copy Markdown
Contributor

Summary

Two coordinated changes that land together because they depend on each other: the server stops inlining binary artifacts in tool results, and the UI is rebuilt as a proper synapse app that consumes those resources over the ext-apps bridge.

Server side:

  • preview, preview_template, export_pdf, compile_typst return MCP-spec resource_link content blocks instead of inline base64 bytes. Results shrink from multi-MB to ~450 bytes.
  • New collateral://exports/{export_id}.{ext} resource template for rendered PDFs + new collateral://assets/{filename} resource template for user-uploaded assets. Clients read bytes via standard resources/read.
  • Rendering unified on PDF output — preview stops rasterizing to PNGs per page. Browser renders the PDF via the new viewer.
  • export_pdf drops its include_data: bool param (was meaningless once bytes moved to a resource). Breaking change for direct MCP callers; LLM-facing behavior is unchanged.

UI side:

  • Rebuilt as a proper synapse app: <SynapseProvider> wrapping, decomposed into views/ + components/ + hooks/ (mirrors the synapse-crm pattern). Previously a 1298-line monolith that imported synapse hooks but never wrapped with a provider.
  • PDF viewer renders directly to <canvas> via unpdf / PDF.js (no intermediate PNG data URLs). DPR-aware sharpness, ImageBitmap cache for instant zoom-back, render cancellation, float-quantized fit-scale that doesn't flicker at container boundaries.
  • Fully theme-aware: consumes the host's LIGHT_TOKENS / DARK_TOKENS design system. Host tokens injected as CSS custom properties on :root, styles reference via var(--token). Dark mode flips automatically (host re-sends tokens on mode change).
  • Responsive: 1024 / 768 / 640 breakpoints. Left rail + preview stack below 768px; action cluster collapses to a kebab menu; dialogs + settings go full-width on mobile. Respects prefers-reduced-motion.
  • Assets promoted to a first-class tab (third peer of Documents / Templates) with upload (drag-and-drop + picker) and in-app preview.
  • Bytes fetched via synapse.readResource(uri) (shipped in synapse 0.4.3), so the iframe doesn't need to know anything about the host's URL space.

Why

In a 15-hour production session the primary artifact workflow kept crashing because preview, preview_template, and export_pdf returned 1MB-5MB of inline base64 PNG/PDF data, exceeding the engine's 1,000,000-char maxToolResultSize cap:

Tool result too large (4,896,022 chars, limit: 1,000,000). Ask the user to constrain the query or use pagination.

The PDF export is the whole point of the tool. The cap isn't wrong — inlining megabytes of base64 into every tool result is a bad pattern. The MCP spec already offers resource_link content blocks for exactly this: tools return a tiny URI, clients fetch bytes via resources/read.

Wire format

Before:

{
  "content": [
    { "type": "text", "text": "Exported 10-page PDF." },
    { "type": "image", "data": "<base64 MB>", "mimeType": "image/png" }
  ]
}

After:

{
  "content": [
    { "type": "text", "text": "Export of Demo: 10 pages, 1.1MB" },
    {
      "type": "resource_link",
      "uri": "collateral://exports/exp_97aceaf3649231ea.pdf",
      "name": "Export of Demo",
      "mimeType": "application/pdf",
      "annotations": { "audience": ["user"] }
    }
  ],
  "structuredContent": {
    "export_id": "exp_97aceaf3649231ea",
    "page_count": 10, "size_bytes": 1108461, "mime_type": "application/pdf"
  }
}

Total tool-result payload: ~450 bytes regardless of PDF size.

Design decisions

  • Unified on PDF output. Dropped the PNG-per-page raster branch — preview, preview_template, export_pdf, and compile_typst all now call the same PDF compile path and return a single resource_link. The UI renders the PDF in-browser (canvas via PDF.js). Kept tool names distinct for agent clarity even though implementations collapse.
  • Kept existing ui://collateral/preview.pdf for backward compat with any external MCP client relying on it.
  • No embedded UI app for asset bytes. collateral://assets/{filename} is a standard resource template, not a ui:// app. The iframe calls synapse.readResource to fetch bytes; rendering happens in-iframe.
  • Host theme tokens over hand-rolled palette. Killed every hardcoded hex fallback (#2563eb etc.) in favor of var(--color-text-accent) with light-mode fallbacks so standalone file:// renders still work.
  • Canvas rendering over PNG data URLs. unpdf.renderPageAsImage was simple but rasterized to PNG → base64 → <img>. The new path is getDocumentProxy + pdfPage.render({canvasContext, viewport}) directly into a sized <canvas>. Sharper, smaller memory footprint, sets up for a future selectable text layer.
  • Fit scale: both axes + floor + 24px gutter + max-width/height CSS cap. Belt-and-suspenders against container-scrollbar oscillation at the fit boundary.

File-level overview

~30 files changed, +3170 / -1300 lines.

Server (Python):

  • src/mcp_collateral/server.py — tool return rewrites, new resource templates, MIME helper
  • src/mcp_collateral/workspace.pystore_export / load_export, _exports dir + TTL cleanup, PNG compile branch removed
  • src/mcp_collateral/compiler.py — PDF-only
  • src/mcp_collateral/models.py — dead PreviewResult / ExportResult / PagePreview removed
  • tests/test_tool_contracts.py — 10+ new contract tests + asset resource tests

UI (TypeScript/React):

  • ui/src/main.tsx — wrapped in <SynapseProvider>
  • ui/src/App.tsx — thin shell (~250 lines, down from 1298)
  • ui/src/{views,components,hooks,styles}/ — new structure mirroring synapse-crm
  • ui/src/components/PDFViewer.tsx — canvas renderer, zoom controls, page nav
  • ui/src/views/AssetsView.tsx — new first-class assets tab
  • ui/src/styles/responsive.ts — media queries for 1024 / 768 / 640 + dark mode + reduced motion

Test plan

  • uv run pytest tests/76 passing, 10+ new contract tests covering the new return shapes and resource templates (asset + export round-trips, MIME-per-extension dispatch, missing-file handling, directory-traversal refusal)
  • uv run ruff check — clean
  • npm run build in ui/ — clean, final bundle 2053KB / 629KB gzip
  • @nimblebrain/synapse pinned at ^0.4.3 (shipped readResource), installed from registry, readResource confirmed present in the bundled SDK

Not yet covered: UI-level unit tests (follow-up; synapse-crm doesn't have them either — test infra is a separate investment).

Breaking changes

  1. export_pdf loses its include_data: bool param. The old contract returned base64 PDF bytes in the result when this was true; the new contract never does. Direct MCP callers passing this arg will now error on schema validation. LLM-facing behavior is unchanged (the agent doesn't reason about include_data).
  2. preview / preview_template stop returning ImageContent blocks. Anything that was reading result.content[n].data to get a PNG is broken. Replacement: fetch via resources/read on the returned resource_link URI.

If you want to cushion the breakage, happy to add a deprecation pass where the old params are accepted but emit a warning and return empty bytes. Just say the word — default position is to ship the clean cut given this is an internal platform.

Dependencies

  • synapse 0.4.3readResource / readServerResource ✅ merged and released
  • nimblebrain#36POST /v1/resources/read + bridge case + chat renderer. Required for the iframe to work end-to-end in the host (the iframe's synapse.readResource lands on a host endpoint that doesn't exist without it). The collateral server itself works against any MCP client that understands resource_link; the UI specifically needs nimblebrain#36.

preview, preview_template, export_pdf, and compile_typst now write
rendered artifacts to an exports/ directory and return resource_link
content blocks pointing at collateral://exports/<id>.<ext>. Results
stay under 10KB even for multi-page documents, avoiding the 1MB
tool-result cap that crashed the agent loop.

A ResourceTemplate at collateral://exports/{export_id}.{ext} serves
the bytes with the correct MIME type for standard resources/read
access. Old inline base64 returns (PreviewResult.image_base64,
ExportResult.pdf_base64) are gone from the wire format.
The preview tools now return resource_link blocks per the MCP spec; the
UI fetches PNGs and PDFs via the platform's resource proxy instead of
decoding inline base64.
The UI now goes through the ext-apps resources/read bridge instead of
hitting the host's resource URL directly. Preview PNGs and exported PDFs
arrive as base64 blobs over the bridge, get decoded into object URLs,
and are revoked on replacement or unmount.
preview, preview_template, export_pdf, and compile_typst now all call
the same PDF compile path and return a single resource_link. Drops the
PNG rasterization branch in the compiler, the _cached_pngs workspace
state, and the PagePreview/PreviewResult/ExportResult models (unused
once tools return resource_links directly). The UI renders the PDF in
a native iframe viewer — no more per-page images or custom page nav.
Preview pane gains a proper card frame with a compact header, file
icon + truncated document name, and a download action that uses the
existing object URL. Panel padding is fluid via clamp(), collapsing
to edge-to-edge below 640px. Empty/loading/error states live inside
the frame with animated dots on loading. This replaces the hardcoded
minHeight: 600 iframe with a flex-filling layout that respects theme
tokens.

This lands the frame chrome; the iframe viewer inside is about to be
replaced by a unpdf-based renderer in the next commit, but the
surrounding card structure is stable.
Decompose ui/src/App.tsx (~1300 lines) into the synapse-app pattern used
by synapse-crm: thin App shell, views/, components/, hooks/, and a
module-level responsive style injector.

main.tsx now wraps the tree in <SynapseProvider name="synapse-collateral"
version="0.3.3"> under StrictMode. Tool calls, preview blob fetches, and
downloadFile move into dedicated hooks (useDocuments, useTemplates,
usePreview, useExport, useAssets, useBrand). No behavior change; the
iframe preview is retained here and swapped in the next commit.
PreviewPane renders PDFs via unpdf's renderPageAsImage into an <img>
instead of handing the blob URL to a browser iframe. The viewer fits
the page to container width by default, rescales on ResizeObserver,
and caches up to ten rendered (page, scale) data URLs per blob so
repeat zooms are instant.

Controls live in a header strip: zoom − / zoom % / zoom + / Fit. When
a document spans multiple pages, a footer shows Page N of T with prev
and next buttons plus ← / → keyboard shortcuts. The download icon in
the card header still emits a blob URL and keeps its existing behavior.

unpdf 1.6.0 ships the PDF.js v5.6.205 serverless build with the worker
inlined, so the viewer needs no worker URL configuration in the
browser; DOMCanvasFactory is picked automatically.
useCallTool returns a fresh {call, isPending, error, data} object on
every render. Depending on that wrapper in useCallback made the
hook-returned refresh functions change identity every render, which
chained into App.tsx effects and re-fired list_documents repeatedly
until the rate limiter tripped.

Destructure the stable call function (internally useCallback'd) from
each useCallTool result and depend on that instead of the whole
object.
PreviewPane's outer card (with document name + download) was stacking
on top of the PDFViewer's own toolbar. Duplicate chrome. Removed the
outer frame and header; PDFViewer now owns the single card (border,
radius, shadow, bg) and hosts a download link alongside the zoom
controls in its toolbar. PreviewPane reduces to a thin wrapper that
dispatches between status states and the viewer.
Previously Fit computed scale from container width vs. page width,
so a letter-size page filled the viewer horizontally but overflowed
vertically. Switched to height-based fit so the whole page is visible
by default; users can zoom in past that with the + button.
Removed the PDFViewer's own border/radius/shadow and the right
pane's padding so the viewer sits flat against the top-nav and
left-rail borders. No more nested card-in-card; regions are
separated by thin 1px lines only.

The Export PDF button in the top nav is gone — the download
arrow in the viewer toolbar now calls the export_pdf tool via
useExport(), which correctly triggers synapse.downloadFile.
Previously the arrow used the preview blob URL with an <a download>,
which browsers (notably when Content-Disposition is inline) opened
in-tab instead of downloading.
Add useInjectThemeTokens hook that subscribes to the Synapse theme and
writes every token from theme.tokens onto document.documentElement as a
CSS custom property whenever the host updates the mode. Styles can now
consume tokens as var(--color-text-primary) etc., and dark mode flips
automatically when the host re-sends the map.

The legacy useThemeTokens t() helper is preserved as a thin adapter so
existing components compile; it is replaced in the follow-up styling
pass.
Rebuild styles.ts around var(--token, fallback) using the host-injected
design system: Inter + Erode fonts, the text-xs/sm/base + heading-sm
type scale, text-accent/danger/background-primary/secondary/tertiary
colors, border-radius-xs/sm/md and shadow-sm/md/lg scales. Drop the t()
helper pattern — components now read tokens directly as CSS variables
and a small typed token map for the few inline cases.

Buttons move to 32px height with var(--font-text-sm-size), the PDF
toolbar gets 16px icons and breathing room, tabs use text-accent for
the active underline, dialog titles and the logo use nb-font-heading.
Legacy hardcoded #2563eb, #ef4444, rgba(15,23,42,0.06), and the
prefers-color-scheme hover shim are replaced with token-aware values
that follow the host theme automatically.
Expand responsive.ts with breakpoints at 1024/768/640px and a
prefers-reduced-motion pass. At ≥1024 the desktop layout is unchanged.
At <1024 the left rail shrinks to 220px. At <768 the rail and preview
stack vertically (rail ~40vh), buttons grow to 36px touch targets,
the PDF toolbar buttons grow to 40px, and the Save / Save as Template
/ Rename cluster collapses into an overflow kebab menu exposed by the
TopNav. At <640 dialogs go full-width with column-reversed stacked
actions, the Settings slide-over fills the viewport, the logo
collapses to a CS mark, and the zoom percent label hides. Reduced-
motion disables the loading dot animation and zeroes out transitions.
Promote the assets section from a nested Settings panel to a top-level
tab alongside Documents and Templates. The Tab type is extended to
"assets" and the TopNav tablist now renders all three labels.

Add AssetsView (views/AssetsView.tsx) — left rail lists uploaded assets
with extension-badged thumbs, selection opens a metadata pane, per-item
delete button mirrors the document list. New AssetUpload component
(components/AssetUpload.tsx) provides a keyboard-accessible drop zone
with a file-picker fallback, drives uploads through useAssets.upload,
and refreshes the list on completion.

The data path for fetching asset bytes is a TODO — the server does not
yet expose assets via a readable MCP resource template, so inline
preview of images and PDFs is stubbed until a collateral://assets/{name}
resource is added. Once it lands, the pane can render images with <img>
and PDFs via the existing PDFViewer, matching the preview flow.

Remove the Assets section from SettingsPanel, which now contains only
Voice and Components. The Assets tab auto-refreshes on useDataSync
events just like documents and templates.
Server: new collateral://assets/{filename} resource template so the UI
can fetch asset bytes via standard resources/read. MIME map expanded
to cover common image and text types.

UI: AssetsView now renders image assets inline with <img> and PDF
assets via PDFViewer, using synapse.readResource.

UI: removed the Save button. The workspace already auto-saves after
every edit (set_source, patch_source, update_section, set_theme,
create_document) — the button was a no-op.
Fit was computed against container height only. When the scaled page
was wider than the container, a horizontal scrollbar appeared, which
stole vertical pixels, which recomputed fit to a slightly smaller
scale, which removed the scrollbar, which restored the pixels, which
recomputed fit to a larger scale — oscillation.

Three fixes:
- Fit both axes (min of widthFit, heightFit) so the page always fits
  entirely and no scrollbar can ever appear at the fit scale.
- Quantize the computed scale to 1% so sub-pixel clientRect jitter
  (subpixel DPR layout, scrollbar reservation, etc.) can't repeatedly
  cross the equality threshold.
- Coalesce ResizeObserver fires through requestAnimationFrame so a
  burst of resize events during layout only recomputes once per frame.
At the fit boundary the rasterized PNG could be 1-2px taller than the
container because (a) the fit scale rounded to nearest, not down, and
(b) the img had maxWidth: none and no maxHeight, so intrinsic PNG
dimensions drove layout unconstrained.

Belt-and-suspenders fix:
- Floor-quantize the fit scale and bump the gutter from 16 to 24px so
  the image is always strictly smaller than the container.
- Cap the img with maxWidth/maxHeight: 100% in fit mode so even if the
  floor math leaves a 1px residual, CSS absorbs it. User-zoom bypasses
  the cap so the image can overflow and scroll normally.
Previously the viewer called unpdf's renderPageAsImage which internally
rasterizes each page to a PNG and returns a data URL. We then displayed
the PNG in an <img>. Correct in spirit (PDF.js runs in the browser) but
indirect in practice — double copy, base64 bloat, extra decode.

Switched to PDF.js's canonical canvas render path via the same unpdf
getDocumentProxy. The PDF renders straight into a <canvas> 2D context
at DPR-aware resolution (backing store = nativePx * scale * dpr; CSS
size = native * scale). Rendered frames are cached as ImageBitmaps
for instant fit/zoom-back blits, with cache eviction calling close()
so we don't leak GPU memory.

Benefits: no PNG encode/decode per render, sharper on non-integer DPR
monitors, smaller working set, and the code is now set up to add a
selectable text layer later if we want it. Bundle size is essentially
unchanged (-1.7kB raw; the unpdf raster helper we dropped is still in
the bundle because unpdf doesn't tree-shake per-export).
4 regression tests for the new asset template: correct bytes+mime for
a known asset, correct mime per extension for png/jpeg/svg/md/bin,
empty payload for a missing file, and empty payload for a path that
resolves to a directory (prevents accidental directory traversal
reads).

Also picks up a ruff autofix: the 'from . import store, templates'
consolidation split into two imports during format.
- server.py collateral_asset now resolves the candidate path and
  rejects anything outside ASSETS_DIR. Uploads already sanitize via
  _validate_filename, but the resource handler is a separate trust
  boundary (resource reads can target any URI, not only things this
  tenant uploaded) and should enforce containment directly.
- tests/test_tool_contracts.py: new test_asset_resource_rejects_path_traversal
  covering the classic ../../../etc/passwd, a mid-path dot-dot, and
  an absolute /etc/passwd.
- README.md tool list refreshed to match the actual v3 surface
  (was still referencing removed v2 tools like set_template,
  set_content, get_brand). Added a Resources section listing the
  ui://, collateral://, and skill:// namespaces the server exposes.
- CLAUDE.md gains a 'Breaking changes' section explicitly documenting
  the export_pdf include_data removal and the PNG-output removal
  from the render tools, for anyone reading the repo post-merge.
@mgoldsborough
Copy link
Copy Markdown
Contributor Author

Addressed QA review in 185f090.

Critical 1 (path traversal) — fixed. collateral_asset now resolves the candidate path and rejects anything outside ASSETS_DIR.resolve() before reading bytes. The existing upload-side _validate_filename stays — this is defense-in-depth at the resource-handler boundary. On any escape attempt (traversal, absolute path, OS error during resolve) the handler returns an empty payload rather than throwing, matching the existing missing-file behavior.

Suggestion 1 (traversal test) — added. test_asset_resource_rejects_path_traversal covers three vectors: classic ../../../etc/passwd, mid-path sub/../../etc/passwd, absolute /etc/passwd. All return empty content.

Warning 1 (breaking change doc) — added. New Breaking changes section at the top of CLAUDE.md explicitly documents the export_pdf include_data removal and the render tools' PNG-output removal. Covers direct MCP callers who'd otherwise only find the change in the PR body.

Suggestion 2 (stale README) — fixed. README Tools list was referencing v2 tools that no longer exist (set_template, set_content, get_brand, update_brand_colors, set_logo, list_brand_presets, load_brand_preset, get_content, list_sections, get_template_schema, reset_workspace, preview_page). Replaced with the real v3 surface grouped by lifecycle, and added a Resources section listing the ui://, collateral://, and skill:// namespaces.

Skipped:

  • Warning 2 (pre-existing tsc --noEmit error in ui/src/types.ts) — QA flagged not a blocker; this is pre-existing auto-generated Python-syntax leak from the Pydantic → TS codegen, out of scope for this PR.
  • Suggestion 3 (byte-size sanity check in usePreview.ts) — nice-to-have, skipped to keep this PR tight. Happy to land as a follow-up if you want a specific cap.

Verification:

  • uv run pytest tests/77/77 (+1 new traversal test)
  • uv run ruff check — clean
  • cd ui && npm run build — clean, bundle unchanged at 2053KB / 629KB gzip

Ready for re-review. Still gated on nimblebrain#36 for the host-side POST /v1/resources/read endpoint — the collateral server itself works against any spec-compliant MCP client today, but the iframe in nimblebrain needs that endpoint to fetch bytes.

@mgoldsborough
Copy link
Copy Markdown
Contributor Author

nimblebrain#36 is merged — the host-side POST /v1/resources/read endpoint is on main. The iframe path is now end-to-end unblocked (once nimblebrain deploys). This PR has no other external dependencies.

The prior README still listed removed v2 tools (set_template, set_content,
get_brand, preview_page). Replace with the actual v3 surface grouped to
match CLAUDE.md lifecycle (Theme, Templates, Documents, Workspace &
Editing, Assets, Voice & Components, Fonts, Rendering).

Also add a Resources section enumerating ui://, collateral://, and
skill:// URIs the server exposes, and note that rendering tools return
resource_link blocks rather than inline bytes.

The previous "Address QA review" commit claimed this refresh but the
file was never staged.
@mgoldsborough mgoldsborough added the qa-reviewed QA review completed with no critical issues label Apr 17, 2026
@mgoldsborough mgoldsborough merged commit 57a806d into main Apr 17, 2026
@mgoldsborough mgoldsborough deleted the feat/spec-resource-links branch April 17, 2026 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

qa-reviewed QA review completed with no critical issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant