Muxll

A CVM-based LLM router. Operators plug in upstream LLM providers (Anthropic, OpenAI, …) and Muxll serves their models over the CVM JSON-RPC interface — MCP-over-Nostr — so any Nostr-native client can call them. Streamed responses ride CEP-41 open-ended streams.

This repository currently holds a proof of concept: chat completions (with and without streaming) and model listing. Payments, quotas, and provider config files are intentionally out of scope for now.

How it works

Nostr client ──CVM/MCP JSON-RPC──▶ Muxll CVM server ──▶ pi-ai ──▶ upstream LLM
                 (Nostr relays)      McpServer + NostrServerTransport

• pi-ai (@earendil-works/pi-ai) is the unified provider layer: one streaming API across many providers, with built-in model catalogs.
• CVM SDK (@contextvm/sdk) exposes the MCP server over Nostr via NostrServerTransport, with CEP-41 open streams enabled for token-by-token streaming.
• MCP SDK (@contextvm/mcp-sdk) provides the McpServer that registers the RPC tools.

RPC interface

Two MCP tools are exposed (called via the standard tools/call method):

models.list

Returns every model across configured providers. Shape follows OpenAI GET /models (plus router-useful capability fields).

// request arguments
{}

// structuredContent
{ "object": "list", "data": [
  { "id": "anthropic/claude-opus-4-5",   // provider/id tag, sent back as `model`
    "object": "model", "created": 0, "owned_by": "anthropic",
    "name": "Claude Opus 4.5", "api": "anthropic-messages",
    "contextWindow": 200000, "maxTokens": 8000,
    "reasoning": true, "input": ["text", "image"] }
  // …
] }

chat.complete

// arguments
{
  "model": "anthropic/claude-opus-4-5",  // provider/id tag (bare id = first match)
  "messages": [
    { "role": "system", "content": "You are helpful." },
    { "role": "user", "content": "Hello" }
    // assistant turns may carry `tool_calls`; `tool` turns carry
    // `tool_call_id` + `content` (the result) to close a tool loop.
    // user `content` may be an array of {type:"text"} / {type:"image_url"}
    // parts for multimodal input (data-URL or http(s) images).
  ],
  "tools": [                       // optional, OpenAI function-tool shape
    { "type": "function",
      "function": { "name": "get_weather",
                    "description": "Get current weather",
                    "parameters": { "type": "object",
                                    "properties": { "location": { "type": "string" } } } } }
  ],
  "tool_choice": "auto",          // optional: "auto" | "none" | "required" | {type:"function",function:{name}} (normalized per provider)
  "stream": false,                  // true → stream chat.completion.chunk objects over CEP-41
  "temperature": 0.7,               // optional
  "max_tokens": 1024                // optional
}

// structuredContent (non-streaming) — OpenAI CreateChatCompletionResponse
{
  "id": "chatcmpl-…", "object": "chat.completion", "created": 1730000000,
  "model": "claude-opus-4-5",
  "choices": [
    { "index": 0,
      "message": { "role": "assistant", "content": "Hi there!" },
      "finish_reason": "stop" }
  ],
  "usage": {
    "prompt_tokens": 12, "completion_tokens": 3, "total_tokens": 15,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0, "total": 0.0000045 }
  }
}

When tools are provided, a tool-requesting turn comes back with finish_reason: "tool_calls", message.content: null, and a message.tool_calls array ({id, type:"function", function:{name, arguments}}); streamed as delta.tool_calls fragments ({index, id, type, function:{name}} then {index, function:{arguments}}). Send each result back as a tool role message to continue.

When stream: true, the client must include an MCP progressToken. Each CEP-41 chunk frame carries one JSON-serialized OpenAI chat.completion.chunk object (delta.content for text, delta.reasoning_content for thinking, then a final chunk with finish_reason + usage). The stream is the response — the closing JSON-RPC tool result carries only metadata (id, model, finish_reason, usage), never the assembled text. cost rides inside usage (OpenRouter's usage.cost convention) so it also appears in the final stream chunk. A bridge to OpenAI SSE wraps each chunk as data: <chunk>\n\n plus a trailing data: [DONE].

Install

Requires Bun ≥ 1.2.

bun install
cp .env.example .env   # then edit: set provider keys and (optionally) a stable server key

Run

bun start

On startup the server prints its Nostr public key and relays. Clients target that pubkey over the configured relays. Set MUXLL_ANNOUNCED=1 to publish public discovery announcements.

CLI

@muxll/cli is the reference command-line client over @muxll/client, built on @earendil-works/pi-tui. Point it at a running server's pubkey and relays (env or flags):

export MUXLL_SERVER_PUBKEY=<pubkey printed by `bun start`>
bun run cli models
bun run cli chat --model anthropic/claude-sonnet-4-5        # full-screen TUI (Ctrl+C to quit)
bun run cli chat --model anthropic/claude-sonnet-4-5 "hi"   # one streamed turn to stdout

chat with no prompt opens an interactive full-screen chat — streamed markdown replies and a status bar with live usage (prompt/completion tokens, cost, turns). chat with a prompt prints one streamed turn to stdout, for scripting. MUXLL_RELAYS (default wss://relay.contextvm.org) and MUXLL_MODEL are optional; --pubkey/--relays/--model/-m override per invocation.

Test

Integration tests run the full server + client over an in-process mock Nostr relay using pi-ai's fauxProvider (no real API keys or network needed):

bun run test        # runs `bun test packages/*/tests`
bunx tsc --noEmit   # typecheck the whole workspace

Bare bun test (or bun test packages/) is too broad: it also scans the vendored docs/ tree and follows the workspace node_modules symlinks into Bun's package store, running dependency test suites. bun run test scopes to packages/*/tests, which holds every project test.

Project layout

A Bun workspace monorepo (see tsconfig.json paths — packages import each other by @muxll/* name and resolve to source, so dev needs no build step):

packages/
  core/    @muxll/core     OpenAI wire shapes (zod schemas + types), shared by server + clients
  server/  @muxll/server   the CVM/Nostr adapter operators run
    src/main.ts           env-driven startup: relays, signer, provider registry
    src/server.ts         McpServer, tool registration, CEP-41 transport wiring
    src/wire.ts           OpenAI ↔ pi-ai translation (transport-agnostic)
    tests/                end-to-end: models.list, chat.complete (both modes), error path
  client/  @muxll/client   reference CVM client (transport + tool callers + CEP-41 stream parser)
    tests/                drives the server over a mock relay via MuxllClient
  cli/     @muxll/cli      command-line client over @muxll/client (models list + streaming chat REPL)
    tests/                parseArgs + streamTurn driven over a mock relay
docs/                 vendored references (cordn, routstr-core, yalr, cvm + pi docs)
                      — read-only, excluded from build and tests

The shared wire shapes (@muxll/core) are the seam: the server validates inputs and builds outputs against them, clients build inputs and parse outputs against them. @muxll/client is the programmatic surface the CLI (@muxll/cli), HTTP proxy, and web app build on, and the one the test harness drives.

Status & roadmap

Proof of concept. Deliberately deferred until the core is validated:

• Provider config file — currently env-var-only built-ins (builtinModels() reads ANTHROPIC_API_KEY, OPENAI_API_KEY, …). Add a models.json loader for custom base URLs / providers.
• Stream cancellation — a client CEP-41 abort does not yet cancel the upstream provider stream.
• Structured stream chunks — text_delta, thinking_delta, and tool-call deltas are all streamed as chat.completion.chunk deltas (delta.content, delta.reasoning_content, delta.tool_calls).
• OpenAI feature gaps — structured outputs (response_format) and sampling params (top_p, stop, seed, penalties) are supported per-API by pi-ai but not yet surfaced (need the onPayload seam). Function/tool calling is fully wired (tools, tool_choice normalized per target API, tool role); multimodal image input is wired (user content accepts text + image_url parts, data-URL or http(s)).
• Pricing & quotas — out of scope per the design brief.

See AGENTS.md for contributor/agent instructions, and docs/idea.md for the original design note.