Skip to content

AGENTS.md

Latest commit

 

History

History
260 lines (228 loc) · 14.4 KB

AGENTS.md

File metadata and controls

260 lines (228 loc) · 14.4 KB
Raw

AGENTS.md

Project Overview

Muxll is a CVM-based LLM router. It exposes upstream LLM providers (Anthropic, OpenAI, …) over the CVM JSON-RPC interface — MCP-over-Nostr — so Nostr-native clients can call them. Streamed responses use CEP-41 open-ended streams.

This is a proof of concept: only chat.complete (streaming and non-streaming) and models.list are implemented. See README.md and docs/idea.md for the design.

This is a Bun workspace monorepo with three packages. Packages import each other by @muxll/* name; tsconfig.json paths map each name to its source, so dev needs no build step (Bun runs TS directly and honours the same paths at runtime). Publish-time builds can be added later without changing import paths.

Architecture, in one line per layer:

  • packages/core/src/index.ts — @muxll/core: the OpenAI wire shapes (zod schemas + inferred TS types) and the tool-method names. Pure data contracts, no transport, no pi-ai. Shared by the server and every client.
  • packages/server/src/server.ts — McpServer from @contextvm/mcp-sdk registers the two tools; NostrServerTransport from @contextvm/sdk exposes them over Nostr with openStream: { enabled: true }.
  • packages/server/src/wire.ts — OpenAI ↔ pi-ai translation (toContext, chatResult, toUsage, toFinishReason, resolveModel, mapToolChoice). Transport-agnostic.
  • packages/server/src/main.ts — entry point; reads env, builds the provider registry via pi-ai's builtinModels(), connects the transport.
  • packages/client/src/client.ts — @muxll/client: the reference CVM client. A typed wrapper over the MCP Client + NostrClientTransport that calls chat.complete / models.list and parses CEP-41 streams into OpenAI chunks. Never touches pi-ai — that lives in the server.
  • packages/cli/src/index.ts — @muxll/cli: the reference command-line client over @muxll/client + @earendil-works/pi-tui. Two commands (models, chat) with hand-rolled argv parsing (commander is the upgrade if the surface grows). chat with a prompt streams one turn to stdout (streamTurn); with no prompt it opens a full-screen pi-tui app (runTui): tagged turns, markdown replies, a bottom-pinned StatusBar with live formatUsage stats (a Fill spacer pins status+input to the last row), Ctrl+C to quit. Both are the seam the test harness drives.
  • packages/proxy/src/index.ts — @muxll/proxy: the OpenAI-compatible HTTP bridge. A Bun.serve shell over @muxll/client that exposes /v1/chat/completions and /v1/models so any OpenAI client plugs into a muxll provider by changing a base URL. A client of muxll, not a sibling of the server: never touches pi-ai, holds no upstream keys. Translation is near-zero (CEP-41 chunks are already OpenAI-shaped): non-streaming returns the parsed completion as JSON; streaming wraps each chunk as an SSE data: line ending in data: [DONE]. startProxy({ client, port, hostname }) takes an already-connected client (DI, like startServer({ models })); main() builds it from env. Binds 127.0.0.1, no auth (local-only by design).
  • packages/test-utils/src/ — @muxll/test-utils: shared test fixtures. MockRelayHub (an in-process RelayHandler) and standUpFauxClient() (a faux-backed server + connected MuxllClient over a fresh mock relay) — the harness every integration test reuses. Server integration/smoke tests use only MockRelayHub; client/cli/proxy tests use both.
  • Provider abstraction is @earendil-works/pi-ai (Models.stream() / Models.complete()). Streaming events are AssistantMessageEvents (text_delta, done, error, …).
  • CEP-41 streaming: the transport injects an OpenStreamWriter at extra._meta.stream inside the tool handler when the client sent a progressToken.

Key dependencies: @contextvm/sdk, @contextvm/mcp-sdk, @earendil-works/pi-ai, @earendil-works/pi-tui (CLI), zod. Runtime/package manager: Bun.

Setup Commands

bun install
cp .env.example .env   # set provider keys (e.g. ANTHROPIC_API_KEY) and optionally a stable server key

.env is gitignored. Provider keys are consumed directly by pi-ai's built-in providers — see .env.example for the full list.

Development Workflow

bun start            # bun run packages/server/src/main.ts; prints server pubkey + relays
bun run cli models   # run the CLI against MUXLL_SERVER_PUBKEY (see packages/cli)
bunx tsc --noEmit    # typecheck the whole workspace (no separate build step; Bun runs TS directly)

There is no build output and no bundler. The package is type: "module" with moduleResolution: "bundler" and allowImportingTsExtensions: true, so source imports use explicit .ts extensions — keep that convention in new files.

tsconfig.json includes only packages/*/src and packages/*/tests and excludes node_modules and docs. Per-package tsconfig.build.json files should be added only when a package needs to emit for publishing.

Testing Instructions

bun run test        # runs the package script: `bun test packages/*/tests`
bunx tsc --noEmit   # must pass before committing

⚠️ Use bun run test (or pass explicit dirs/files). Bare bun test is too broad: it scans the vendored docs/ tree and follows the workspace node_modules symlinks into Bun's package store, running dependency test suites (pi-ai's, etc.) — noisy and unrelated.

  • packages/server/tests/smoke.test.ts is a real end-to-end round-trip against OpenRouter. It is skipped unless SMOKE_TEST=1 is set (it hits the network and costs money). Run it with SMOKE_TEST=1 bun run test.

  • Tests live in packages/*/tests/. The server integration test stands up the full server and a mock Nostr relay using pi-ai's fauxProvider, so no real API keys or network relays are required. The shared fixtures — MockRelayHub (in-process RelayHandler) and standUpFauxClient() (faux server + connected MuxllClient over a fresh mock relay) — live in @muxll/test-utils. Server integration/smoke tests use only MockRelayHub (they drive a raw MCP Client for callToolStream); client/cli/proxy tests use both and drive the server through MuxllClient. The proxy test drives the HTTP API over a real loopback socket (fetch against a random port), not Bun's in-process server.fetch() (which bypasses the routes table and calls the fetch fallback directly).

  • To run one test by name: bun test packages/server/tests/integration.test.ts -t "streams text deltas".

When changing packages/server/src/server.ts, the wire schemas, or adding a tool, add or update a test in the relevant packages/*/tests/ even if not asked.

Code Style

  • TypeScript strict is on (plus noUncheckedIndexedAccess, noFallthroughCasesInSwitch, noImplicitOverride). Don't weaken these.
  • No lint/format tool is enforced in CI; match surrounding style (2-space indent, double quotes, trailing commas). eslint/prettier configs exist for manual use (bun run lint, bun run format).
  • Tool input/output schemas are Zod raw shapes (e.g. { model: z.string() }), defined in @muxll/core and passed to McpServer.registerTool — not z.object(...).
  • Reusable wire types (e.g. ChatInput) are exported from @muxll/core using z.input (not z.infer/z.output): callers build requests, so .default() fields like stream stay optional on input and are filled by the server.
  • Keep the codebase minimal. Mark deliberate shortcuts with a // ponytail: comment naming the ceiling and the upgrade path. Don't add abstractions, config, or deps that aren't needed yet.

Where things live / change guide

  • Add an RPC tool → registerTools() in packages/server/src/server.ts; add the name to METHODS in packages/core/src/index.ts; add an integration test.
  • Change provider configuration → packages/server/src/main.ts. Today it is builtinModels() (env-var resolved). To inject a custom provider for tests, build a Models via createModels() + models.setProvider(...) and pass it to startServer({ models }).
  • Change model resolution → resolveModel() in packages/server/src/wire.ts (parses a provider/id tag; a bare id falls back to first-match across providers).
  • Change the OpenAI wire shape → @muxll/core. The server consumes the shapes in chatResult()/toUsage()/toFinishReason() (wire.ts) and toContext() (wire.ts, async — fetches http image URLs) maps the OpenAI-shaped {role, content} input into pi-ai's Context. Clients (@muxll/client) parse responses against the same schemas.
  • Add/change a client caller → @muxll/client (packages/client/src/client.ts). The CLI (@muxll/cli), HTTP proxy (@muxll/proxy), and web app (TBD) are consumers of @muxll/client over CVM/Nostr — clients, not siblings of the server. The HTTP proxy is a client: it translates a Nostr-native upstream muxll provider into an OpenAI HTTP API.
  • Change the CLI → @muxll/cli (packages/cli/src/index.ts). Two commands (models, chat) over @muxll/client; hand-rolled argv parsing (swap in commander if the command surface grows past two). chat is a pi-tui app (runTui): turns are tagged (you ❯ / model short-name), reply rendering is Markdown + mdTheme, usage is formatUsage + StatusBar, and a Fill spacer pins status+input to the bottom row; one-shot mode reuses streamTurn.
  • Change the HTTP proxy → @muxll/proxy (packages/proxy/src/index.ts). startProxy({ client, port, hostname }) over @muxll/client + Bun.serve; routes /v1/models and /v1/chat/completions (SSE when stream:true). Error mapping: 400 validation, 404 unknown model (by message match), 502 upstream/transport. server.timeout(req, 0) on the chat route — LLM generation idles past Bun's 10s idleTimeout. Local-only: binds 127.0.0.1, no auth (deferred).
  • Change shared test fixtures → @muxll/test-utils (packages/test-utils/src/). MockRelayHub + standUpFauxClient(); consumed by server/client/cli/proxy tests. Add helpers here when a third test repeats the same setup.

Build and Deployment

No build step. Run directly with Bun (bun start, or bun run packages/server/src/main.ts). Deployment is TBD; for now the server is a long-running Bun process that connects to configured Nostr relays. Configuration is entirely via environment variables (see .env.example).

Important Gotchas

  • Tool errors are isError results, not rejected promises. When a tool handler throws, the MCP SDK returns a result with isError: true and the message in content. MuxllClient turns these into thrown Errors for callers; raw Client.callTool tests must inspect result.isError, not .rejects.
  • Streaming requires both ends to opt in. The server sets openStream: { enabled: true }; the client must too, and must send a progressToken, or extra._meta.stream is absent and getOpenStreamWriter() throws.
  • Large payloads need CEP-22 and @contextvm/sdk ≥ 0.13.6. Two things must both hold, or any request/response past ~64 KB throws invalid plaintext size: must be between 1 and 65535 bytes: (1) oversizedTransfer: { enabled: true } on every transport so the SDK CEP-22-chunks large messages — MuxllClient and the server set it, raw transports in tests must too, and request-side chunking fires only when the request carries a progressToken (MuxllClient always does, via onprogress); (2) nostr-tools ≥ 2.23.4, which fixed a NIP-44 bug capping encryption at 65 535 bytes. The SDK brings this from 0.13.6 onward (it declares nostr-tools ^2.23.9); 0.13.5 pinned ~2.18.2 (the bug), so don't downgrade below 0.13.6. (Earlier a root package.json overrides field shimmed the fix; it's removed now that the SDK ships it.) With both, requests chunk up to the SDK's 100 MiB reassembly cap — no practical ceiling. A >64 KB regression test in packages/client/tests/ guards this.
  • docs/ is reference material, not part of the project. Do not edit it, do not import from it. It contains vendored copies of cordn, routstr-core, yalr, the CVM docs, and the pi readme. Read them for guidance only.
  • Workspace resolution is by source, not symlinks. @muxll/* resolves to source via tsconfig.json paths for both tsc and Bun runtime — there are no node_modules/@muxll links. The workspace:* declarations in each package.json declare the relationship for tooling and future publishing.
  • The CVM SDK logs to stderr, which is the same TTY a TUI renders to. The transport's pino logger (module nostr-client-transport) captures LOG_LEVEL/LOG_ENABLED at module-load time and writes info logs to fd 2, which corrupts a full-screen client. Setting LOG_LEVEL in-process is too late (the level is already locked), so MuxllClientOptions.logLevel threads the transport's logLevel option through instead (evaluated at transport construction); the CLI passes "silent". Non-TUI clients (proxy) keep the default.
  • Bun.serve's in-process server.fetch() bypasses routes. It calls the fetch fallback directly, so it can't exercise routing or per-request server.timeout. The proxy test drives the HTTP API over a real loopback socket (fetch against the bound port) instead; keep doing that for any routes-based handler test.

Pull Request Guidelines

  • Run bunx tsc --noEmit and bun run test before committing; both must be green.
  • Add or update tests in packages/*/tests/ for any behavior change in src/.
  • Keep diffs minimal and focused — this project is intentionally small.

Additional Notes

  • Client cancellation does not yet propagate to the upstream provider stream (a ponytail: deferral in runStreamed()).
  • Each CEP-41 chunk is one JSON-serialized OpenAI chat.completion.chunk (delta.content for text, delta.reasoning_content for thinking, delta.tool_calls for tool-call openers/fragments, a final chunk with finish_reason + usage incl. cost); the streaming tool result carries only metadata (usage, no text). tool_choice is normalized per target API in mapToolChoice() (anthropic/bedrock/google use any; the object form becomes {type:"tool",name}; google can't pin a specific function). Still deferred via the onPayload seam: response_format and sampling params (top_p/stop/seed/penalties).
  • Pricing/quotas are explicitly out of scope for the POC.