Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,45 @@
# discover the server. Default: off (private server, clients must know the pubkey).
# MUXLL_ANNOUNCED=0

# --- Model visibility policy ------------------------------------------------
# The server trims/scope the model catalog exposed via models.list and resolvable
# via chat.complete. Rules are case-insensitive substrings matched against the
# `provider/id` tag (e.g. "anthropic" matches all Anthropic models, "gpt"
# matches every model whose tag contains "gpt" across providers).
#
# These env vars SEED the policy only when no config file exists yet; once
# admin.models.setPolicy persists a change to MUXLL_CONFIG_FILE, that file is
# authoritative (delete it + restart to re-seed from env).
# MUXLL_MODELS_ALLOW=anthropic,gpt
# MUXLL_MODELS_DENY=*-vision-*
#
# Path to the JSON policy file mutated by admin.models.setPolicy.
# Default: ./muxll-config.json
# MUXLL_CONFIG_FILE=muxll-config.json

# --- Admin tools ------------------------------------------------------------
# Comma-separated client pubkeys allowed to call admin.* tools (gated by the
# authenticated caller pubkey the Nostr transport injects). Empty/unset = no
# admin surface is registered at all. Not mutable at runtime — restart to change.
# MUXLL_ADMIN_PUBKEYS=

# --- Billing (Phase 1: prepaid USD balance, admin-funded) --------------------
# Set MUXLL_BILLING=1 to enable per-caller billing: chat.complete caps output
# tokens to what the caller's balance can afford and debits the real cost after
# the turn. Disabled by default (chat is free, as before). Lightning top-up and
# the CEP-8 payment wrap arrive in a later phase; for now balances are funded
# via admin.balance.credit.
# MUXLL_BILLING=1
#
# SQLite file backing the ledger (default ./muxll-ledger.sqlite).
# MUXLL_LEDGER_DB=muxll-ledger.sqlite
#
# Markup over upstream cost in percent (10 = +10%). Default 0 = passthrough.
# MUXLL_MARKUP_PCT=10
#
# Floor charge in USD applied to every chargeable request (anti-spam).
# MUXLL_MIN_CHARGE_USD=0.0001

# --- Upstream LLM provider API keys ------------------------------------------
# Consumed by pi-ai's built-in providers (see @earendil-works/pi-ai). Only the
# keys for providers you want to serve need to be set.
Expand Down
32 changes: 26 additions & 6 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,16 +148,26 @@ asked.

## Where things live / change guide

- **Add an RPC tool** → `registerTools()` in
- **Add an RPC tool** → `registerTools()` (inference tools) or
`registerAdminTools()` (admin tools, gated by caller pubkey) in
`packages/server/src/server.ts`; add the name to `METHODS` in
`packages/core/src/index.ts`; add an integration test.
- **Change provider configuration** → `packages/server/src/main.ts`. Today
it is `builtinModels()` (env-var resolved). To inject a custom provider for
tests, build a `Models` via `createModels()` + `models.setProvider(...)` and
pass it to `startServer({ models })`.
- **Change model resolution** → `resolveModel()` in
`packages/server/src/wire.ts` (parses a `provider/id` tag; a bare id falls
back to first-match across providers).
- **Change model resolution / visibility** → `packages/server/src/catalog.ts`.
`Catalog` layers a `ModelPolicySet` over a pi-ai `Models` and is the single
seam for both `models.list` (`list`/`visibleModels`) and `chat.complete`
(`resolve` — a policy-hidden model returns `undefined`, i.e. "Unknown model",
so it can't be probed). Rules and the `models.list` `search` param share one
matcher: case-insensitive substring on the `provider/id` tag. Tag parsing /
bare-id first-match still lives in `resolveModel()` (`wire.ts`), which
`Catalog.resolve` wraps. Policy is seeded in `main.ts` from
`MUXLL_CONFIG_FILE` (authoritative once it exists) or `MUXLL_MODELS_ALLOW`/
`MUXLL_MODELS_DENY`; `admin.models.setPolicy` patches it and persists via the
`onPolicyMutate` hook. Per-client overrides key on the caller pubkey the
transport injects (`injectClientPubkey: true` in `startServer`).
- **Change the OpenAI wire shape** → `@muxll/core`. The server consumes the
shapes in `chatResult()`/`toUsage()`/`toFinishReason()` (wire.ts) and
`toContext()` (wire.ts, async — fetches http image URLs) maps the
Expand Down Expand Up @@ -198,7 +208,17 @@ Configuration is entirely via environment variables (see `.env.example`).
handler throws, the MCP SDK returns a result with `isError: true` and the
message in `content`. `MuxllClient` turns these into thrown `Error`s for
callers; raw `Client.callTool` tests must inspect `result.isError`, not
`.rejects`.
`.rejects`. This includes the admin guard (`admin.*` from a non-admin →
`isError` "unauthorized") and policy-hidden models (`chat.complete` on a
disallowed model → `isError` "Unknown model", same shape as a genuinely
unknown one).
- **Caller identity comes from the transport, not `_meta`.** `startServer` sets
`injectClientPubkey: true`, so the Nostr transport overwrites
`extra._meta.clientPubkey` with the authenticated gift-wrap signer pubkey
(it can't be spoofed by the client). Per-client model policy and admin
gating both read it via the `clientPubkey(extra)` helper in `server.ts`.
`MUXLL_ADMIN_PUBKEYS` is env-only and immutable at runtime (bootstrap, not
self-administered) — restart to change it.
- **Streaming requires both ends to opt in.** The server sets
`openStream: { enabled: true }`; the client must too, and must send a
`progressToken`, or `extra._meta.stream` is absent and `getOpenStreamWriter()`
Expand Down Expand Up @@ -257,4 +277,4 @@ Configuration is entirely via environment variables (see `.env.example`).
becomes `{type:"tool",name}`; google can't pin a specific function).
Still deferred via the `onPayload` seam: `response_format` and sampling
params (`top_p`/`stop`/`seed`/penalties).
- Pricing/quotas are explicitly out of scope for the POC.
- Billing (Phase 1) is opt-in via `MUXLL_BILLING=1`: `Ledger` (`bun:sqlite`, USD stored as integer nano-dollars) keys balances on the authenticated caller pubkey; `chat.complete` caps output tokens to what the balance affords (`computeOutputCap`, in `pricing.ts`) and debits the real cost (recomputed via pi-ai's `calculateCost`, since the faux provider reports `cost.total = 0`) × `(1+MUXLL_MARKUP_PCT/100)`. `balance.*` / `admin.balance.*` tools fund/query balances. Native-Lightning top-up and the CEP-8 `withServerPayments` wrap are the next phases; see `docs/billing.md`. Billing is disabled entirely when no `ledger` is passed to `startServer`, so existing callers are unaffected.
47 changes: 41 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,9 @@ Nostr-native client can call them. Streamed responses ride CEP-41 open-ended
streams.

This repository currently holds a **proof of concept**: chat completions (with
and without streaming) and model listing. Payments, quotas, and provider config
files are intentionally out of scope for now.
and without streaming) and model listing. Billing is in progress (Phase 1:
a prepaid USD balance ledger funded by admin grants, with a markup over
upstream cost); see [`docs/billing.md`](docs/billing.md) for the design.

## How it works

Expand All @@ -27,12 +28,15 @@ Two MCP tools are exposed (called via the standard `tools/call` method):

#### `models.list`

Returns every model across configured providers. Shape follows OpenAI
`GET /models` (plus router-useful capability fields).
Returns the models the caller may use, across configured providers, after the
server's [model policy](#model-policy--admin-tools) is applied. Shape follows
OpenAI `GET /models` (plus router-useful capability fields). Optional arguments
narrow the policy-visible set; `{}` returns all of it.

```jsonc
// request arguments
{}
{ "search": "gpt", // optional: case-insensitive substring on `provider/id`; absent = all visible
"limit": 20 } // optional: cap on models returned; absent = no limit

// structuredContent
{ "object": "list", "data": [
Expand All @@ -45,6 +49,37 @@ Returns every model across configured providers. Shape follows OpenAI
] }
```

#### Model policy & admin tools

Operators control which models are exposed (to trim large upstream catalogs
and to scope per client). Rules are case-insensitive substrings matched
against the `provider/id` tag: `"anthropic"` matches every Anthropic model,
`"gpt"` matches every model whose tag contains `gpt` across providers.
Precedence: `deny` wins; when `allow` is present only matching tags pass.

The policy is `{ default: { allow?, deny? }, byPubkey?: { <pubkey>: { allow?, deny? } } }`
— a per-client override is patch-merged over `default`. A hidden model neither
lists nor resolves: `chat.complete` reports it as `Unknown model`, so callers
can't probe for disallowed models.

On startup the policy is seeded from `MUXLL_MODELS_ALLOW`/`MUXLL_MODELS_DENY`
(or a `MUXLL_CONFIG_FILE` JSON file once one exists). Two admin tools — gated
to the caller pubkeys in `MUXLL_ADMIN_PUBKEYS` (authenticated by the Nostr
transport, not self-reported) — manage it at runtime:

```jsonc
// admin.models.getPolicy → { default, byPubkey? } of the current policy
{}

// admin.models.setPolicy → patch, returns the new full policy.
// Per field: an array replaces, null clears (field/entry removed), absent unchanged.
{ "default": { "deny": ["*-vision-*"] },
"byPubkey": { "<clientPubkey>": { "allow": ["gpt"] } } }
```

`admin.models.setPolicy` persists to `MUXLL_CONFIG_FILE`; once that file exists
it is authoritative (delete it + restart to re-seed from env).

#### `chat.complete`

```jsonc
Expand Down Expand Up @@ -193,7 +228,7 @@ Proof of concept. Deliberately deferred until the core is validated:
- **Stream cancellation** — a client CEP-41 `abort` does not yet cancel the upstream provider stream.
- **Structured stream chunks** — `text_delta`, `thinking_delta`, and tool-call deltas are all streamed as `chat.completion.chunk` deltas (`delta.content`, `delta.reasoning_content`, `delta.tool_calls`).
- **OpenAI feature gaps** — structured outputs (`response_format`) and sampling params (`top_p`, `stop`, `seed`, penalties) are supported per-API by pi-ai but not yet surfaced (need the `onPayload` seam). Function/tool calling is fully wired (`tools`, `tool_choice` normalized per target API, `tool` role); multimodal image input is wired (`user` content accepts `text` + `image_url` parts, data-URL or http(s)).
- **Pricing & quotas** — out of scope per the design brief.
- **Pricing & quotas** — Phase 1 landed: a prepaid USD balance ledger (`bun:sqlite`), a `maxTokens` cap that bounds each request to what the caller's balance affords, markup-over-upstream pricing, and `balance.*` / `admin.balance.*` tools (balances funded via `admin.balance.credit`). Disabled unless `MUXLL_BILLING=1`. Still to come: native-Lightning top-up (CEP-8 `balance.topup`), fixed per-Mtok pricing overrides, and the CEP-8 payment wrap. See [`docs/billing.md`](docs/billing.md).

See [AGENTS.md](AGENTS.md) for contributor/agent instructions, and
[`docs/idea.md`](docs/idea.md) for the original design note.
Loading
Loading