Skip to content

providers.md

Latest commit

 

History

History
276 lines (205 loc) · 10.4 KB

providers.md

File metadata and controls

276 lines (205 loc) · 10.4 KB

Provider Configuration

Phantom routes every LLM query (the main agent and every evolution judge) through the Anthropic Agent SDK by default. By setting environment variables that the bundled cli.js already honors, you can point that subprocess at any Anthropic Messages API compatible endpoint without changing a line of code.

The provider: block in phantom.yaml is a small config surface that translates into those environment variables for you. When agent_runtime: murph is selected, the same block translates into native Murph route variables instead.

Agent Runtime Selector

agent_runtime selects the Agent SDK implementation. It is separate from provider, which selects the model endpoint.

agent_runtime: anthropic # anthropic | murph

Anthropic is the default and existing deployments do not need to set this field. Murph is opt-in and loads @murph/anthropic-sdk-shim from the Phantom project. If the package is not installed or linked, startup fails with an actionable message.

Runtime selection precedence is:

  1. phantom start --agent-sdk-module <specifier>
  2. PHANTOM_AGENT_SDK_MODULE
  3. phantom start --agent-runtime <runtime>
  4. PHANTOM_AGENT_RUNTIME
  5. YAML agent_runtime
  6. Default anthropic

Use the custom module escape hatch for a local compatibility harness:

phantom start --agent-sdk-module file:///path/to/runtime.mjs

Use the named Murph runtime when @murph/anthropic-sdk-shim is installed or linked:

export PHANTOM_AGENT_RUNTIME=murph
phantom start --agent-runtime murph

Supported Providers

Type Base URL API Key Env Notes
anthropic (default) https://api.anthropic.com ANTHROPIC_API_KEY Claude Opus, Sonnet, Haiku
openai https://api.openai.com OPENAI_API_KEY Murph runtime only
zai https://api.z.ai/api/anthropic ZAI_API_KEY GLM-5.1 and GLM-4.5-Air, roughly 15x cheaper than Opus
openrouter https://openrouter.ai/api/v1 OPENROUTER_API_KEY 100+ models through a single key
vllm http://localhost:8000 none Self-hosted OpenAI-compatible inference
ollama http://localhost:11434 none Local GGUF models, zero API cost
litellm http://localhost:4000 LITELLM_KEY Local proxy bridging OpenAI, Gemini, and others
custom (you set it) (you set it) Any Anthropic Messages API compatible endpoint

Quick Reference

Anthropic (default)

No configuration needed. Existing deployments continue to work unchanged.

# phantom.yaml
model: claude-opus-4-7
# No provider block = defaults to anthropic
# .env
ANTHROPIC_API_KEY=sk-ant-...

Murph with OpenAI

Use this shape only when Murph is installed or linked into the Phantom project.

# phantom.yaml
agent_runtime: murph
model: gpt-5.5
provider:
  type: openai
# .env
OPENAI_API_KEY=sk-...

Phantom passes gpt-5.5 as the concrete SDK model and sets native Murph route env such as MURPH_PROVIDER=openai, MURPH_MODEL=gpt-5.5, and MURPH_OPENAI_MODEL=gpt-5.5. Judge calls use the same concrete model unless you add model_mappings for cheaper tier-specific routes.

Murph with Z.AI / GLM

# phantom.yaml
agent_runtime: murph
model: sonnet
provider:
  type: zai
  api_key_env: ZAI_API_KEY
  model_mappings:
    sonnet: glm-5.1
    haiku: glm-5
# .env
ZAI_API_KEY=<your-zai-key>

The Murph route is native GLM, not the legacy Anthropic-compatible bridge. Phantom sets MURPH_PROVIDER=glm, MURPH_MODEL=<resolved>, and MURPH_GLM_MODEL=<resolved>.

Z.AI / GLM-5.1

Z.AI provides an Anthropic Messages API compatible endpoint at https://api.z.ai/api/anthropic. Phantom ships with a zai preset that points there automatically. Get a key at docs.z.ai.

# phantom.yaml
model: claude-sonnet-4-6
provider:
  type: zai
  api_key_env: ZAI_API_KEY
  model_mappings:
    opus: glm-5.1
    sonnet: glm-5.1
    haiku: glm-4.5-air
# .env
ZAI_API_KEY=<your-zai-key>

Both the main agent and every evolution judge route through Z.AI. The claude-sonnet-4-6 model name is translated to glm-5.1 on the wire by the model_mappings block.

Ollama (local, free)

Run any GGUF model on your own GPU. No API key needed.

# phantom.yaml
model: claude-sonnet-4-6
provider:
  type: ollama
  model_mappings:
    opus: qwen3-coder:32b
    sonnet: qwen3-coder:32b
    haiku: qwen3-coder:14b

Ollama must be running at http://localhost:11434 (the preset default). The model must support function calling to work with Phantom's agent loop.

vLLM (self-hosted)

For organizations running their own inference clusters.

# phantom.yaml
model: claude-sonnet-4-6
provider:
  type: vllm
  base_url: http://your-vllm-server:8000
  model_mappings:
    sonnet: your-model-name
  timeout_ms: 300000  # local models can be slow on first call

Start vLLM with --tool-call-parser matching your model for tool use to work.

OpenRouter

Access 100+ models through a single OpenRouter key.

# phantom.yaml
model: claude-sonnet-4-6
provider:
  type: openrouter
  api_key_env: OPENROUTER_API_KEY
  model_mappings:
    sonnet: anthropic/claude-sonnet-4.5

LiteLLM (proxy)

Run a local LiteLLM proxy to bridge OpenAI, Gemini, and other formats.

# phantom.yaml
model: claude-sonnet-4-6
provider:
  type: litellm
  api_key_env: LITELLM_KEY
  # base_url defaults to http://localhost:4000

Custom endpoint

For the default Anthropic runtime, custom means any Anthropic Messages API compatible proxy. Under Murph, custom means an OpenAI-compatible custom endpoint.

# phantom.yaml
model: claude-sonnet-4-6
provider:
  type: custom
  base_url: https://your-proxy.internal/anthropic
  api_key_env: YOUR_CUSTOM_KEY_ENV
# phantom.yaml, Murph OpenAI-compatible custom endpoint
agent_runtime: murph
model: your-model
provider:
  type: custom
  base_url: https://your-proxy.internal/v1
  api_key_env: YOUR_CUSTOM_KEY_ENV

Configuration Fields

Field Type Default Purpose
type enum anthropic One of the supported provider types
base_url URL preset default Override the endpoint URL
api_key_env string preset default Name of the env var holding the credential
model_mappings.opus string none Concrete model ID for the opus tier
model_mappings.sonnet string none Concrete model ID for the sonnet tier
model_mappings.haiku string none Concrete model ID for the haiku tier
disable_betas boolean preset default Sets CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1. Defaulted true for every non-anthropic preset.
timeout_ms number none Sets API_TIMEOUT_MS for slow local inference

Environment Variable Overrides

For operators who prefer env variables over YAML edits:

Variable Effect
PHANTOM_AGENT_RUNTIME Override agent_runtime with anthropic or murph
PHANTOM_AGENT_SDK_MODULE Override the Agent SDK boundary with a custom module specifier
PHANTOM_PROVIDER_TYPE Override provider.type (validated against the supported values)
PHANTOM_PROVIDER_BASE_URL Override provider.base_url (validated as a URL)
PHANTOM_MODEL Override config.model

These are applied on top of the YAML-loaded config during startup.

How It Works

With the default Anthropic runtime, the Agent SDK runs as a subprocess. The SDK's bundled cli.js reads ANTHROPIC_BASE_URL and the ANTHROPIC_DEFAULT_*_MODEL aliases at call time. When ANTHROPIC_BASE_URL points at a non-Anthropic host, all Messages API requests go there instead.

With the Murph runtime, Phantom emits native Murph route variables such as MURPH_PROVIDER, MURPH_PROVIDER_CONFIG, MURPH_MODEL, and provider-specific credential keys. Ambient route keys are shadowed so stale shell credentials do not accidentally select a different provider.

The provider: block is translated by helpers in src/config/providers.ts. The resulting map is merged into main agent, chat, judge, and reflection queries, so changing providers flips every Agent SDK path in lockstep.

Why keep a Claude model name in model: on the Anthropic runtime?

The bundled cli.js has hardcoded model-name arrays for capability detection (thinking tokens, effort levels, compaction, etc.). Passing a literal glm-5.1 as the model can break those checks. The recommended pattern is:

  1. Set model: claude-sonnet-4-6 (or Opus) in phantom.yaml so cli.js treats the call as a known Claude model
  2. Set model_mappings.sonnet: glm-5.1 in the provider block so the wire call goes to GLM-5.1

This is the same pattern Z.AI's own documentation recommends.

Troubleshooting

Phantom responds but the logs show Claude-shaped costs. The bundled cli.js calculates total_cost_usd from its local Claude pricing table based on the model name string. Cost reporting is not provider-aware, so the logged cost will look like Claude pricing even when the request went to Z.AI or another provider. The actual charge on your provider's bill will differ.

Auto mode judges fall back to heuristic mode. resolveJudgeMode in auto mode enables LLM judges when any of these are true: (a) a non-anthropic provider is configured, (b) provider.base_url is set, (c) ANTHROPIC_API_KEY is present, or (d) ~/.claude/.credentials.json exists. If none hold, judges run in heuristic mode. Set judges.enabled: always in config/evolution.yaml to force LLM judges on.

Third-party proxy rejects a beta header. disable_betas: true is already the default for every non-anthropic preset, which sets CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1. If you still see beta header errors, explicitly set disable_betas: true on your provider block to make sure it overrides any custom disable_betas: false.

Tool calls fail with small local models. Phantom's tool system assumes strong function-calling capability. Models like Qwen3-Coder and GLM-5.1 handle it well; smaller models often fail on complex multi-step tool chains. Test with a strong model first, then drop down.

Subprocess fails with a missing-credential error. Phantom does not validate credentials at load time. The subprocess only sees the provider env vars when a query runs. If api_key_env names a variable that is not set in the process environment, the subprocess will fail at call time with the provider's own error message.