LiteLLM provider falls through to native/OpenRouter routing

[zdurm]

Summary

LiteLLM provider routing appears to be broken for litellm@... / ll@... models. Even with a working LiteLLM proxy and valid LITELLM_BASE_URL / LITELLM_API_KEY, Claude Code requests can be routed to the native Anthropic/OpenRouter path instead of the LiteLLM handler.

There is also an auth conflict in proxy mode when both ANTHROPIC_API_KEY and ANTHROPIC_AUTH_TOKEN are injected or inherited.

Environment

• Claudish version: 7.1.2
• Claude Code version: 2.1.143
• LiteLLM proxy: http://localhost:4000
• LiteLLM model group tested: bedrock-opus

Reproduction

1. Start a LiteLLM proxy with a model group:

model_list:
  - model_name: bedrock-opus
    litellm_params:
      model: bedrock/us.anthropic.claude-opus-4-7
      aws_region_name: os.environ/AWS_REGION

2. Verify LiteLLM works directly:

curl http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer dummy" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bedrock-opus",
    "messages": [{"role": "user", "content": "Say ok only"}],
    "max_tokens": 8
  }'

Expected direct LiteLLM result includes:

{
  "model": "bedrock-opus",
  "choices": [
    {
      "message": {
        "content": "ok",
        "role": "assistant"
      }
    }
  ]
}

3. Run Claudish:

export LITELLM_BASE_URL=http://localhost:4000
export LITELLM_API_KEY=dummy

claudish --model litellm@bedrock-opus "Say ok only"

Also tested:

claudish --model ll@bedrock-opus "Say ok only"
claudish --default-provider litellm --model bedrock-opus "Say ok only"

Expected behavior

Claudish should create the LiteLLM handler and call:

http://localhost:4000/v1/chat/completions

with model:

bedrock-opus

Actual behavior

The request falls through to native Anthropic/OpenRouter behavior instead of reaching the LiteLLM proxy.

Debug output shows native handling:

[NATIVE] Claude Code → Anthropic API Request
Request body (Model: litellm@bedrock-opus)
[NATIVE] Response: not_found_error model: litellm@bedrock-opus

Suspected cause

LiteLLM is defined as an env-based provider:

baseUrl: ""
baseUrlEnvVars: ["LITELLM_BASE_URL"]

But remote provider registration appears to filter out providers with an empty baseUrl before env-based base URL resolution:

.filter((def) => !def.isLocal && def.baseUrl !== "" && ...)

This causes the LiteLLM provider to be excluded from remote provider resolution even when LITELLM_BASE_URL is set.

A local patch that allowed env-only base URLs made provider probing succeed:

{
  "model": "litellm@bedrock-opus",
  "nativeProvider": "litellm",
  "isExplicit": true,
  "directProbe": {
    "state": "live"
  }
}

However, direct --model litellm@... still appeared to hit Claude Code model validation/native routing. A role-mapping workaround did route correctly after patching:

claudish \
  --model-opus litellm@bedrock-opus \
  --model-sonnet litellm@bedrock-opus \
  --model-haiku litellm@bedrock-opus \
  "Say ok only"

Working patched path debug output:

[Proxy] Handler: provider=litellm, model=bedrock-opus
[Proxy] Created LiteLLM handler (composed): bedrock-opus (http://localhost:4000)
[LiteLLM] Calling API: http://localhost:4000/v1/chat/completions
[LiteLLM] Response status: 200

Auth conflict

In proxy mode, Claude Code can warn when both an API key placeholder and a claude.ai token are present:

Auth conflict: Both a token (claude.ai) and an API key (ANTHROPIC_API_KEY) are set.

For pure proxy mode, unsetting inherited Anthropic auth variables before launching Claude Code avoided the warning while still allowing the proxy handler to work.

Suggested fix

1. Include env-only remote providers when their base URL env var is present.

Example:

def.baseUrl !== "" || def.baseUrlEnvVars?.some((envVar) => process.env[envVar])

2. Ensure litellm@... / ll@... direct model selection routes through the LiteLLM handler, not native model validation/fallback.

3. In non-native proxy mode, avoid setting or inheriting conflicting Anthropic auth variables in the Claude Code child process.

Notes

No provider credentials or account-specific details are required to reproduce. Any local LiteLLM proxy with a model group should expose the same routing behavior.