-
Notifications
You must be signed in to change notification settings - Fork 460
Fetch models from models_gateway and prefer gateway-served models in picker #1606
Copy link
Copy link
Closed
Copy link
Labels
area/agentFor work that has to do with the general agent loop/agentic features of the appFor work that has to do with the general agent loop/agentic features of the apparea/providersFor features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)area/providers/docker-model-runnerDocker Model Runner (DMR) local inferenceDocker Model Runner (DMR) local inferencearea/tuiFor features/issues/fixes related to the TUIFor features/issues/fixes related to the TUI
Description
Activity
Metadata
Metadata
Assignees
Labels
area/agentFor work that has to do with the general agent loop/agentic features of the appFor work that has to do with the general agent loop/agentic features of the apparea/providersFor features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)area/providers/docker-model-runnerDocker Model Runner (DMR) local inferenceDocker Model Runner (DMR) local inferencearea/tuiFor features/issues/fixes related to the TUIFor features/issues/fixes related to the TUI
Summary
When a
models_gatewayis configured, cagent currently assumes gateway supports core providers and shows a static list (configured models + models.dev catalog). It does not query the gateway for a live list of models. This causes incorrect/irrelevant models to appear in the model picker.Proposal
Add gateway model discovery when
models_gatewayis set: query<gateway>/v1/models(OpenAI-style) or a configurable metadata endpoint and use the returned model list to populate/suppress models in the picker.Acceptance criteria
models_gatewayis set and/v1/modelsreturns a list, the model picker shows only models served by that gateway (merged with explicitly defined models in config).models_gateway.discover: true/falseto control discovery.AGENTS.mdandREADME.md.Notes
/v1/models, fall back to current behavior.