Skip to content

Support local LLM servers (Ollama / llama.cpp / LM Studio) #3

Description

@8crsk

Users running Ollama or llama.cpp on a home machine should be able to point 4AIs at it — the ultimate no-cloud setup: phone agent + local model, zero external API calls.

Design questions to settle first (comment here!)

  • Custom base-URL field in Settings vs a dedicated 'Local server' provider entry?
  • How to handle servers with no API key (the gateway config currently always sends one)?
  • mDNS/manual discovery of the server on the LAN?

Where: ProviderCatalog.kt + the Settings provider UI (ui/settings/). The gateway already speaks openai-completions to any baseUrl.

Acceptance criteria

  • User can enter an arbitrary OpenAI-compatible baseUrl (+ optional key)
  • Works with Ollama's /v1 endpoint end-to-end
  • Graceful error surface when the server is unreachable

Metadata

Metadata

Assignees

No one assigned

    Labels

    ⚔️ questA meaty feature with clear scope — real XP

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions