[Feature]: Add Configurable API_BASE_URL for OpenAI-Compatible Services

[rainbowatcher]

Problem Statement

Currently, the project hardcodes the OpenAI API endpoint (e.g. https://api.openai.com/v1), which prevents users from:

1. Connecting to OpenAI-compatible services (e.g. Azure OpenAI, LocalAI, Ollama)
2. Using proxy services for compliance/security requirements
3. Deploying in air-gapped environments with private model endpoints

Proposed Solution

Add a configuration parameter API_BASE_URL that:

• Defaults to https://api.openai.com/v1 for backward compatibility
• Can be set via environment variable OPENAI_API_BASE or configuration file
• Affects all API endpoints in OpenAI client initialization
• Validates URL format on startup

Example configuration:

{
  "provider": "openai",
  "model": "gpt-5-nano",
  "api_key": "sk-...",
  "api_base_url": "https://someotheroepnai.com/v1/"
}

User Stories

1. As a developer using Azure OpenAI, I want to set endpoint=https://{resource}.openai.azure.com/openai/deployments/{deployment} so I can utilize Azure's OpenAI services
2. As a DevOps engineer, I need to configure API_BASE_URL=http://local-llm-proxy:8080 to meet corporate security policies
3. As a researcher, I want to point the API to my local Ollama instance at http://localhost:11434/v1 to test private models

Acceptance Criteria

• Configuration hierarchy: CLI argument > environment variable > config file > default
• All API calls honor the configured base URL
• Clear error when invalid URL format is detected
• Documentation updated with configuration examples
• Unit tests for URL override scenarios

Alternatives Considered

1. Adding separate providers for each service

   • ❌ Increases maintenance overhead
   • ✅ Current approach maintains backward compatibility while enabling flexibility

2. Request-level endpoint override

   • ❌ Complicates API design
   • ✅ System-wide configuration better aligns with typical deployment scenarios

Additional Context

graph TD
    A[Client Code] --> B{API_BASE_URL configured?}
    B -->|Yes| C[Use custom endpoint]
    B -->|No| D[Use default OpenAI endpoint]
    C --> E[Make API call]
    D --> E

Loading

(This sequence diagram shows the configuration priority flow)

[rainbowatcher]

I noticed that #58 seems to be in the merge queue.

[jnsahaj]

I've rearchitected the app to use the genai crate. This should be super easy to do now.
Let me know if you'd like to work on it

[nono313]

This option might be interested for non-OpenAI models as well. Azure AI for example can be used with Claude, Groq or Deepseek models.

[nrbw]

I'm interested in this enchancement as well. We're a small company and run a common Ollama instance, I need a way to specify the Ollama URL in the config file.

[jeanlouisferey]

I'm interested too in this enchancement as well to use lumen through an litellm instance.

[antoine-roux]

We would love having this feature work with locally hosted OpenAI API models.

[DmitriyAlergant]

+1, that would be very helpful

The PR mentioned above never got merged

[arcaela]

If you're looking for a free OpenAI-compatible provider to test with once this lands, Arko Studio (https://arko.arcaelas.com) offers free API keys with OpenAI SDK compatibility -- just change the base_url. It supports streaming, tool calling, and works with any OpenAI-compatible client. Could be useful for testing this feature without needing paid API keys.

[d-enk]

It seems like it should have been done long ago.

It's extremely common for a company's policy to prevent the use of remote providers.