We currently get LLM completions via OpenRouter; however, given the ties to Cerebras provided inference for speed, we want to also just provide the direct connection so users don't need to have an OpenRouter key, and BYOK through OpenRouter.
- We currently have
RAYPASTE_API_KEY which actually takes the user's OpenRouter key and as a part of our LLM package we make a req to OpenRouter.
- We should make separate
OPENROUTER_API_KEY and CEREBRAS_API_KEY options for config.
- Maintain the
RAYPASTE_API_KEY but that will be used in the future, paired with our internal backend/web app (in-progress). We will provide an option for direct to provider CLI <--> inference (so users don't have to pass their API key through our backend/servers), OR if users authenticate/want to use our backend (we provide a Raypaste API key and manage usage/billing, routing requests, and provide extra services/features possible with stored memories/vectors.
We currently get LLM completions via OpenRouter; however, given the ties to Cerebras provided inference for speed, we want to also just provide the direct connection so users don't need to have an OpenRouter key, and BYOK through OpenRouter.
RAYPASTE_API_KEYwhich actually takes the user's OpenRouter key and as a part of our LLM package we make a req to OpenRouter.OPENROUTER_API_KEYandCEREBRAS_API_KEYoptions for config.RAYPASTE_API_KEYbut that will be used in the future, paired with our internal backend/web app (in-progress). We will provide an option for direct to provider CLI <--> inference (so users don't have to pass their API key through our backend/servers), OR if users authenticate/want to use our backend (we provide a Raypaste API key and manage usage/billing, routing requests, and provide extra services/features possible with stored memories/vectors.