Perplexity and Mistral models are incorrectly mapped (potential more)

[dragos199993]

Description
Hey there 👋 ! I am trying to integrate a router model solution into Braintrust Proxy and I encountered a problem that Perplexity and Mistral models are not working.

Steps to reproduce:

import { OpenAI } from "openai";
const client = new OpenAI({
  baseURL: "https://api.braintrust.dev/v1/proxy",
  apiKey: '<API-KEY>'
});

async function main() {
  const start = performance.now();
  const response = await client.chat.completions.create({
    model: "pplx-7b-online",
    messages: [{ role: "user", content: "how's the weather in tokyo?" }],
    seed: 1, // A seed activates the proxy's cache
  });
  console.log(response.choices[0].message.content);
  console.log(response.model);
  console.log(`Took ${(performance.now() - start) / 1000}s`);
}

main();

By using this you get this error:

Error code: 400 - {'error': {'message': "Invalid model 'pplx-7b-online'. Permitted models can be found in the documentation at https://docs.perplexity.ai/docs/model-cards.", 'type': 'invalid_model', 'code': 400}}

The model is taken from here: https://www.braintrust.dev/docs/guides/proxy#list-of-supported-models-and-providers
In my account I correctly set the perplexity and mistra api keys and used the correct key.

The request fails with the 400 error mentioned above.
Expected Behaviour:
The response should respond with a 200 and output the text.

Impact:
This issue is blocking to use the service and also to propose new integration for it.

[ankrgyl]

Hi there! Very open to contributions if you want to update the corresponding model entries:

braintrust-proxy/packages/proxy/schema/models.ts

Line 4 in 279491a

export type PromptInputType = (typeof PromptInputs)[number];

[dragos199993]

Hey @ankrgyl! I looked into the mapping issue and I tried to add (for example) the perplexity model llama-3.1-sonar-large-128k-online where you pointed in braintrust-proxy/packages/proxy/schema/models.ts. The problem is that this model is not mapped to any secret key when we are calling https://www.braintrust.dev/api/secret.

       const response = await fetch(
        `${opts.braintrustApiUrl || DEFAULT_BRAINTRUST_APP_URL}/api/secret`,
        {
          method: "POST",
          headers: {
            Authorization: `Bearer ${authToken}`,
            "Content-Type": "application/json",
          },
          body: JSON.stringify({
            model,
            org_name,
            mode: "full",
          }),
        },
      );

I suspect this is done via packages/proxy/schema/index.ts on type AvailableEndpointTypes. I added that there as well, but of course this one needs to be deployed to https://www.braintrust.dev/api in order to test it.. 😢

Basically now trying to call like this:

async function main() {
  const start = performance.now();
  const response = await client.chat.completions.create({
    model: "llama-3.1-sonar-large-128k-online",
    messages: [
      {"role": "user", "content": "Your message here"}
    ],
    seed: 1,
  });
}

responds with a 400 error, because the secret key is not found even if it's added in braintrust dashboard.

[ankrgyl]

You should be able to test by using your perplexity API key directly against the proxy (instead of your BRAINTRUST_API_KEY)