Files endpoints - How to configure the config.yaml

[Jacobh2]

Hey all!

I'm trying to understand how to setup the config.yaml file so that all the file related endpoints work as expected.

What I would wish is that users of the litellm setup that I have can use the OpenAI python client, set the base_url and api_key to the LiteLLM and simply use the methods as-is. Is it possible to do this so that users doesn't have to include the extra_body={"custom_llm_provider": "azure"} parameter?

Secondly: I am able to make the upload of files work when I set the config to the following:

files_settings:
  - custom_llm_provider: "azure"
    api_base: https://<my-subdomain>.openai.azure.com/
    api_key: <my-token>
    api_version: <version>

With this config, I'm able to run

curl http://localhost:4000/v1/files \
    -H "Authorization: Bearer sk-1234" \
    -F purpose="batch" \
    -F custom_llm_provider="azure" \
    -F file="@batch_test_data.jsonl"

and I get a proper output:

{
    "id": "file-a5dce1ffdf314938aa621a9c678b77ab",
    "bytes": 818,
    "created_at": 1737550680,
    "filename": "batch_test_data.jsonl",
    "object": "file",
    "purpose": "batch",
    "status": "pending",
    "status_details": null
}

but I have to provide the extra -F custom_llm_provider="azure" which I'd like to not have to.

Also, running

curl http://localhost:4000/v1/files \
    -H "Authorization: Bearer sk-1234" \
    -H "custom_llm_provider: azure"

or

curl http://localhost:4000/v1/files \
    -H "Authorization: Bearer sk-1234"

both give the same error:

{
    "error": {
        "message": "The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable",
        "type": "None",
        "param": "None",
        "code": "500"
    }
}

with the same above config: So it works to upload files, but not to fetch files. What is it I am missing? I have tried to find documentation on this but unable to find any.

Thank you! 🙌

[Jacobh2]

Looking through the code of LiteLLM, how the list files endpoint work, it still seems to try to fetch the custom_llm_provider from the request body as a fallback, which IMO is a bit strange consider it is a GET request so no body is to be expected to exist.