Is it possible to use LocalAI in combination with OpenAI Evals ?

[VitoVoelker]

Is it possible to use OpenAI evals framework to evaluate LLM with Local AI ? For OpenAI evals a api key has to be provided, but as far as i know LocalAI replaces the API base but not the key. I have tried using openai evals but I am getting errors for missing api keys. Anyone else tried this ?

Thanks for any answers in advance :)