Added API support for local Zonos.

[PhialsBasement]

Add REST API Endpoints

This PR adds FastAPI endpoints to Zonos, allowing programmatic access to the model's functionality alongside the existing Gradio interface.

Added Features

• /models endpoint to list available models
• /generate endpoint for text-to-speech generation
• /speaker_embedding endpoint for creating speaker embeddings

Changes

• Added FastAPI integration
• Model responses are streamed as WAV files
• Added Pydantic models for request validation

Testing

Tested with curl commands:

• GET /models works as expected
• POST /generate successfully generates audio
• POST /speaker_embedding successfully creates embeddings

The implementation reuses existing model management code and runs alongside the Gradio interface on a different port.

[darkacorn]

i would maybe separate that into a different api file without gradio -
as you use one or the other most likely not the same time

and have a api consuming gradio ui - as a refactor - if that is the goal

also as a request -

maybe trying to keep in alignment to openai's tts api
that is very much integrated and supported everywhere,optional features as separate parameters

this would allow easy integration for 3rd party systems without much hassle and with sane defaults

[Steveboy123]

Thank you @PhialsBasement , you are a lifesaver.

[darkacorn]

thats more akin to what im proposing .. ( mind you uploading a voice file for every request to a remote mashine maybe suboptimal)

we may even want to isolate loading transformer and hybrid at the same time so there is no need to swap over .. models are small enough to fit even in peanut cards - ( model loading time would hurt throughput ) ( optional pinning or full override-able but i would make that the default behaviour for any load bearing api)

in an api scenario batch processor with queue could be prefixed with just what model to take as both are present in vram ( i work on that once we get a go ahead or at least a LGTM from the team)

voice could be embedded as tensors on voice upload - and on usege we just pull in the tensor to save computation

atm i support mp3/wav while always converting to wav as a baseline

happy to help out .. but i think api and gradio should be clearly separated .. can someone from zyphra chip in here ?

[zaydek]

Just want to mention this thread as relevant for when a teammate comes around to see this PR: #37.

[darkacorn]

agreed but that is different as there api has different sampling .. that should be compensate able once we know what they use
the model cond. has params for min p top k / top p / temp and rep_pen .. which are not exposed or used atm in oss only min_p for the time beeing

[Ph0rk0z]

With OAI endpoint and speakers from folder as returned voices it would work straight away in sillytavern. Unconditional emotions and it would be good "as-is".

[darkacorn]

With OAI endpoint and speakers from folder as returned voices it would work straight away in sillytavern. Unconditional emotions and it would be good "as-is".

pretty much why i proposed it that way .. integration in hundreds of systems would work w/o any extra work

[PhialsBasement]

@darkacorn just threw in some of your suggestions, check it out and tell me if its what you were thinking

[darkacorn]

amazing thanks for pulling that in, good baseline

[ther3zz]

I'm currently testing the openai endpoint, will report back if I run into any issues!
That being said, it makes sense to include a swagger docs endpoint as well (or at least some variable to enable/disable the docs page)

[ther3zz]

Has anyone been able to create embeddings? I'm running into this error:

{
    "detail": "'int' object has no attribute 'query'"
}

[PhialsBasement]

@ther3zz Fixed. Issue was in api.py, i was tryina use .query() on a CUDA stream handle, now its just a normal UNIX timestamp instead.

[ther3zz]

@ther3zz Fixed. Issue was in api.py, i was tryina use .query() on a CUDA stream handle, now its just a normal UNIX timestamp instead.

Looks like it's working!

[ther3zz]

Another issue I noticed is that MODEL_CACHE_DIR=/app/models doesnt seem to work. I'm not seeing the models cached there. I see them going here: /root/.cache/huggingface/hub/

[PhialsBasement]

Whack, ill look into it and see whats going on there

[Ph0rk0z]

Why can't we just load models from a folder we manually saved? I get that huggingface hub is used for docker, but not all of us are doing that.

[darkacorn]

i dont think there is anything that prevents it .. you can even use it offline with the hf client

[Ph0rk0z]

I've had to change loading to from_local in gradio and all. The from_pretrained is hijacked away from torch.

[mathematicalmichael]

hope this helps: HF hub config:
https://huggingface.co/docs/huggingface_hub/package_reference/environment_variables#hfhubcache

[PhialsBasement]

@ther3zz can you move this to issues tab over on my fork?

[ther3zz]

Another issue I noticed is that MODEL_CACHE_DIR=/app/models doesnt seem to work. I'm not seeing the models cached there. I see them going here: /root/.cache/huggingface/hub/

I don't actually see an issues tab on when on your fork