How to use /backend/*
endpoints
#1628
Unanswered
TheDarkTrumpet
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Good morning,
This morning I was about to open a bug request surrounding the ability to shutdown a model from the API. The use case is if I want to test different models, on a multi-GPU machine, to be able to shutdown the currently running one and load the new one in the case I need it.
I came across a pull request, #938 which mentioned a endpoint called
/backend/shutdown
. Upon playing with that (which appears to be a post), I'm unclear on what I should be sending to it.I also saw one for
/backend/monitor
, which I figured would be simpler. It's a GET request, and figured it'd show me the general status. Upon executing the request below, I get a 422 error code with an incredibly unhelpful error message. I was looking for API docs (like Text-Gen-WebUi has), as an endpoint, that can describe the APIs and what's expected and nada.Results in:
I'd love to debug this further on my end. I already dug into the source code, but my go skills aren't quite up to snuff. https://github.com/mudler/LocalAI/blob/master/api/api.go has some of the code, and https://github.com/mudler/LocalAI/blob/master/api/localai/backend_monitor.go has the rest for the monitor.
Any help or pointers would be useful.
Thanks,
Beta Was this translation helpful? Give feedback.
All reactions