Is there a solution to not reload the model if there's only system prompt changed? #3589
Unanswered
sebaxakerhtc
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi all!
I really trust you will help me with my idea.
I have a single GPU with 5GB VRAM, where is located a single Ollama model for my purposes (chatbots).
It works very fast and just perfect!
So, I created second workflow with exactly the same setting and model, but changed SYSTEM prompt and using it on another website.
But I don't understand why it reloads the same model (unloads and loads it again) for each website.
All the settings are the same! Only SYSTEM prompt changed.
For example, when I use the same model for different SYSTEM prompt with OpenWebUI - there's no problems, model loaded and only answers changed.
Please, help me with that. Is it possible to not unload and load again the same LLM?
A half-related issue here about missed ollama functional
KEEP_ALIVE=-1
Beta Was this translation helpful? Give feedback.
All reactions