Is there a solution to not reload the model if there's only system prompt changed? #3589
Unanswered
sebaxakerhtc
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi all!
I really trust you will help me with my idea.
I have a single GPU with 5GB VRAM, where is located a single Ollama model for my purposes (chatbots).
It works very fast and just perfect!
So, I created second workflow with exactly the same setting and model, but changed SYSTEM prompt and using it on another website.
But I don't understand why it reloads the same model (unloads and loads it again) for each website.
All the settings are the same! Only SYSTEM prompt changed.
For example, when I use the same model for different SYSTEM prompt with OpenWebUI - there's no problems, model loaded and only answers changed.
Please, help me with that. Is it possible to not unload and load again the same LLM?
A half-related issue here about missed ollama functional
KEEP_ALIVE=-1
Beta Was this translation helpful? Give feedback.
All reactions