Run Low Parameter Model

[Shaf2665]

It would be good if we reduce the idle resource consumption, such that it can able to run low parameter models without the need of GPU.

[Hiburger]

Hi!

Thank you for the suggestion 😊

The service we use to handle the models, ollama, unloads models from memory (VRAM or RAM) after 5 minutes of inactivity. This behavior is designed to free up resources when the model is not in use. We can tweak this if needed 👍

Also, we're planning on letting users choose their preferred AI model from a pre-selected list during OS setup, from small to bigger models (if they're interested in using AI-based features). This way, they can pick one that suits their needs and won't put too much strain on their computer! 😀