Move Model to RAM temporaly #727
RodrigoSantiago
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Currently, there's a
free_params_immediately
option, which is quite useful when we're not going to use the model any time soon. However, it would be nice to be able to leave the weights that won't be used in RAM, for fast reading, until we need them again. This way, the total VRAM usage when generating the VAE (or anything else) is reduced, eliminating the need to reload the entire model from the hard drive.Beta Was this translation helpful? Give feedback.
All reactions