-
How can I release a model and free up memory before loading a new one? I tried model.cleanup() but that doesn't seem to do anything, in terms of VRAM. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 1 reply
-
okay ... no idea if that's the right way to do it, but it seems to work. I'll write the answer here, in case someone else is looking for it: given a model previously loaded like this :
this seems to free up everything :
|
Beta Was this translation helpful? Give feedback.
-
Python is a managed language, so there's nothing you need to do to free up the model other than removing all references to it. In fact there's nothing else you can do, since
You can force garbage collection with: import gc, torch
...
gc.collect()
torch.cuda.empty_cache() This takes a little while to complete (some milliseconds) so you won't want to call it constantly, but between unloading a model and loading the next one it should be fine. Just remember that any (indirect) reference to the model forces the garbage collector to retain it and anything else that it references in turn. |
Beta Was this translation helpful? Give feedback.
Python is a managed language, so there's nothing you need to do to free up the model other than removing all references to it. In fact there's nothing else you can do, since
del
only destroys the reference, not the object being referenced.del model
is largely equivalent tomodel = None
. Neither will free any memory used by the model unlessmodel
is the last remaining reference. Even then the garbage collector and PyTorch's CUDA cache can take a little while to catch up, so you may not see memory become available right away (and in some cases you may not actually have the memory to allocate from, even though it should be available in theory.)You can force garbage collection with: