Skip to content

Release a loaded model #303

Closed Answered by turboderp
sammyf asked this question in Q&A
Oct 8, 2023 · 2 comments · 1 reply
Discussion options

You must be logged in to vote

Python is a managed language, so there's nothing you need to do to free up the model other than removing all references to it. In fact there's nothing else you can do, since del only destroys the reference, not the object being referenced.

del model is largely equivalent to model = None. Neither will free any memory used by the model unless model is the last remaining reference. Even then the garbage collector and PyTorch's CUDA cache can take a little while to catch up, so you may not see memory become available right away (and in some cases you may not actually have the memory to allocate from, even though it should be available in theory.)

You can force garbage collection with:

import gc

Replies: 2 comments 1 reply

Comment options

You must be logged in to vote
0 replies
Comment options

You must be logged in to vote
1 reply
@sammyf
Comment options

Answer selected by sammyf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants