You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The default checkpoint file naming scheme uses only epoch number and step number as keys for making checkpoint files names different for different epochs/steps. Such naming scheme is not sufficient when many checkpoints are created withing the same notebook for different models (or same models but with different hyper parameters). We should an adaptive naming scheme that accounts for
(optionally) Jupyter notebook name
Model class name or/and experiment id
run id (needed when the same model is trained multiple times)
We should also incorporate a warning mechanics that warns a user about the checkpoint directory growing to much due to containing too many outdated checkpoints
The text was updated successfully, but these errors were encountered:
pgagarinov
changed the title
Use checkpoint names unique for each trial within the jupyter notebook
Use more sophisticated checkpoint naming scheme
Feb 1, 2021
The default checkpoint file naming scheme uses only epoch number and step number as keys for making checkpoint files names different for different epochs/steps. Such naming scheme is not sufficient when many checkpoints are created withing the same notebook for different models (or same models but with different hyper parameters). We should an adaptive naming scheme that accounts for
We should also incorporate a warning mechanics that warns a user about the checkpoint directory growing to much due to containing too many outdated checkpoints
The text was updated successfully, but these errors were encountered: