Patches for remote sync waiting and log_local checkpointing#290

Open

jeffreywpli wants to merge 11 commits intomainfrom allow_nccl_timeout_increase

+21-9

Commits on Jul 2, 2024

add param
Jeffrey
committed
add timeout to both
Jeffrey
committed
fix type to datetime.timedelta
Jeffrey
committed
fix typo
Jeffrey
committed

Commits on Jul 3, 2024

fix typo in 2nd location
Jeffrey
committed
do not sleep before first attempt at remote sync
Jeffrey
committed

Commits on Jul 4, 2024

patch args.log_local
Jeffrey
committed
change rank0_only behavior when log_local
Jeffrey
committed
fix sleep behavior only for last iteration, also address the parent dir issue
Jeffrey
committed
0 sleep for initial sync
Jeffrey
committed
linting
Jeffrey
committed