This repository was archived by the owner on Oct 31, 2022. It is now read-only.
This repository was archived by the owner on Oct 31, 2022. It is now read-only.
avg stays in 2.6-2.9 range #86
Description
I'm trying to finetune 355M, 744M models, but having issue with avg. it doesn't fall at all!
have tried learning_rate 0.00001, 0.00002 , 5e-5 - result is the same.