Gradient accumulation + DDP scenario

[Dengligedeng]

Thanks very much for the great work. I think if grad acc and DDP is used, gradients will only do all-reduce in optimizer's closure calling time. Besides, the closure will only contains last batch's training data and can't do step2's gradient recalculation.