ema_restart_decay与decay衰减没有起到预期的效果 #331
Description
# EMA init
self.decay = decay
self.warmups = warmups
self.before_start = 0
self.start = start
self.updates = 0 # number of EMA updates
if warmups == 0:
self.decay_fn = lambda x: decay
else:
self.decay_fn = lambda x: decay * (
1 - math.exp(-x / warmups)
) # decay exponential ramp (to help early epochs)
for p in self.module.parameters():
p.requires_grad_(False)
# 部分使用
if self.ema:
self.ema.decay -= 0.0001
self.load_resume_state(str(self.output_dir / 'best_stg1.pth'))
print(f'Refresh EMA at epoch {epoch} with decay {self.ema.decay}')
对 ema 类中 self.decay 操作并不会影响到 self.decay_fn，因为 self.decay_fn 在初始化时就已经固定，在二阶段调整时，真正起到作用的只有 self.updates
同时我尝试修复了该问题并尝试训练，发现对最终结果原预期功能可以有部分负面影响，但是也可能是随机误差导致
修复前
Metrics: {'f1': 0.6143946020242409, 'precision': 0.6858697168363789, 'recall': 0.5564105465655765, 'iou': 0.37533777245904165, 'TPs': 4917, 'FPs': 2252, 'FNs': 3920}
Averaged stats:
Accumulating evaluation results...
COCOeval_opt.accumulate() finished...
DONE (t=3.92s).
IoU metric: bbox
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.428
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.602
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.459
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.223
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.474
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.615
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.343
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.568
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.634
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.396
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.697
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.836
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.849
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.681
best_stat: {'epoch': 159, 'bbox': 0.4277051512983154}
修复后
Metrics: {'f1': 0.6126069844443055, 'precision': 0.6838214783821478, 'recall': 0.5548262985175965, 'iou': 0.37417987036795197, 'TPs': 4903, 'FPs': 2267, 'FNs': 3934}
Averaged stats:
Accumulating evaluation results...
COCOeval_opt.accumulate() finished...
DONE (t=3.61s).
IoU metric: bbox
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.427
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.600
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.458
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.222
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.473
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.614
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.344
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.568
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.633
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.394
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.697
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.846
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.848
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.680
best_stat: {'epoch': 159, 'bbox': 0.42738959196056475}