AMPL loss calculation method.

[MLPharmaTron]

Hi AMPL staff.

I've been running into some training failure as my dataset goes up. One potential reason I want to explore is additive error in metrics calculation.

1. Does AMPL calculate R2/RMSE per batch and average, or calculate the values once over the whole dataset?
2. If it is the former, is there a way to switch it to the latter?

Thanks in advance.

[stewarthe6]

Hello,

I believe we do the later and calculate the metrics over the whole dataset per epoch.

Maybe if you share a stack-trace I can see where the error is coming from.

Thanks

[MLPharmaTron]

Hi Stwearthe6,

There is no traceback error if that is what you're referring to (>). The failure was training failure, that is on the R^2 v epoch there is no training and just a reduction in score or haphazard fluctuations. Since investigating further this might be a model complexity problem, as these issues only precipitated once we went from 3M SMILES to 5M.

I want to investigate this route first as I strongly believe this to be the case (a GCNN with 5 layers 256-->16 neurons).

Thanks.

[stewarthe6]

Hello,

It's possible to set the max_epochs parameter to stop the training when you observe strange R^2 behavior. Then you can make predictions on your training set and verify that the metrics are calculated correctly. Maybe you'll find a few compounds with high magnitude predictions.