Training (or Start/End of epoch) signal to Loss function

[liamaltarac]

If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with the documentation (for small docs fixes please send a PR instead).
The form below must be filled out.

Here's why we have that policy:.

Keras developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information.

TensorFlow version (you are using):
Are you willing to contribute it (Yes/No) :

Describe the feature and the current behavior/state.

Describe the feature clearly here. Be sure to convey here why the requested feature is needed. Any brief description about the use-case would help.

I am developing a custom loss function, that, throughout training, stores in memory previous loss values from each mini-batch, and uses those past values to compute the new loss (similar to reward calculations in RL) .

The problem I am facing is that (as far as I can tell), there is no way to tell the loss function whether or not it is in a training phase, or in a validation phase. Therefore, all the validation loss values get indiscriminately stored in memory as well, and get used in training in the later epochs.

I'm trying to make my loss function as user-friendly as possible, so ideally it could be simply called from model.fit() without any need for extra callbacks.

If possible, I think it would be nice if Loss.__call__() had an extra parameter (training) which would automatically be raised by model.fit() in the validation/training phase, similar to the training argument in tf.keras.layers.Layer.call().

Will this change the current api? How?
It would change the parameters in tf.keras.losses.Loss.__call__ by adding an extra (optional) training parameter.

Who will benefit from this feature?
Anyone who writes a custom loss function that requires different behaviour in training from testing.

Contributing

• Do you want to contribute a PR? (yes/no): No
• If yes, please read this page for instructions
• Briefly describe your candidate solution(if contributing):

[tilakrayal]

@liamaltarac,
Could you please elaborate about your Feature. Also, please specify the Use Cases for this feature. Thank you!

[liamaltarac]

@tilakrayal What I am describing would essentially be a way for a Loss function to differentiate between training and testing. Now that I think about it, I suppose I could simply use the tf.keras.backend.learning_phase flag, but in my original post I was describing an optional argument in Loss.__call__ that would be specified by Model.fit. I would look something like this : Loss.__call__(y_true, y_pred, training)

[fchollet]

I am developing a custom loss function, that, throughout training, stores in memory previous loss values from each mini-batch, and uses those past values to compute the new loss (similar to reward calculations in RL) .

It sounds like such a loss would not be differentiable -- the terms you're adding would just be treated as constants with 0 gradient. Perhaps you meant to use a Metric here rather than a loss?

[liamaltarac]

@fchollet Currently I am implementing more of an objective function, but I am just inheriting from the Loss class. Right now, with only backprop, I am able to maximize my objective; so I don't know if all the functions that I'm using are differentiable, but so far it doesn't seem to cause any problems (does a function need to be fully differentiable, or will Keras just ignore 0 gradients ?). I don't just want a metric, because I need to train on that value. So in reality, I'm not implementing a loss function, but an objective function that is acting as a loss function in the way that it gets called by model.compile and model.fit