Calculation of Predictive Variance?

[leimao]

Hello Yarin,

It looks like that the description of the outputs in your predict method of the net class does not match to the actual output.

DropoutUncertaintyExps/net/net.py

Lines 95 to 108 in 6eb4497

	def predict(self, X_test, y_test):
	"""
	Function for making predictions with the Bayesian neural network.
	@param X_test The matrix of features for the test data
	@return m The predictive mean for the test target variables.
	@return v The predictive variance for the test target
	variables.
	@return v_noise The estimated variance for the additive noise.
	"""

According to your publication, the predictive variance should be the sample variance of T stochastic forward passes plus the inverse model precision tau. (In your case, because the output y is a scalar, the variance are also scalars.) But it looks like that you did not add the inverse of tau when you are calculating the predictive "rmse". In addition, what is the estimate variance with additive noise?

Thank you very much.

Best,

Lei

[malharjajoo]

I agree with this (although I'm not sure about the last bit: why you are mentioning that variance is required for calculating predictive RMSE ....)

[leimao]

I agree with this (although I'm not sure about the last bit: why you are mentioning that variance is required for calculating predictive RMSE ....)

It has been half a year since I opened this. Now I almost totally forgot what I have written. I will come back to this later Orz.