Diference between calculation of rSME and lossfunction during training of a neuralnetwork

Hello,

I am doing a code in matlab, and i would like to know why values of rsme and lossfunction are not the same or the square root of the other, during training. Both use train and validation data and both are the mean of the squares of errors but why, in each epoch, cant i relate one to the other?....

Thank you.

Answer 1

[Binaya]

Hi Ana

I believe you want to understand why the loss functions of your neural network does not output the same values as rmse function.

The difference can occur due to various reasons. Here are some general reasons on why the two values are different:

1. The loss function might contain regularization terms along with the rmse function leading to different values.
2. The input to loss function while training the network might be normalized as is generally the case to avoid drastic changes in gradients and make the training faster.. You can check is the inputs to loss functions and rmse are exactly the same.