difference between true and apparent error in neural network

hi What is the difference between apparent and true error in neural network. My impression is that true error is the error related to test data set while apparent error is the error returned by neural network namely sum(abs(target-output)), and bias is the difference between these two errors, is my idea correct? can we use some other error measurements like MSE of MAE instead of these errors, for instance calculate bias from difference of training and test MSE? best

Answer 1

[Greg Heath]

Where did you find that terminology?

There are several quantities that can be identified.

 1. The underlying error-free I/O transformation 
 2. The contaminated sample (noise, interference and measurement error) from which the neural network is designed.
 3. The training, validation and test set targets and outputs.

The true error, which is unknown, is the difference between 1 and 2.

The true bias is the average of the true error.

The apparent error is the difference between the test set target and output.

The apparent bias is the average of the test set error.

For a good minimum mean-squared-error design, the test set bias should be zero and the test set variance is the mean-squared-error.

Hope this helps.

Thank you for formally accepting my answer

Greg

P.S. One of several basic assumptions is that the training, validation and test sets are sufficiently large. In other words, if they were larger, the difference in results would be insignificant.