Why does the TRAIN function within the Neural Networks Toolbox return incorrect converged errors and network outputs when using BFGS training?

My neural network uses the TRAINBFG function as its "trainFcn" property. After training, the output argument "E" should contain the final network output errors, and "Y" should contain the converged network outputs. They actually contain the initial output errors and outputs.

The following code illustrates the problem:

p = [0 1 2 3 4 5 6 7 8];
t = [0 0.84 0.91 0.14 -0.77 ...
-0.96 -0.28 0.66 0.99];
net = newff([0 8],[10 1],...
{'tansig' 'purelin'},'trainbfg');
net.trainParam.epochs = 50;
net.trainParam.goal = 0.01;
[net,tr,Y,E,Pf,Af] = train(net,p,t);
y2=sim(net,p);
final_outputs = y2
Y
final_error = t-y2
E

This results in:

 final_outputs =

   Columns 1 through 3 

     0.0830    0.7615    0.8620    

   Columns 4 through 6 

     0.0412   -0.6370   -0.8537   

   Columns 7 through 9 

     -0.1970   0.6032    1.0597

 Y =

   Columns 1 through 3 

     1.8211    0.7829   -0.7174  

   Columns 4 through 6

    -2.5872   -3.1501   -4.4639   

   Columns 7 through 9 

    -5.0563   -4.0460   -4.6061

 final_error =

   Columns 1 through 3 

    -0.0830    0.0785    0.0480 

   Columns 4 through 6

     0.0988   -0.1330   -0.1063

   Columns 7 through 9 

     -0.0830   0.0568   -0.0697

 E =

   Columns 1 through 3 

    -1.8211    0.0571    1.6274    

   Columns 4 through 6

     2.7272    2.3801    3.5039    

   Columns 7 through 9 

     4.7763    4.7060    5.5961

The converged error is "final_error", whose elements are the differences between the converged network outputs and the targets. The values in the "E" output argument of TRAINBFG should be equal to those in "final_error".

Answer 1

[MathWorks Support Team]

This is a bug in the way the TRAINBFG function within the Neural Network Toolbox updates the network outputs and errors during training.

Currently, to work around this, use the SIM function to find the network outputs after the network has completed training. The final errors are computed by finding the difference between the final outputs and the targets. The following sample code illustrates this:

p = [0 1 2 3 4 5 6 7 8];
t = [0 0.84 0.91 0.14 ...
-0.77 -0.96 -0.28 ...
0.66 0.99];
net = newff([0 8],[10 1],...
{'tansig' 'purelin'},'trainbfg');
net.trainParam.epochs = 50;
net.trainParam.goal = 0.01;
[net,tr,Y,E,Pf,Af] = train(net,p,t);
y2=sim(net,p);
final_outputs = y2
final_error = t-y2