MATLAB Answers

0

Overfitting of Regression Plot for a Feedforward Neural Network

Asked by Zeeshan Ahmad Khan on 29 Jan 2019
Latest activity Commented on by Zeeshan Ahmad Khan on 30 Jan 2019
I am trying to make an estimation for the state-of-charge of a battery using the Feedforward Network. My inputs to the network are current, voltage and time. The network has 3 inputs (Voltage, Current and Time) with 1 Output (State-of-charge(SOC)). The network consists of 2 hidden layers with 10 neurons each and output layer with 1 neuron. Each Input has 54043 samples, so has the output.
I am training the network using the 'trainlm' and have set the performance goal to 1e-5. However, after training the network, the regression plot seems to have a R of 0.9999 in all the cases, which indicate to me that the network I am using is Overfitting the target.
Attched is the code and mat file containing the measurement data, any leads would be appreciated.
Below is the obtained regression plot:

  0 Comments

Sign in to comment.

1 Answer

Answer by Greg Heath
on 30 Jan 2019
 Accepted Answer

OVERFITTING IS NOT "THE" PROBLEM !!!
In general, the problem is
OVERTRAINING an overfit net.
My solution is simple: DO NOT OVERFIT !!!
Use no less training equations Ntrneq than the number of unknown weights Nw, i.e.
Ntrneq >= Nw
For a double hidden layer I-H1-H2-O = 3-10-10-1 net,
Ntrneq = Ntrn*O = 0.7 * 54043 * 1 = 37,830
Nw = ( I + 1 )*H1 + (H1+1)*H2 + (H2+1)*O
= 4*10 + 11*10 + 11*1
= 161
OBVIOUSLY, YOU HAVE SO MUCH DATA THAT YOU COULD HAVE EVEN USED MANY, MANY MORE HIDDEN NODES!
HOPE THIS HELPS.
Thank you for formally accepting my answer.
Greg
Greg

  1 Comment

Thank you so much Greg, your insight helped me solve the problem.

Sign in to comment.