Is that possible to have the same training-MSEs for the same training data set with different weights at end of the training phase?
1 Ansicht (letzte 30 Tage)
Ältere Kommentare anzeigen
Darshana Abeyrathna
am 13 Jun. 2016
Kommentiert: Greg Heath
am 14 Jun. 2016
Matlab NN toolbox starts its weights and bias randomly. Therefore, I trained the network 200 times with random weights and bias to the same given training data set. I planed to get the minimum MSE out of 200 training MSEs, so that I can use the corresponding weights and bias in the testing phase. But in that case, I found it gives the same training-MSEs (most of the time) for the same training data set with different weight values at end of the training phase.
Is that possible? Can training stop at a local or global minima with different values for weights and bias?
really appreciate if anyone can clarify this issue.
Regards, Dara
0 Kommentare
Akzeptierte Antwort
Greg Heath
am 13 Jun. 2016
If you have H hidden nodes you can reorder them H! ways.
If you have an odd parity transfer function you can change the sign of its input weights and counter that by changing the sign of it's output weight.
Consequently, there are M = H! *2^H different nets that have identical input/output performance.
The default value of H = 10 yields M = 3.7159e+09.
Hope this helps.
Thank you for formally accepting my answer
Greg
2 Kommentare
Greg Heath
am 14 Jun. 2016
You misunderstand.
1. M is the number of rearrangements of the same weight values.
2. H is the number of nodes (neurons) in the single hidden layer. The MATLAB default is H = 10. How many input/target pairs do you have: N = ?
[ I N ] = size(input) % [ 192 N ]
[ O N ] = size(target)% [ 48 N ]
Assuming default trn/val/tst division ratios of 0.7/0.15/0.15, the number of training samples and equations are
Ntrn = N -2*round(0.15*N))
Ntrneq = Ntrn*O
whereas the number of unknown weights is
Nw = (I+1)*H+(H+1)*O = 1930+ 528 = 2458
Obviously, to have at least as many equations as unknowns you need AT LEAST
N >~ 2458/48 ~ 51
3. You might want to reduce I=192 using PLSregress
Hope this helps.
Greg
Weitere Antworten (0)
Siehe auch
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!