Problems training deep neural network
2 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
I have a custom neural network with 3 hidden layers and I'm having trouble initializing it. The problem is that when training stops I get a very small gradient so no much learning in the last epochs but my error is still huge even for the training set! I've investigated the problem and it come out that my last layer of sigmas is saturated and there is no remarquable change for the weights of the early layers. I've red that this problem of vanishing gradient can be solved using the Relu transfer functions but now I have a huge error due to enormous weights. I'm stuck for 2 weeks tryings Relu and its variants (Leaky RElu and Elu ) with different training functions (trainrp and trainscg) with no success. I doubt that I need a good initialization but I can't find anything in Matlab other the NW and I thinks that's strange for Matlab. Can anyone help me or at least put me on the right truck ?! Thanks
Regards
Amine
1 Kommentar
Greg Heath
am 5 Jan. 2020
How are your inputs and targets normalized? i,e, what are the boundaries of each component?
Greg
Antworten (1)
Siehe auch
Kategorien
Mehr zu Deep Learning Toolbox finden Sie in Help Center und File Exchange
Produkte
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!