Correct weight Initialization in CNN
20 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Andres Ramirez
am 29 Jul. 2018
Bearbeitet: Maria Duarte Rosa
am 5 Jul. 2019
When a very deep DAG network is built from scratch, the initialization of the weights made by matlab is not very good since it presents a vanishing gradient problem which causes the CNN not to learn.
What is the function with which Matlab does the initiation of CNN weights?
Why do you implement initialization functions in Matlab such as XAVIER or RELU AWARE SCALALED?
Thank you for your answers.
2 Kommentare
Greg Heath
am 31 Jul. 2018
I do not understand
"Why do you implement initialization functions in Matlab such as XAVIER or RELU AWARE SCALALED?"
Please explain.
Greg
Akzeptierte Antwort
Maria Duarte Rosa
am 5 Jul. 2019
Bearbeitet: Maria Duarte Rosa
am 5 Jul. 2019
In R2019a, the following weight initializers are available (including a custom initializer via a function handle):
'glorot' (default) | 'he' | 'orthogonal' | 'narrow-normal' | 'zeros' | 'ones' | function handle
Glorot is also know as Xavier initializer.
Here is a page comparing 3 initializers when training LSTMs:
I hope this helps,
Maria
0 Kommentare
Weitere Antworten (2)
Andres Ramirez
am 31 Jul. 2018
1 Kommentar
Greg Heath
am 1 Aug. 2018
Bearbeitet: Greg Heath
am 1 Aug. 2018
Do you have a reference for
RELA AWARE SCALALED
I have no idea what this is.
Thanks
Greg
fareed jamaluddin
am 4 Aug. 2018
I think you can take a look at this example https://www.mathworks.com/help/images/single-image-super-resolution-using-deep-learning.html
I am also looking for a way on weight initialization options, you can see in the example it create the initialization with He method for every conv layer.
0 Kommentare
Siehe auch
Kategorien
Mehr zu Image Data Workflows finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!