How to use Nadam optimizer in training deep neural networks

Training_Options = trainingOptions('sgdm', ...

'MiniBatchSize', 32, ...

'MaxEpochs', 50, ...

"InitialLearnRate", 1e-5, ...

'Shuffle', 'every-epoch', ...

'ValidationData', Resized_Validation_Data, ...

'ValidationFrequency', 40, ...

"ExecutionEnvironment","gpu",...

'Plots','training-progress', ...

'Verbose',false);

[Joss Knight]

What is your question?

Answer 1

[Nayan]

Hi

I assume you want to use "adam" optimizer in place "sgdm". You need to simply replace the "sgdm" key with "adam" keyword.

options = trainingOptions("adam", ...
    InitialLearnRate=3e-4, ...
    SquaredGradientDecayFactor=0.99, ...
    MaxEpochs=20, ...
    MiniBatchSize=64, ...
    Plots="training-progress")