How can I freeze layers for training a network with multiple outputs and reduce time for the training?
24 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Brian Park
am 12 Jul. 2023
Kommentiert: Brian Park
am 15 Jul. 2023
Since I try to train the model with multiple outputs, I have an issue to use 'deepNetworkDesigner' and its Training option.
So, I follow the direct training method on this example with freezing net's layers using the code below
lgraph = net.layerGraph;
target = 290
for i = 1 : target
try
L = freezeWeights(lgraph.Layers(i));
lgraph = replaceLayer(lgraph,lgraph.Layers(i).Name,L);
catch
end
end
net = dlnetwork(lgraph)
I checked WeightLearnRateFactor and BiasLearnRateFactor become zero, which means the layers are frozen.
However, it still takes too much time on the stage of training.
So,
Q1: is this the right way to freeze layers for the training multiple output network?
Q2: How can I reduce time for training, ignoring the layers which are frozen.
Here is the base code from example that I used for network training
[loss,gradients,state] = dlfeval(@modelLoss,net,X,T1,T2);
function [loss,gradients,state] = modelLoss(net,X,T1,T2)
[Y1,Y2,state] = forward(net,X,Outputs=["softmax" "fc_2"]);
lossLabels = crossentropy(Y1,T1);
lossAngles = mse(Y2,T2);
loss = lossLabels + 0.1*lossAngles;
gradients = dlgradient(loss,net.Learnables);
end
0 Kommentare
Akzeptierte Antwort
Aniketh
am 12 Jul. 2023
Yeah this a right way to freeze the layers, as long as you are not seeing any drastic changes in the output it shouldn't be any different for the case of predicting multiple outputs.
To further reduce training time and ignore the frozen layers, you can modify the modelLoss function to only compute and backpropagate gradients for the unfrozen layers using the find function in matlab. Here's a sample code for how this will work:
function [loss, gradients, state] = modelLoss(net, X, T1, T2)
[Y1, Y2, state] = forward(net, X, Outputs = ["softmax" "fc_2"]);
lossLabels = crossentropy(Y1, T1);
lossAngles = mse(Y2, T2);
loss = lossLabels + 0.1 * lossAngles;
% Compute gradients only for unfrozen layers
unfrozenLayers = find([net.Layers.WeightLearnRateFactor] > 0);
gradients = dlgradient(loss, net.Learnables(unfrozenLayers));
end
1 Kommentar
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu Image Data Workflows finden Sie in Help Center und File Exchange
Produkte
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!