can i use a custom activation function in this neural network and in what way?
11 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Dimitris Kastoris
am 17 Feb. 2023
Kommentiert: Dimitris Kastoris
am 23 Mär. 2024
x = linspace(0,1,10000)';
inputSize = 1;
layers = [
featureInputLayer(inputSize,Normalization="none")
fullyConnectedLayer(10)
sigmoidLayer
fullyConnectedLayer(1)
sigmoidLayer];
dlnet = dlnetwork(layers);
numEpochs = 15;
miniBatchSize =100 ;
initialLearnRate = 0.5;
learnRateDropFactor = 0.5;
learnRateDropPeriod = 5;
momentum = 0.9;
icCoeff = 7;
ads = arrayDatastore(x,IterationDimension=1);
mbq = minibatchqueue(ads,MiniBatchSize=miniBatchSize,MiniBatchFormat="BC");
figure
set(gca,YScale="log")
lineLossTrain = animatedline(Color=[0.85 0.325 0.098]);
ylim([0 inf])
xlabel("Iteration")
ylabel("Loss (log scale)")
grid on
velocity = [];
iteration = 0;
learnRate = initialLearnRate;
start = tic;
% Loop over epochs.
for epoch = 1:numEpochs
% Shuffle data.
mbq.shuffle
% Loop over mini-batches.
while hasdata(mbq)
iteration = iteration + 1;
% Read mini-batch of data.
dlX = next(mbq);
% Evaluate the model gradients and loss using dlfeval and the modelGradients function.
[gradients,loss] = dlfeval(@modelGradients, dlnet, dlX, icCoeff);
% Update network parameters using the SGDM optimizer.
[dlnet,velocity] = sgdmupdate(dlnet,gradients,velocity,learnRate,momentum);
% To plot, convert the loss to double.
loss = double(gather(extractdata(loss)));
% Display the training progress.
D = duration(0,0,toc(start),Format="mm:ss.SS");
addpoints(lineLossTrain,iteration,loss)
title("Epoch: " + epoch + " of " + numEpochs + ", Elapsed: " + string(D))
drawnow
end
% Reduce the learning rate.
if mod(epoch,learnRateDropPeriod)==0
learnRate = learnRate*learnRateDropFactor;
end
end
0 Kommentare
Akzeptierte Antwort
Christopher Erickson
am 17 Feb. 2023
There are two solutions I would suggest first:
If you have an elementwise activation function with no learnables (such as "exp" or "sin") you could use "functionLayer". For more generic workflows you could use a custom layer. I would only suggest using a custom layer if your activation function did not operate elementwise or if it has learnables; such as "prelu" in the examples.
Good luck!
2 Kommentare
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu Install Products finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!