How to create a positional encoder layer using deep network designer?
8 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Im creating the network architecture of my transformer but I don't know how to put the positional information "positional encoding" with Deep Network Designer, this is a code that I have so far...
function lgraph = setNeuralNetworkArchitecture(inputSize, numClasses)
% Create layer graph
lgraph = layerGraph();
% Add layer branches
tempLayers = [
sequenceInputLayer([13 24 8],"Name","sequence")
% Añadir positional encoding
lambdaLayer(@(x) addPositionalEncoding(x, encodingDim), 'Name', 'positionalEncoding')
flattenLayer("Name","flatten")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = selfAttentionLayer(16,64,"Name","selfattention");
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
additionLayer(2,"Name","addition")
fullyConnectedLayer(6,"Name","fc_1")
softmaxLayer("Name","softmax")
classificationLayer("Name","classoutput")];
lgraph = addLayers(lgraph,tempLayers);
% clean up helper variable
clear tempLayers;
lgraph = connectLayers(lgraph,"positionalEncoding","flatten");
lgraph = connectLayers(lgraph,"flatten","selfattention");
lgraph = connectLayers(lgraph,"flatten","addition/in2");
lgraph = connectLayers(lgraph,"selfattention","addition/in1");
end
0 Kommentare
Antworten (1)
Siraj
am 24 Aug. 2023
Hii! @Luis Gabriel Macias Santillan It is my understanding that you want to add a positional encoder layer using the deep network designer app available in MATLAB. According to my research, the deep network designer doesn’t have a positional encoder layer that could be directly added to the network.
You can refer to the following documentation to understand how we can integrate a custom deep-learning layer.
Also, you can check out the following file, which has an implementation for a positional encoder.
Hope this helps.
0 Kommentare
Siehe auch
Kategorien
Mehr zu Image Data Workflows finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!