Using backpropagation on a pre-trained neural network
1 Ansicht (letzte 30 Tage)
Ältere Kommentare anzeigen
minomic
am 25 Mär. 2015
Beantwortet: minomic
am 27 Mär. 2015
I am developing a project about autoencoders (based on the work of G. Hinton) and I have a neural network which is pre-trained with some Matlab scripts that I have already developed. Now I need to perform a fine-tuning stage through backpropagation, and I am trying to use the 'Neural Network Toolbox'. All my data are already pre-processed (zero mean, unit variance, and so on) and I don't need any more pre-processing, nor post-processing. How do I disable them? Besides, I want this backpropagation stage to begin from the weights and biases found with the pre-training (that are stored in some matrices).
The code I have written so far is
% set the network hidden layers' size and the training function
net = feedforwardnet([layer1, layer2, layer3, layer4, layer3, layer2, layer1], 'traincgp');
% the training data are stored as rows in Dtrain matrix
net = configure(net, Dtrain', Dtrain');
net.divideParam.trainRatio = 70/100;
net.divideParam.valRatio = 15/100;
net.divideParam.testRatio = 15/100;
% initialize weights and biases to the value found with pre-training
% Note: the weights are symmetric but the biases are not
net.IW{1} = vishid';
net.LW{8,7} = vishid;
net.LW{2,1} = hidpen';
net.LW{7,6} = hidpen;
net.LW{3,2} = hidpen2';
net.LW{6,5} = hidpen2;
net.LW{4,3} = hidtop';
net.LW{5,4} = hidtop;
net.b{1} = hidrecbiases';
net.b{2} = penrecbiases';
net.b{3} = penrecbiases2';
net.b{4} = toprecbiases';
net.b{5} = topgenbiases';
net.b{6} = hidgenbiases2';
net.b{7} = hidgenbiases';
net.b{8} = visbiases';
net.trainParam.epochs = 200;
% set the transfer functions: logsig for every layer
% except for the last one which is linear
for ii=1:size(net.layers, 1)-1
net.layers{ii}.transferFcn = 'logsig';
end;
net.layers{size(net.layers, 1)}.transferFcn = 'purelin';
% train the net
net = train(net, Dtrain', Dtrain');
% save the new weights
weight_i1_b = (net.IW{1})';
weight_12_b = (net.LW{2,1})';
weight_23_b = (net.LW{3,2})';
weight_34_b = (net.LW{4,3})';
weight_45_b = (net.LW{5,4})';
weight_56_b = (net.LW{6,5})';
weight_67_b = (net.LW{7,6})';
weight_78_b = (net.LW{8,7})';
hidrecbiases_b = (net.b{1})';
penrecbiases_b = (net.b{2})';
penrecbiases2_b = (net.b{3})';
toprecbiases_b = (net.b{4})';
topgenbiases_b = (net.b{5})';
hidgenbiases2_b = (net.b{6})';
hidgenbiases_b = (net.b{7})';
visbiases_b = (net.b{8})';
% save everything to file
% ...
The problem is that it's not working the way I expect: the error seems to go down but the result (which is the reconstruction of an image) is worse that what I obtain by just pre-training.
Can anyone tell me what I am missing here? Thank you!
2 Kommentare
Akzeptierte Antwort
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu Image Data Workflows finden Sie in Help Center und File Exchange
Produkte
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!