Will train(net) continue training 'net' if it hasn't been initialised?

Hello,

I am trying to train a neural network in batches. I have the train(net) function in a loop and would like to get a better understanding of how it works.

My code is something like this:

% I define my network:
net = feedforwardnet(hiddenLayerSize,trainFcn);
% Some more functionalities and details about net 

% start training loop
for epoch = 1:maxepoch
    for i = 1:10 % let's say I have 10 batches
        load(data(i))
        net = train(net, inputs, targets);
    end
    % Some code down here where I check the performance with validation data and break if I meet a condition
end

My question is, when I do "net = train(net, input(:, i), target(:, i))", am I introducing the trained net from the previous iteration and it's updating from that or is it initialising the weights each time? If the latter, how can I avoid/stop this?

TIA

Answer 1

[Divya Gaddipati]

The network is initialized only once when you use

net = feedforwardnet(hiddenLayerSize, trainFcn) 

And when you do “net = train(net, inputs, targets)” in a loop, the network gets updated with the new weights and biases, and this updated network is used in the next iteration and so on.

You can also use pre-existing deep learning functionalities. For that, you would have to transform your feedforward net into a simple deep learning network that only has 1 input layer, 1 fully connected layer, 1 custom layer and 1 output classification layer. Define the custom layer as the tansig activation function of the feedforward nets. This would reproduce a standard feedforward net.

This approach automatically uses stochastic gradient descent as the training algorithm, which works with mini-batches of data.

Please refer to the following link for more information on how to create custom layers:

https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layers.html