Using Matlab Neural Network Toolbox in real-time application

I am trying to implement Neural Fitted Q iteration as described by Riedmiller (2005): http://ml.informatik.uni-freiburg.de/_media/publications/rieecml05.pdf This is a real-time application, where I update the weights of the neural network using the samples stored over a number of iterations (so, a small batch). I have tried to use the Neural Network Toolbox, but I have found out that I cannot update the weights of the neural network when I want to do it.

Trying to be a bit clearer, I use a function to initialize the neural network, which contains the following code:

net = feedforwardnet(5,'trainlm');    % say I use 5 hidden nodes to start with
net.trainParam.showWindow = false;
net.numInputs = 2;                    % say I have 2 inputs
net.inputs{1}.range = [-1 1;-1 1];    % any range in fact
% Network weight and bias initalization: 
net = init(net);

Now, with the code above I have initialized the weight of the neural network, but if I try to run

y = net([x1;x2]);

the network would return an empty matrix. This is a problem for the first iteration of the loop in the Fitted Q-learning program, since I have to call this function before I have any actual data with which to update the neural network. So, a solution could be:

net = configure(net,[0;0],0);  % configuring neural network to 2 inputs, 1 output
net = init(net);

or otherwise:

net = train(net,[7;0],1);

But by doing this, I then observe that the neural network will return 0 from now on indefinitely, i.e. I CANNOT update the weights of the neural network anymore despite training it again multiple times in the main function whenever I have a new batch of data (at first 20 items, then 40, then 60 until I stabilize with 100 values).

So, is there a way I can use the neural network in a pseudo-real-time application? Should I store the weights in external variables, then use them as input before the next training step? Any other solutions?

Thank you in advance for your help!

Answer 1

[Greg Heath]

net = init(net)

> Now, with the code above I have initialized the weight of the neural network,

No. All weights are either 0 or [] !

See (no semicolon)

 wb = getwb(net)
 IW = net.IW
 b  = net.b
 LW = net.LW

Hope this helps.

Thank you for formally accepting my answer

Greg

[Enrico Anderlini]

Thank you, Greg. You are right. However, my problem is still there.

How can I initialize the weights of the neural network to small random values, and then update them in real time whenever I have a new batch of data?

Thanks!

[Greg Heath]

 help configure

 doc configure