Help with single GPU computing for training neural networks

Hello,

I am trying to speed up training of a time-delayed neural network by means of GPU computing. My code works fine with no parallel processing (so no errors there).

I have followed the instructions on single GPU computing from here and here and when adding the extra name-value pair 'useGpu','yes' I receive the following error:

Error using trainlm (line 105)
Layer states Ai is not a matrix or cell array.

Error in network/train (line 106)
[net,tr] = feval(net.trainFcn,net,X,T,Xi,Ai,EW,net.trainParam);

Error in test_nn (line 28)
net2 = train(net,Xs,Ts,Xi,'useGPU','yes');

I did not find any other resources on this so any help would be kindly appreciated.

[Mark Hudson Beale]

Sorry to hear about the problem. Could you give some more information such as:

1) The release of MATLAB you are using? It will be returned by "ver". Example: R2013a 2) The script prior to the call to train? If you are training with time series then data dimensions, call to preparets, etc. might shed light on what is causing the error. Thanks

Answer 1

[moustafa]

i have the same problem ... :(

but i think, the reason is, as shown in the example there, the trained network is a simple feed forward network while you (and me) are trying to train NAR(X) network which need the previous time step.

so, in other words, the problem is, can NAR(X) network be trained using parallel computing toolbox??

[Mircea Stoica]

Hi moustafa,

I've also seen the example but I am not training NAR(X) networks, but time-delayed networks instead. Quote:

if the network has only input delays, with no layer delays, the delayed inputs can be precalculated so that for the purposes of computation, the time steps become different samples and can be parallelized. This is the case for networks such as timedelaynet and open-loop versions of narxnet and narnet. If a network has layer delays, then time cannot be "flattened" for purposes of computation, and so single series data cannot be parallelized. This is the case for networks such as layrecnet and closed-loop versions of narxnet and narnet

From this I understand that it should work in my case.

[Xiangming]

Hi Mircea,

Very goog explaination. But when I set the feedbackdelays to be zeros, errors occurr.

    % code
   min feedbackdely is zero causing zero-loop delays...

end

So I'm wondering ----can NAR(X) network be trained using parallel computing toolbox??

[Greg Heath]

Although an input delay of zero is allowable, a feedback delay of zero is not allowed.

Hope this helps.

Greg