Elman Neural Network

Hi everybody,

I'm trying to train an Elman neural network for predicting time series. My observation matrix dimension is 15*30000 and for every 15 inputs, one sample is as target. I use an Elman network which has 10 neurons in hidden layer.

net = newelm(Pseq,Tseq,10,{'tansig' 'purelin'},'traingdx');

It takes at least 12 hours to train such a network for 25 epochs. Is it normal?

Is there any trick to train the Elman network faster?

Answer 1

[Greg Heath]

If

[ I N ] = size(P)

[ O N ] = size(T)

The number of training equations is

Neq = prod(size(T)) = N*O

For an I-H-O elman net the number of weights to be estimated is

Nw = net.numWeightElements

    = (I+1+H)*H+(H+1)*O = H^2 + (I+O+1)*H + O

The following condition is usually sufficient for successful training

Neq = N*O >> Nw

Depending on the complexity of the underlying I/O relationship

Neq/Nw ~ 10 to 30

is a reasonable range to investigate

Therefore, for I-H-O = 15-10-1

N >> H^2 + (I+O+1)*H + O = 100+170+1 = 271

Taking into consideration that H < 10 may be adequate,

Try starting your search with

N = 3000 for H = 1:10 and Ntrials = 10 different weight initializations

Hope this helps.

Greg