Predictions using NARX Network

I am trying to build a network to do some long term predictions. My data is comprised of an INPUT and TARGET that spawns over 50 years (about 3500 points of data). At first I used the GUI to quickly get a network using default values. The network response seemed good but the ERROR Correlation and INPUT-ERROR cross correlation seem off( from what I understood from reading around here and the documentation, the peak should be at 0 lag). I tried adjusting the delays according to what I read in other question using the correlations but I don't understand how this works exactly. Where do I look to find the correct number of delays?

Another question I have is for long term predictions.Using the GUI I trained the network using around half of the data available, the network returned a very good approximation (with very little error between output and targets). Then in the next tab I used the TEST NETWORK and used the remaining points to see if it could predict the rest of the data. I would expect that at some point the error between output and target would grow but what I generally get is an excelent result where the output seems to be just a bit shifted under the target. (looks like the network learned everything when it is tested)

How can I correctly form predictions outside of the data I have?

I hope I was clear in my query.

Answer 1

[Greg Heath]

You do not explain how you obtained your correlation functions.

Before you design a NARX, obtain the target-target autocorrelation function and the target-input cross-correlation function. Find the significant delays corresponding to peaks above the 95% confidence level of the cross-correlation of the target with random Gaussian noise

If you use nncorr, search my posts to avoid bugs in the code (e.g., the correct cross correlation function is not symmetric about zero lag).

greg nncorr

greg narxnet

Otherwise use ifft(conj(fft(x)).*fft(t)),or xcorr or crosscorr functions in other toolboxes .

The only thing that should have a peak at zero lag is the target autocorrelation function.

In order to predict future data, you need the preceding values of input and target to fill the delay buffers.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Pedro]

Hi, thank you for your reply, sorry for the late reply.

I didn't obtain the correlation functions because I haven't understand how to read the LAG correctly and how it relates exactly with the network delays. If you could help with that it would be much appreciated.

Also, you say: "In order to predict future data, you need the preceding values of input and target to fill the delay buffers."

If I understand this, you mean the Input delays and FeedDelays that we learn from the correlations, correct?

But after, what should be used: a closed loop or open loop?

Thank you for your time.

[Greg Heath]

With NARXNET, if you want to predict beyond your design outputs using just future known inputs, convert your open-loop to a closed-loop.

If the closed-loop doesn't work well on design data, it can be trained on the design data starting from the design that had been converted from open-loop.

Then use the future inputs with the closed-loop design.

Hope this helps.

Thank you for formally accepting my answer

Greg

[Pedro]

"If the closed-loop doesn't work well on design data, it can be trained on the design data starting from the design that had been converted from open-loop."

Can you please clarify this for me?

For instance, I have data from 1 - 30. I train my open network until 20 and want to see if it predicts the remaining 10. Lets call it net1.

I close net1 and becomes net2 and simulate this new net with the new input. If the output is not good enough, I train net2 with the same data used with net1? Effectivly training the network again but now in close mode?

Thank you in advance.

Answer 2

[Greg Heath]

1. Design multiple open-loop (OL) nets in a double loop over the number of hidden nodes (outer loop) and random weight initializations(inner loop). For examples, search NEWSGROUP and ANSWERS with

greg Hub Ntrials

2. Use divideblock instead of dividerand to preserve correlations

3. Use the validation MSE tr.best_vperf to choose the best design and test MSE tr.best_tperf to estimate performance on unseen data.

4. To use the net on unseen data with only known inputs, convert the OL design to closed-loop (CL).

5. Evaluate the CL net on the design data.

6. If performance is significantly worse than the OL performance, use train to improve the CL performance.

7. Use the CL design with future inputs to predict future outputs.

8. If you know the corresponding future targets, you can evaluate the result.

Hope this helps.

Greg

[Pedro]

Thank you for your reply. I think I understood, except point 6. You mean train only the CL net, or do I train everything again? Because I tried training only the CL with the design data but never worked.

As an exercise I am using a Sinus to learn and try to predict the rest. After the net is closed I used the design data to see how it responds and gave her the new input data to predict the rest of the sinus but it didnt work.

Below the code ( I am using the advanced script generated by the GUI and then adapted it):

    clear,clc,close all
    % Solve an Autoregression Problem with External Input with a NARX Neural Network
    % Script generated by NTSTOOL
    % Created Fri Jun 07 14:58:24 CEST 2013
    %
    % This script assumes these variables are defined:
    %
    %   input - input time series.
    %   target - feedback time series.

% The real function where the net will learn a part until 730 and 
% try to predict the rest

    realin = linspace(1,1800,1800);  
    real = sin(2*pi*realin/365);

    plot(real,'c')
    hold on

    input = linspace(1,730,140);
    target = sin(2*pi*input/365);

    pred_in = linspace(731,1800,140);
    pred_target = sin(2*pi*pred_in/365);

    inputSeries = tonndata(input,true,false);
    targetSeries = tonndata(target,true,false);

    predinSeries = tonndata(pred_in,true,false);
    predtargSeries = tonndata(pred_target,true,false);

    % Create a Nonlinear Autoregressive Network with External Input
    inputDelays = 1:3;
    feedbackDelays = 1:3;
    hiddenLayerSize = 10;
    net = narxnet(inputDelays,feedbackDelays,hiddenLayerSize);
  % --- SNIP 

    net.divideFcn = 'divideblock';  % Divide data randomly

%----- SNIP Used all defaults except divideblock  

    % Closed Loop Network
    % Use this network to do multi-step prediction.
    % The function CLOSELOOP replaces the feedback input with a direct
    % connection from the outout layer.
    netc = closeloop(net);
    netc.name = [net.name ' - Closed Loop'];
    %view(netc)
    [xc,xic,aic,tc] = preparets(netc,inputSeries,{},targetSeries);
    yc = netc(xc,xic,aic);
    closedLoopPerformance = perform(netc,tc,yc)

    % Here I added the prediction without knowing the targets, but my ycp gives NaN. Don't understand why.

    [xcp,xicp,aicp] = preparets(netc,predinSeries,{});
    ycp = netc(xcp,xicp,aicp);
    plot(cell2mat(yc),'g')
    hold on
    plot(cell2mat(ycp),'+g')
    hold on

[Greg Heath]

It is preferable, for helpers, to use MATLAB nndata rather than your data when demonstrating problems and solutions.

 help nndatasets
 doc nndatasets

Why?

Because we are familiar with the nndatasets. Thus making it easier for us to concentrate on your code.

I'll take a look at your code, perhaps using a nndataset.

Greg

PS I don't see why this is so hard to understand:

 1. Convert an OL design to a CL design and test the latter on        the data used to obtain the OL design
 2. If the CL design performance is unsatisfactory, use train to 
improve its performance on that data.
 3.How can you possibly hope to obtain good CL performance on new data when it does not perform well on old data?
 4. If you are wondering why not just design the CL net from scratch, you can but it is much more difficult and takes much longer.

[Pedro]

Thank you again. I understand what you mean with the 4 points. I will use one of the datasets and see if I can recreate my problem.

PS: One of my problems is this:

Lets suppose that I trained the network and got a good performance from my CL on design data. Now if I want to use this CL to predict the values for the future, how do I write it, I mean, if I dont have targets for my prediction how do I write the code below? I tried removing the predtargetSeries but the ycp returns NaN...

[xcp,xicp,aicp,tcp] = preparets(netc,predinputSeries,{},predtargetSeries);
ycp = netc(xcp,xicp,aicp);

[Alhaji Grema]

Hello,

I've been following this conversation with a keen interest because I've the same problem as Pedro, but I've not got the solution. Please Greg can you look at Pedro's last question, I quote 'Lets suppose that I trained the network and got a good performance from my CL on design data. Now if I want to use this CL to predict the values for the future, how do I write it, I mean, if I dont have targets for my prediction how do I write the code below? I tried removing the predtargetSeries but the ycp returns NaN...

[xcp,xicp,aicp,tcp] = preparets(netc,predinputSeries,{},predtargetSeries); ycp = netc(xcp,xicp,aicp);'

I will be waiting for your response. Thanks

[Greg Heath]

When you train the CL design the command is

 [ netc trc Ycs Ecs Xcf Acf ] = train(netc,Xcs,Tcs,Xci,Aci);

where

 Ycs = netc(Xcs,Xci,Aci);
 Ecs = gsubtract(Tcs,Ycs);

Therefore, if, Xcsnew, the continuation of Xcs, is avavailable, future outputs can be predicted via

 Ycsnew = netc(Xcsnew, Xcsf, Acsf);

However, Tcsnew is not, in general, known. Therefore Ecsnew and the corresponding performane estimate cannot be estimated.

[Dan]

 Xcsf, Acsf = Xcf, Acf ? 
what do s and f stand for ( start, finish?)?

Answer 3

[Dan]

excellent explanation : http://www.mathworks.com/matlabcentral/answers/14970-neural-network-multi-step-ahead-prediction