To optimize a neural network of multiple inputs using a genetic algorithm.

I have found the answer from the matlab team but the code is applied for a single input. I have tried to modify for 4 inputs but it was not success. Could you please give me the code to modify the below code for the 4 inputs?

The GA function requires a function handle as an input argument to which it passes a 1xN vector, where N is the number of variables in the system to be optimized.

For a neural network, the weights and biases are a Mx1 vector. These may be optimized using GA.

A function can be written to accept the network, weights and biases, inputs and targets. This function may return the mean squared error based on the outputs and the targets as GA requires a function handle that only returns a scalar value.

The following code example describes a function that returns the mean squared error for a given input of weights and biases, a network, its inputs and targets.

function mse_calc = mse_test(x, net, inputs, targets)

% 'x' contains the weights and biases vector

% in row vector form as passed to it by the

% genetic algorithm. This must be transposed

% when being set as the weights and biases

% vector for the network.

% To set the weights and biases vector to the

% one given as input

net = setwb(net, x');

% To evaluate the ouputs based on the given

% weights and biases vector

y = net(inputs);

% Calculating the mean squared error

mse_calc = sum((y-targets).^2)/length(y);

end

The following code example describes a script that sets up a basic Neural Network problem and the definition of a function handle to be passed to GA. It uses the above function to calculate the Mean Squared Error.

% INITIALIZE THE NEURAL NETWORK PROBLEM %

% inputs for the neural net

inputs = (1:10);

% targets for the neural net

targets = cos(inputs.^2);

% number of neurons

n = 2;

% create a neural network

net = feedforwardnet(n);

% configure the neural network for this dataset

net = configure(net, inputs, targets);

% create handle to the MSE_TEST function, that

% calculates MSE

h = @(x) mse_test(x, net, inputs, targets);

% Setting the Genetic Algorithms tolerance for

% minimum change in fitness function before

% terminating algorithm to 1e-8 and displaying

% each iteration's results.

ga_opts = gaoptimset('TolFun', 1e-8,'display','iter');

% PLEASE NOTE: For a feed-forward network

% with n neurons, 3n+1 quantities are required

% in the weights and biases column vector.

%

% a. n for the input weights

% b. n for the input biases

% c. n for the output weights

% d. 1 for the output bias

% running the genetic algorithm with desired options

[x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts);

[Greg Heath]

show YOUR code and error messages

[Matt Talebi]

Hi Greg, two things regarding the use of these codes: first, unless adding x = getwb(net) to the fitness function it returns error (Undefined function or variable 'x'). Following this modification and by running the [x_ga_opt, err_ga] = ga(h, 3*n+1, ga_opts), it shows no improvement in minimizing the MSE. The MSE that I get is equal to the one that I obtain for the trained network without using GA. I appreciate your comment on this. Cheers/Matt.

Answer 1

[Greg Heath]

function NMSE_calc = NMSE( wb, net, input, target)

% wb is the weights and biases row vector obtained from the genetic algorithm.

% It must be transposed when transferring the weights and biases to the network net.

 net = setwb(net, wb');

% The net output matrix is given by net(input). The corresponding error matrix is given by

 error = target - net(input);

% The mean squared error normalized by the mean target variance is

 NMSE_calc = mean(error.^2)/mean(var(target',1));

% It is independent of the scale of the target components and related to the Rsquare statistic via

% Rsquare = 1 - NMSEcalc ( see Wikipedia)

[Chuthamat]

Hi Greg, My problem is not about the objective function. It is nothing wrong about the function. The code that I gave in my question is okay, but I want to modify them for 4 inputs. For example if I need 4 inputs and 1 target, as follow Input1=(1:10) Input2=(21:30) Input3=(41:50) Input4=(61:70)

target=(31:40)

What are the command codes needed to modify? I only know that the code for nvars in GA need to be changed for 4inputs. With n neurons, 6n+1 quantities are required in the weights and biases column vector.

[Greg Heath]

Search NEWSGROUP and ANSWERS using

 greg Nw

[Chuthamat]

I have no idea what's your advise. Could you please give me the link?

[Greg Heath]

My advice is to search in either the NEWSGROUP or answers using the search words

 greg Nw

You can probably just search on Nw, the number of unknown weights to estimate.

For I-dimensional inputs, O-dimensional outputs and H hidden nodes

 Nw = (I+1)*H +(H+1)*O

where the "1"s indicate bias connections.

Answer 2

[anurag kulshrestha]

I applied the above code for a single input with 3 hidden neurons and a single output in Matlab 7.10.1(R 2010a). But mse is not getting minimized. Code is given below with few change in function

net = newff(inputs,targets,n); where n=3

x= getx(net);

y=sim(net,inputs); where y is the output

e= targets-y; where e is error

h = @(x)mse(e,y,x);

ga_opts = gaoptimset('TolFcn', 1e-8,'display','iter');

[x_ga_opt, err_ga] = ga(h,3*n+1,ga_opts);

[Greg Heath]

h = @(x)mse(e,y,x);

is an error. Check the documentation

 help mse
 doc mse

Hope this helps.

Greg

Answer 3

[Alexandra Sikinioti-Lock]

I had the same problem it seems to have been resolved with the formwb. I altered the function like this.

function mse_calc = mse_test(wb, net, inputs, targets, inindelst, inlaydelst)
% 'x' contains the weights and biases vector 
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
% To set the weights and biases vector to the
% one given as input
wb = formwb(net,net.b,net.IW,net.LW);
net = setwb(net, wb');
% To evaluate the ouputs based on the given
% weights and biases vector
%y = net(inputs);
y = net(inputs,inindelst,inlaydelst);
target=cell2mat(targets);
target1=target(3:254208);
% Calculating the mean squared error
mse_calc = sum((cell2mat(y)-target1).^2)/length(y);
end

[Niall O' Mahony]

What are variables inindelst and inlaydelst?

[Greg Heath]

GUESS:

 input inputdelay
 input layerdelay

st = state???

Greg

[Greg Heath]

WHAT WAS YOUR FINAL RESULT?

 mse(target-output)/mean(var(target',1)) = ?

Answer 4

[MD NOUSHAD JAVED]

Respected Sir, Want to implement GA tool for multi-objective optimization.

I have three different objectives Y1 , Y2 and Y3. All three are polynomial equations of independent variables x1, x2 and x3.

The problem is that I am finding a solution to maximized the Y1 while minimize Y3 , while Y2 are of moderate importance (no such preference).

I would be rally helpful of you if you help me to do same functions in matlab. How will I write script code and define such importance to each variables ?

I will be thankful of you.

[Greg Heath]

I would try maximizing something like

 f = Y1 + 0.5*Y2 - Y3

Hope this helps.

Greg

Answer 5

[Ahmed Ryad]

You can change the number of variables in the ga function from (3*n+1) to the number of the weight and bias of your net where the number of weight and bias is (no) wb=getwb(net); no=length(wb);