bad results of my neural network _ newsgroup

Hi Dr Greg i tried to reply to your message in the NEWSGROUP in this link several times https://www.mathworks.com/matlabcentral/newsreader/view_thread/348826 but but it dosen't post , i don't know why so i decide to post it here as a question because i couldn't have any solution .

This is the message that i wanted to post :

I appology for that DR Greg this will not be repeated

> x = patientInputs;
> t = patientTargets;
> inputs = mapstd(x); 
> targets = mapstd(t); 
>> I don't remember suggesting mapstd. Data is automatically normalized 
and unnormalized by the training function. The only reason to use mapstd 
is to detect and delete or modify outliers.

I asked you because of the format of my inputs as you can see in this link https://www.mathworks.com/matlabcentral/answers/344813-how-to-do-when-the-inputs-ranges-for-neural-network-are-not-so-uniform-in-magnitude Actually I tried to use MAPSTD but it dosen't improve so i applied the logarithme to the inputs that have a very high value numerically compared to other inputs but also it doesn't work .

> N=981
>>Why does this differ from the 1012 of previous posts?

It was at first 981 but i wanted to increase the number of N to see if this will improve the peformance and as i see that there is no big progress i returned to 981

> Ntrn = N-2*round(0.15*N) % 
> Ntrneq = Ntrn*O % 

>>If you are not going to use an ending semicolon, then print the answer
after the percent sign. For example
[ I N ] = size(x) % [ 9 981]
[ O N ] = size(t) % [ 2 981 ]
Ntrn = N-2*round(0.15*N) % 687
Ntrneq = Ntrn*O % 1374

> %For a robust design desire Ntrneq >> Nw or
> H=10
or H = 10 ???
>>That doesn't make any sense

This was just a typing error

 > Hub = -1+ceil( (Ntrneq-O) / (I+O+1)) % Hub =117

>>117?? Your arithmetic sucks.

> Ntrials = 10
> rng(0)
> j=0
> for h =round([Hub])

>>Where in the world did you get that from? Hub = 114 !

> j = j+1
> h = h %12

>> h = 114! Where did you get 12 from??

Actually i got this value after running the code and it was from the given results

> trueclasses = vec2ind(t);
> for i = 1:Ntrials
> net = configure(net,x,t);
> [ net tr outputs errors ] = train(net,x,t);
> assignedclasses = vec2ind(outputs);
> classerr = assignedclasses~=trueclasses;
> Nerr(i,j) = sum(classerr);
> % FrErr = Fraction of Errors (Nerr/N) 
> [FrErr(i,j),CM,IND,ROC] = confusion(t,outputs);
> FN(i,j) = mean(ROC(:,1)); % Fraction of False Negatives
> TN(i,j) = mean(ROC(:,2)) ; % Fraction of True Negatives
> TP(i,j) = mean(ROC(:,3)); % Fraction of True Positives
> NMSE = mse(errors)/mean(var(t',1)) 
> end
> end
> PctErr=100*Nerr/N
> NMSE = mse(errors)/mean(var(t',1)) 

>>This makes no sense: You have Ntrials designs but only take 
the last one instead of the best one

the original code that i got is from you answer in this link https://www.mathworks.com/matlabcentral/answers/130271-how-to-improve-accuracy-for-unseen-data

% NEURAL NETWORK MODEL

but i tried to add NMSE = mse(errors)/mean(var(t',1)) and it was a fault from me to put it after the loop.

My real problem is how can i use this result to improve the performance of my neural network So as i understand that for Not overfitting Ntrneq >= Nw so i can have idea about the limit of H because Nw = 12*H+2 and i should decrease H to have NMSE <= 0.01 * MSE00 . Ami right ?

Again i apology for this bad Etiquette but i hope that you understand me because i have no idea about neural network and Matlab and i have no time for my project this is why .

Thank you Dr greg for you patience .

Answer 1

[Greg Heath]

1. You did not define net = patternnet

2. Even though it is a classifier, just try to minimize H subject to the constraint

    mse(e) <= 0.01*mean(var(t',1))  % e=t-y

i.e.

     NMSE <= 0.01
or

     Rsquare >= 0.99

Hope this helps

Thank you for formally accepting my answer

Greg

3. If you overfit, validation stopping will kick in

Good Luck

Thank you for formally accepting my answer

Greg

[afef]

ok i will try it thank you

[afef]

i want to know your advice for the application of the logarithme to to the inputs that have a very high value numerically compared to other inputs. Can you tell me if this help for the improvement of the performance or not ?

[afef]

This is my new code :

 x = patientInputs;
t = patientTargets;
inputs = mapstd(x); 
 N=981
 I=9
 O=2
[ I N ] = size(x) 
 [ O N ] = size(t) 

 Ntrn   = N-2*round(0.15*N) %687 
 Ntrneq = Ntrn*O  % 1374         
    %For a robust design desire Ntrneq >> Nw 

   Hub =( (Ntrneq-O) / (I+O+1)) %   Hub =1372/12 = 114.3

 Ntrials = 10
  rng(0)
 j=0
 for H =round([Hub/10, Hub/2, Hub])
    j  = j+1
    H  = H 
    net           = patternnet(H);

 trueclasses = vec2ind(t);

    for i = 1:Ntrials
        net                          = configure(net,inputs,t);
        [ net ,tr  ] = train(net,inputs,t);
        outputs = net(inputs);
        errors  = gsubtract(t,outputs);
        assignedclasses              = vec2ind(outputs);
        classerr                     = assignedclasses~=trueclasses;
        Nerr(i,j)                    = sum(classerr);
        % FrErr                      = Fraction of Errors (Nerr/N) 
        [FrErr(i,j),CM,IND,ROC]  = confusion(t,outputs);
        FN(i,j) = mean(ROC(:,1));    % Fraction of False Negatives
        TN(i,j) = mean(ROC(:,2)) ;   % Fraction of True Negatives
        TP(i,j) = mean(ROC(:,3));    % Fraction of True Positives
         MSE00 = mean(var(t',1)) 
         NMSE = mse(t-outputs)/MSE00
    end
 end
 PctErr=100*Nerr/N

for H =11 i get

MSE00 =   0.2498
NMSE =    0.9999
MSE00 =    0.2498
NMSE = 0.9774
MSE00 =    0.2498
NMSE =    1.0284
MSE00 =  0.2498
NMSE =    0.9877
MSE00 =  0.2498
NMSE =    0.8624
MSE00 =0.2498
NMSE =   1.0305
MSE00 =    0.2498
NMSE =    0.8812
MSE00 =  0.2498
NMSE =    1.0358
MSE00 =    0.2498
NMSE =    0.9341
MSE00 =  0.2498
NMSE =   1.0011

For H = 57 i get

MSE00 =   0.2498
NMSE =  0.8302
MSE00 =    0.2498
NMSE =  0.9538
MSE00 =    0.2498
NMSE =    0.8976
MSE00 =    0.2498
NMSE =  0.8615
MSE00 =  0.2498
NMSE = 0.8547
MSE00 =   0.2498
NMSE =    0.9835
MSE00 =  0.2498
NMSE =    0.8925
MSE00 =    0.2498
NMSE =    0.9090
MSE00 =   0.2498
NMSE =0.9100
MSE00 =  0.2498
NMSE =   1.0485

And for H =114 i get

 MSE00 =  0.2498
NMSE =   0.9801
MSE00 =   0.2498
NMSE =    0.8758
MSE00 =    0.2498
NMSE =    0.8749
MSE00 =  0.2498
NMSE =  0.9591
MSE00 =0.2498
NMSE =  0.9476
MSE00 = 0.2498
NMSE =   0.8334
MSE00 = 0.2498
NMSE =    0.8863
MSE00 =   0.2498
NMSE = 1.0534
MSE00 =   0.2498
NMSE =   0.8665
MSE00 = 0.2498
NMSE =    0.8977

As you can see i couldn't get the constraint NMSE <= 0.01 all the values are > 0.01.What should i do please?