Reverse Engineering traingd in ANN?

I am sort of trying to reverse engineer traingd algorithm. dWB = param.lr * gWB is the equation shown in traingd.m, would like to understand how gWB is calculated. Any reference/example/sourcecode/backpaper in this regard is much appreciated. Thanks in advance!

Answer 1

[Greg Heath]

type traingd

 % Initialize
  startTime = clock;
  original_net = net;
  [perf,vperf,tperf,gWB,gradient] = nntraining.perfs_grad(net,data,fcns);
 ...
 ...
 % Gradient Descent
    dWB = param.lr * gWB;
    WB = WB + dWB;
    net = setwb(net,WB);
    [perf,vperf,tperf,gWB,gradient] = nntraining.perfs_grad(net,data,fcns);

>> help nntraining

   Neural Network Toolbox Utility Functions

This directory contains utility functions.  It is recommended these functions not be used as they may be removed or altered in future versions of this software.

So, search for the directory containing utility functions

Hope this help.

Thank you for formally accepting my answer

Greg

[Kannan]

Had a hard time going through utility functions, in vain. Anyway, Thanks a lot. By the way, have you ever tried it yourself?

[Greg Heath]

No.

[Kannan]

Hurray! Successfully calculated 'change in weight/bias' for the output layer. Moving on to hidden layer now.

[Greg Heath]

Good Luck