I've been reading Neural_Network_Toolbox_Users_Guide and I have a question about below section.
As what it said in 3-19,
In fact, the gradients and Jacobians for any network that has differentiable transfer functions, weight functions and net input functions can be computed using the Neural Network Toolbox software through a backpropagation process. You can even create your own custom networks and then train them using any of the training functions in the table above. The gradients and Jacobians will be automatically computed for you.
But I don't understand how exactly this works.
Here is what I'm trying to demonstrate.
- We use neural networks as a non-linear function approximator to estimate the real function.
- We have inputs(x) and refer the real function(y) with weights that we try to estimate using neural networks.
- A neural network can be trained by minimizing a loss function that changes at each iteration.
- Loss function can be the average of square of (y'-y) , where y' is the target function. Briefly speaking, the gradient of loss function can be represented with (y'-y)▽y.
- Here, note that the target function(y') depends on weights this is different from general learning (e.x. supervised learning), which are fixed before training, so that the target function is changed at each iteration.
- Therefore, what I'm looking for is how to calculate ▽y at each iteration to update weights.
Could you help me to understand and give an example about this?