How to calculate gradient of output of a neural network with respect to its parameters?
32 views (last 30 days)
I am new to the Deep Learning toolbox. I am working on a Reinforcement Learning problem wherein I need to calculate the derivative of the output of NN with respect to parameters.
More specifically, let the I/O relation of the neural network be defined as , where x is the input, y is the output, and θ contains the weights and biases of the neural network. For a specific input , I am interested in calculating . Any idea how I should go about this with the deep learning toolbox?
Jon Cherrie on 27 Apr 2021
You can use dlgradient and dlfeval to take derivatives with respect to the input or to the parameters
Here's an example:
I'm going to let this function play the role of the neural network. You should use dlnetwork to create a more complex network.
function y = f(x,theta)
y = sum(sin(theta(1)*x+theta(2)));
Then I need to scope the computation of the function so that dlfeval knows where to apply auto-diff. I do that by defining a function that evaluates the network and computes the gradient of interest. In this case:
function [y, dy] = fun_and_deriv(x,theta)
y = f(x,theta);
dy = dlgradient(y,theta);
Then I can compute the derivative:
x = dlarray(0:10);
theta = dlarray([1 2]);
[y, dy] = dlfeval(@fun_and_deriv,x,theta)