Numerical answer to Partial derivative of Hopfield-style Energy in symbolic expression

Hello,

I'm implementing a gradient descent optimizer that uses Hopfield-style energy as function,

the hopfield-style energy is a function of p:

The hopfield energy equation can also be expressed as:

The gradient vector is defined as:

I'm attempting to calculate the gradient vector using the Symbolic Toolbox as follows:

r_mat = sym('r',[4,4]);

r_mat = r_mat(:);

p_mat = sym('p',[4*4,1]);

c_mat = sym('c',[4*4,1]);

Energy = -0.5*p_mat'*r_mat*p_mat' - c_mat'*p_mat;

%the first partial derivative:

q1 = -1*diff(testi1,p_mat(1,1));

which yields as the first term:

conj(c1) + r1_1*conj(p1) + (r1_2*conj(p5))/2 + (r1_3*conj(p9))/2 + (r2_1*conj(p2))/2 + (r1_4*conj(p13))/2 + (r2_2*conj(p6))/2 + (r2_3*conj(p10))/2 + (r3_1*conj(p3))/2 + (r2_4*conj(p14))/2 + (r3_2*conj(p7))/2 + (r3_3*conj(p11))/2 + (r4_1*conj(p4))/2 + (r3_4*conj(p15))/2 + (r4_2*conj(p8))/2 + (r4_3*conj(p12))/2 + (r4_4*conj(p16))/2

How can I replace the symbols with numbers to yield the q vector? I need the q-vector for the gradient descent

the algorithm I'm implementing: https://doi.org/10.1117/1.JEI.23.1.013007 (relevant part starts at Equation 5)

Answer 1

[Torsten]

p is (4x1), not (16x1).

And keep r_mat as matrix, then

E = -0.5*p.'*r_mat*p - c.'*p

( r_mat can be assumed symmetric ).

By the way:

dE/dp = -r_mat*p - c

assuming r_mat is symmetric.

To replace symbols by numerical values, use "subs".

[Pseudoscientist]

Hello,

I redefined: p_mat = sym('p',[4,1],'real');

I get error:

>> c_mat.'*p_mat

Error using symengine

Dimensions do not match.

edit: solved by redefining c_mat = sym('c',[4,1],'real');

[Pseudoscientist]

also how do I replace p1,p2,p3,..pn with numerical vector of length n?

edit: subs(E,[p_mat',g_mat'],[p_num,g_num])

[Torsten]

So you could answer your questions on your own, I guess ?

[Pseudoscientist]

It turned out symbolic expressions was extremely slow, I used symbolic to calculate the partial derivative and then used that to think how to write the equation in vector form and was evetually succesful and the optimizer is now very fast !

[Torsten]

I used symbolic to calculate the partial derivative and then used that to think how to write the equation in vector form

That's the best you can do, I guess.