eqn =
k = [1750 -750 0; -750 1250 -500; 0 -500 500];
m = [75 0 0; 0 75 0; 0 0 50];
D=inv(m)*k;
syms x
I = [x,0,0;0,x,0;0,0,x];
Lambda = D-I;
DET = det(Lambda);
eqn = DET==0
At this point, you have generated a cubic polynomial. It is a symbolic polynomial in x. That would be a good start. You needed to make only one more step.
But, what do you think root does? (Nothing. There is no function named root.) You can use solve.
xsol = solve(eqn,'maxdegree',3)
And that looks pretty messy, but the fact is, the roots of a cubic polynomial are a bit messy for a completely general polynomial.
vpa(xsol)
So there are three real roots. They look like they are complex roots, because they have an imaginary part, but it is an infinitessimal one. Just discard that part.
real(vpa(xsol))
Those are the three roots. Are they correct?
eig(D)
Indeed, what you did was valid. At least until that very last line.