The only keytype which can hold two doubles is the character key type. So one may use char as key type and then simply
function k = getKey(obj)
    k = num2str([obj.a obj.b],'%d');
end
An even faster approach could be 
function k = getKey(obj)
    k = char(typecast([obj.a obj.b],'uint16'));
end
As every char in MATLAB is 16bit. This may produce non-printable keys though.
