We can start with your input
a = int64(1:3)';
out = num2cell(a);
b = int64(1:4)';
We will presume you're looking for something that tells you that "the contents of indexes 1, 2, and 3 match between the two cells", so you're looking for index to return the values 1, 2, and 3.
You have a 3-length and a 4-length cell array. By definition the 4th cell element can't match anything, so we only need to consider up to the smallest array length.
minLen = min(length(out),length(out2));
We can then use cellfun() to compare each pair of elements:
index = find(cellfun(@isequal, out(1:minLen), out2(1:minLen)))
Note that you didn't necessarily need to put integers into cells to do this. Perhaps your real data contains strings or other non-numeric contents, but if you're working with arrays originally you can do the same thing with the raw array contents:
index = find(arrayfun(@isequal, a(1:minLen), b(1:minLen)))
Did this help you out?