Correlation coefficent for vectors (of unequal lengths).

Hi,

I have two separate vectors (A and B) which I want to caculate '[r,p] = corrcoef(A,B)' for. They are the same length but both contain 'NaN' so I can't use corrcoef. Both vectors follow a time series so I can't remove the nans separately in each vector as the rows of data will then be out of sequence.

Is there a way of removing an entire row in both A and B if one of them contains a nan on a particular row so that they remain in synch with each other?

If not is there another method I can use?

Thanks

Answer 1

[Daniel Shub]

First make some dummy data

A = 1:10;
A(2) = nan;
B = 1:10;
B(4)=nan;

Then combine the arrays into a matrix so it is easy to work with

C = [A; B];

Then only keep columns for which not any of the rows are nan

D = C(:, ~any(isnan(C)))

Finally split the matrix into two new arrays

 Aprime = D(1, :);
 Bprime = D(2, :);

[Anna]

Brilliant! Thanks Daniel!

Answer 2

[Narges M]

use isnan() function to skip those rows

[Anna]

Narges M, thanks for your reply. Do I need to create a loop with isnan? Can you explain please? :)

[Narges M]

this will give you the index of the rows containing only numbers: idx = find(~isnan(A)+ ~isnan(B)==2);

[Anna]

I tried using the index to for both vectors together but I get the error; 'Error using + Matrix dimensions must agree'.

I tried finding idx for A and B separately and that works (without '==2'- that didn't work for me I'm afraid) but now I don't know if the vector rows are still in synch. The vectors now are of unequal lengths

[Narges M]

the ~isnan(A) function will give you 1s for numbers and 0s for nan elements. by adding the two together (assuming your A and B matrix are exactly the same length), you will get 2s on those rows containing numbers in both A and B. using find() function you would have that index in the number form.

so for example:

A = [ 1     4     7   NaN     8   NaN     3];
B = [NaN    6     4    5      1   NaN     9];

then

idx = find(~isnan(A)+ ~isnan(B)==2)

will give you:

idx = [2     3     5     7];

which is correct. now you can use

corrcoef(A(idx),B(idx)).

[Anna]

Oh I see that clearly now! Thanks!