MATLAB Answers

Correlation of two variables over time: can this happen?

84 views (last 30 days)
alpedhuez
alpedhuez on 17 Dec 2020
Edited: alpedhuez on 17 Dec 2020
I have a variable x1 and x2 between January 1 till December 31. When I calculate the correlation between Janaury and June it is positive. When I calculate the correlation between July to December it is positive. But the correlation between Janauary and December is negative. Can it happen?

  1 Comment

Matt Gaidica
Matt Gaidica on 17 Dec 2020
Yes. However, you probably want to perform a statistical test to determine if the fluctuating correlation is significant.

Sign in to comment.

Accepted Answer

the cyclist
the cyclist on 17 Dec 2020
Yes. This is known as Simpson's Paradox. Here is an example:
rng default
x1 = [1 2 3 4 6 7 8 9]';
x2 = [1 2 3 4 -6 -5 -4 -3]' + 0.8*rand(8,1);
% Correlation of first half
corrcoef(x1(1:4),x2(1:4))
% Correlation of second half
corrcoef(x1(5:8),x2(5:8))
% Correlation of entire vector
corrcoef(x1,x2)
% Plot it
figure
scatter(x1,x2)
You can see that the first half and the second half are positively correlated with each other, but if you look at the trend over the entire vector, it is negative.

  7 Comments

Show 4 older comments
the cyclist
the cyclist on 17 Dec 2020
To be clear, my comment was directed more at the OP than as a reply to your comment, @IveIve
alpedhuez
alpedhuez on 17 Dec 2020
Yes I now understand confounding factor t.
x1 and x2 has a postive correlation between date 1 and date 2.
x1 and x2 has a positive correltion between date 2 and date 3.
x1 and x2 has a negative correlation betwen date 1 and date 3.
Regression of x2 on x1 between date 1 and date 2 has positive statistically sig coef.
Regression of x2 on x1 between date 2 and date 3 has positive statistically sig coef.
Regression of x2 on x1 between date 1 and date 3 has negative statistically sig coef.
Regression of x2 on x1 and time between date 1 and date 2 has positive statistically sig coef for both x1 and time.
Regression of x2 on x1 and time between date 2 and date 3 has positive statistically sig coef for both x1 and time.
Regression of x2 on x1 and time between date 1 and date 3 has positive statistically sig coef for both x1 and time.
How can it happen?

Sign in to comment.

More Answers (1)

Matt Gaidica
Matt Gaidica on 17 Dec 2020
Edited: Matt Gaidica on 17 Dec 2020
Sure, it can happen. What's your concern about it?

  4 Comments

Show 1 older comment
alpedhuez
alpedhuez on 17 Dec 2020
Thank you. Then, what would be the ELI5 (Explain Like I am Five) explanation?
Image Analyst
Image Analyst on 17 Dec 2020
For a thorough explanation see the Wikipedia link in the Cyclist's answer below. (He beat me to it, posting this fun paradox.)
It would be interesting for us to know what real world measurements x1 and x2 are so we can see a real world situation that gives rise to this parabox. What do x1 and x2 represent?
alpedhuez
alpedhuez on 17 Dec 2020
I am happy to explain to get comments. But I am not sure whether I want to discuss the problem in a public bulletin board. So if there is a suggestion for a venue I will be happy to listen.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by