Can I obtain the covariance matrix of a stochastic process with plenty of measurements?
3 views (last 30 days)
I have implemented the Karhunen-Loève expansion as per this question.
I have tested it with samples of data obtaining possitive results.
However, now I am dealing with a matrix of dimensions 211302*50, meaning that I have roughly 200000 observations of 50 random variables.
If I try to calculate the covariance matrix of these using the observations as columns (as per the previus link), the program crashes and the error returned is:
Error using *
Requested 211302x211302 (332.7GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take
a long time and cause MATLAB to become unresponsive.
Error in cov (line 155)
c = (xc' * xc) ./ denom;
Is there a way to do what I want or is the matrix just too big?
Bjorn Gustavsson on 25 Jun 2021
To me it seems that you have misunderstood the dimensions. Your covariance-matrix should, to my understanding be 50x50 when you have 50 random variables observed 200000 times. With a call like this:
tic,C = cov(randn(211302,50));toc
I get a 50-by-50 covariance-matrix C in ~0.22 s.