memory pca vs pcacov
5 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Dear all,
I'm running a pca on a large matrix (33*500,000) and with pcacov I get a memory error, but pca gives me no trouble. Could anyone explain this to me? Is the matrix somehow being reduced before computing the covariance matrix in pca?
Thanks!
Best
Hans
0 Kommentare
Antworten (3)
Hiro Yoshino
am 2 Nov. 2020
In PCA, your matrix (p x q) will be once converted into the variance-covariance matrix (q x q).
This would reqiure huge memory comsumption. Meanwhile, pcacov accepts a variance-covariance matrix as an input and, therefore the argument (input) should be a square matrix though.
As for big data anaysis, you may want to use tall array - this can be a solution.
0 Kommentare
Hans van der Horn
am 2 Nov. 2020
2 Kommentare
Hiro Yoshino
am 2 Nov. 2020
I do not believe pcacov works with your matrix in the first place since the shape of your matrix is unacceptable. I do not know what the error will be like.
Hans van der Horn
am 2 Nov. 2020
2 Kommentare
Hiro Yoshino
am 2 Nov. 2020
I got your point now!!
OK, actually to avoid memory problem, pca normally takes a different approach to calculate eigen vectors - SVD. This is not a direct method and produces some by-product. This is a well-known fact - you may find the this in your text book too, I'm sure.
Siehe auch
Kategorien
Mehr zu Dimensionality Reduction and Feature Extraction finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!