MATLAB Answers

working with large matrices

47 views (last 30 days)
Bram Stegeman
Bram Stegeman on 6 Mar 2019
Commented: Stephan Koschel on 17 Mar 2020
I have to work with large matrices (e.g. A = 8000 x 100.000, all non-zero values).
I want to calculate T= ((A'*A) + lamda*speye(n))\(A'*A); with lamda =e.g. 1-e-3.
I have installed 64 gb ddr and as expected I run out of memory (also when I introduce a threshold and force A to be sparse, and then create sparse(A) and sparse (A') and try to calculate T.
Are there alternative ways to calculate T without out of memory issues?
Stephan Koschel
Stephan Koschel on 17 Mar 2020
You are only interested in the diagonals of a matrix multiplication?
I would implement an iteration over the diagonal elements and load the corresponding columns and rows from the two matrices. The entry on the diagonal becomes something like sum(current_row .* current_col)
The iteration could slow down the process, but you only need to load two vectors into memory.

Sign in to comment.

Answers (1)

Munish Raj
Munish Raj on 18 Mar 2019
Hello Bram,
You could look at implementing Tall Arrays.
The documentation link can be found here.


Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by