Reshaping is causing memory problem

Hello everyone,

I have 500 matrices of huge size of 40834 x 40834 and I would like to load them from mat files and then flatten them using the reshape function and later do some other things. However in the first flatteing operation I am getting a memory issue and Matlab aborts. How one could solve such a problem ? The code is below, here all Ki.mat are sparse matrices. Thank you.

i=1;

K_matrix_file = strcat('Matrices/K',num2str(i),'.mat');

load(K_matrix_file);

K_flatten = reshape(K_i,1,[]);

Saddam

[Catalytic]

We can't see what you did. We can't see what errors you got. We are not sitting beside you at your computer.

[Matt J]

Are the matrices in sparse format? If they are in non-sparse format and they loaded sucessfully, there's no reason the reshape operation should have breached memory limits, unless the matrix consumes most of your RAM.

[Saddam N Y]

Sorry I updated the question to include the piece of code that produces the issue. The matrices are all in the sparse format.

Answer 1

[Matt J]

Don't make long sparse row vectors, as these do consume lots of memory. Work with column vectors instead:

K_flatten = K_i(:);

[Walter Roberson]

t0 = spalloc(1,0,2);
t1 = spalloc(1,100,2);
t2 = spalloc(1,101,2);
whos t0 t1 t2

  Name      Size             Bytes  Class     Attributes

  t0        1x0                 40  double    sparse    
  t1        1x100              840  double    sparse    
  t2        1x101              848  double    sparse    

t1(1,100) = 1;
t2(1,100) = 1;
whos t1 t2

  Name      Size             Bytes  Class     Attributes

  t1        1x100              840  double    sparse    
  t2        1x101              848  double    sparse    

t1(1,50) = 2;
t2(1,50) = 2;
whos t1 t2

  Name      Size             Bytes  Class     Attributes

  t1        1x100              840  double    sparse    
  t2        1x101              848  double    sparse    

In short, a sparse row vector takes 8 bytes plus 8 bytes per column plus 16 bytes per non-zero.

This is a more than the equivalent non-sparse row vector, which would take 8 bytes per column.

[Saddam N Y]

Thank you very much for your answer !!