Matrix multiplication using multicore

how can i multiply a matrix of dimension (2^18*103) by its transpose using multiple cores of my local system ?

[Jan]

How much RAM do you have? The output has (2^18)^2 elements and needs 549 GB of RAM. If you want to do any calculations with this array, a 1 TB RAM machine is a minimal setup I would start to work with.

Taking into account that the output is symmetric, computing this matrix explicitly might waste half of the memory.

[ANUSAI R]

Thank you for replying.I am currently having a system with 128gbRAM with 18core and 36 threads.I can divide the matrix into 8 parts so that each part can be computed separately.still,it’s taking too much time as it uses only one core.So,I would like to know how to do parallel processing so that all 18cores can be activated simultaneously..Thankyou once again

[Jan]

A 128 GB machine is simply too weak to process a matrix multiplication, which produces a 549 GB output. The bottleneck is the disk access, if you use a huge amount of virtual RAM. The number of cores will not really matter, because the calculations in the CPU will take less time (even with a single thread) than writig the data to the disk.

Answer 1

[James Tursa]

The matrix multiply operator * calls a highy optimized compiled BLAS library to do this calculation. The BLAS library is already multi-threaded and optimized for cache hits. In addition, I believe MATLAB checks for symmetric cases (both operands the same) and calls faster symmetric library routines for this ... at least it used to last time I looked several years ago. So I am not sure there is anything you can do that isn't already being done in the background. The only caveat is the symmetric case ... if MATLAB doesn't do this on your version then it may require a custom mex routine on your part to force this. My MTIMESX submission does this symmetry check, but alas I have not had the time yet to update it for later versions of MATLAB.

See also this link:

https://www.mathworks.com/matlabcentral/answers/319587-is-it-possible-to-declare-the-matrix-is-symmetric-in-advance-before-multiplication?s_tid=answers_rc1-2_p2_MLT

[ANUSAI R]

Thank you for your kind reply, I will try this and update.