I need to remove linearly dependent rows from large sparse matrices (> 250,000 rows and columns). The recommended solutions online call for using "chol", "qr", or "ldl" for this purpose (rref is also supposed to work, but some have advised against it).
- I have tried these approaches on a linux machine running R2015a with 64GB RAM. Unfortunately, I get an "out of memory" error when calling chol, qr, or ldl. rref does not produce such an error, but it is prohibitively slow for the problems I am considering.
- I have also tried these approaches on a mac running R2015b with 8GB RAM. No functions give out-of-memory errors, but the mac OS does allocate 10GB of virtual memory (even though the mac has an SSD, this is still prohibitively slow).
I am looking for a solution satisfying (1) faster performance than rref, and (2) it does not generate out-of-memory errors on the linux machine running R2015a.
Does anyone know of such solutions?
Other question (not as important as question above):
It is very odd that a machine with 64GB RAM generates an out-of-memory error while a machine with 8GB RAM does not. The differences between the machines are that the machine with 64GB RAM is running R2015a on linux, while the 8GB machine is running R2015b on OSX 10.10 (Yosemite). Are there known memory management issues with R2015a, or Matlab linux distributions?