How working on a matrix with a large number of rows?
5 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
l.l.
am 31 Okt. 2014
Kommentiert: l.l.
am 3 Nov. 2014
Hello everybody! I have a very large array, about 80000 rows and 3 columns. I need to calculate the distances between each line and the others.
I tried to calculate the possible combinations with the function nchoosek , but does not: out of memory
with a loop on the array takes too long ...
how can I do?
3 Kommentare
per isakson
am 31 Okt. 2014
See
- matfile, Access and change variables directly in MAT-files, without loading into memory
- matlab.io.MatFile class
it might be worth trying although it will be slow.
Image Analyst
am 31 Okt. 2014
What are you really trying to do? Give us the big picture. It may be possible to do that without having all those distances held in memory at the same time . By the way, 240,000 numbers is not large at all (it's way smaller than a typical digital image) - it's the fact that you need distances of every row/line to every other line that takes up all the memory.
Akzeptierte Antwort
Mohammad Abouali
am 31 Okt. 2014
Bearbeitet: Mohammad Abouali
am 31 Okt. 2014
try this:
Assuming X is storing your coordinates, (1 point per row) and (x,y,z) are the three columns, use pdist as follows:
d=pdist(X);
or if you are running out of memory:
d=pdist(single(X));
if you still go out of memory then you need bigger machine
Using pdist doesn't require you to calculate possible combination manually hence you save some memory there. Your original data 80000x3 at double precision takes less than 2MB of memory; so, I wouldn't worry about that loading it in memory. But your results in single precision would take about 12GB. Double precision make that double.
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu Matrices and Arrays finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!