What code are you using to write the text? Opening and closing files can be slow, and your disk might also be the bottleneck. Modern releases of Matlab tend not to be the bottleneck of a file operation.
Is it possible to accelerate the speed of saving data into files with parallel way?
6 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
wei zhang
am 18 Jul. 2020
Bearbeitet: Bruno Luong
am 21 Jul. 2020
I am trying to save data to files, i.e. txt file. The speed is very low. This step takes almost 80% time over my whole script. My computer has many cores. Could I use them, i.e. parallel way, to help to accelerate the speed in Matlab? I don't know much about know this. May it is impossible. Please just tell me some idea or suggestion about it? Thank you very much.
(edited as Rik suggested)
My code uses fprintf function in a big loop. This is the most time consuming step.
i.e. I need to save a large matrix a, which is 4000000*4
p = fopen(filename,'w');
for i=1:length(a)
fprintf(fp,'%d %d %d %d\n',a(i,1),a(i,2),a(i,3),a(i,4));
end
Is there any method to accelerate it?
4 Kommentare
Rik
am 19 Jul. 2020
Without knowing more about your code, this is only conjecture. You need to post exact code if you want specific advice. You can also run the profiler to find bottlenecks in your code.
Akzeptierte Antwort
Bruno Luong
am 21 Jul. 2020
Bearbeitet: Bruno Luong
am 21 Jul. 2020
You might try to remove the for-loop
fp = fopen(filename,'w');
fprintf(fp,'%d %d %d %d\n', a(:,1:4).'); % remove the indexing with your array has 4 columns
fclose(fp);
A file is a sequential access device. There is no use of parallel CPU. When you write a file a hard disk head must go sequentially in tracks and put a little electrical signal there to store your data. More or less samething happens with all other HW, even SSD.
Another thing that takes time is convert the internal binary format to digital of your fprintf('%d').
If you can, just write file in binary format. That is the best method (fatest).
2 Kommentare
Bruno Luong
am 21 Jul. 2020
Try this:
fp = fopen('test.txt','w')
fwrite(fp, repmat(sprintf('0 0\n'),1,n_face));
fclose(fp);
Or you could do with the same method as I have showed
fp = fopen('test.txt','w');
a = repmat([0 0], n_face, 1);
fprintf(fp,'%d %d\n', a); % no need for transpose since it's all 0s
fclose(fp);
Weitere Antworten (1)
Mohammad Sami
am 21 Jul. 2020
Bearbeitet: Mohammad Sami
am 21 Jul. 2020
Is there a reason why you want to use a for loop to write this ?
You can write the entire matrix to file in one go using writematrix function.
a = randi([0 10],1000000,4);
tic;
writematrix(a,'abc.txt','FileType','text','Delimiter',' ');
toc;
1 Kommentar
Rik
am 21 Jul. 2020
I'm on mobile so I can't write and test the code, but you can also use fprintf. You just need to make sure the orientation of a is correct (it will read the input column by column when writing the lines in the text file).
Siehe auch
Kategorien
Mehr zu Startup and Shutdown finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!