How to run a program on an csv file data in batches
1 Ansicht (letzte 30 Tage)
Ältere Kommentare anzeigen
Hello
I am trying to import data from csv file with size (1046520 1). Using the importdata function.
K = load('caamel.csv');
It is taking over an hour to compute my program.
How do I run my code on the excel file in batches instead of one whole data which is taking forever to see result
I will appreciate your help
thanks
Tino
12 Kommentare
Walter Roberson
am 20 Jun. 2019
You implied that the reason that your code is taking so long is that import (but really load) is slow. I showed timing on a file 50 times larger than yours could take about 20 minutes to load() so your file would take about half a minute to load(). If you were to switch to textscan then it should take roughly 250/50 = 5 seconds. At that point you might as well read in all the data and loop over portions of it.
textscan also permits you to to specify a count to read, if you find that easier than loading everything and looping indexing.
Antworten (0)
Siehe auch
Kategorien
Mehr zu Large Files and Big Data finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!