How to upload 800 csv files that only contains numbers in a cell keeping their names
1 Ansicht (letzte 30 Tage)
Ältere Kommentare anzeigen
Hello, I would like to import my 800 csv files by keeping their name so I can identify them afterwards and perform my image processing. I try this script but it takes too much time. Thank you any help is welcome.
fichiersRecherches = '*.csv';
[FileName,PathName] = uigetfile(fichiersRecherches,'Sélectionnez les fichiers qui ont pour extention csv', 'MultiSelect', 'on');
FileName = cellstr(FileName);
m = cell(1,length(FileName));
for i_file = 1:size(m,2)
m{i_file} = xlsread(fullfile(PathName, FileName{i_file}));
end
6 Kommentare
Jan
am 11 Apr. 2019
For "use the profiler":
doc profile
For the message
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1, field number 3) ==>
There seems to be an unexpected string in row 1 and column 3.
Akzeptierte Antwort
Jan
am 12 Apr. 2019
Bearbeitet: Jan
am 13 Apr. 2019
As I said: The number contain commas as decimal separators. Before such a file can be imported, in much be converted. This costs a lot of time.
Maybe this is more efficient to fix the file contents:
function Comma2Dot(FileName)
file = memmapfile(FileName, 'writable', true);
comma = uint8(',');
point = uint8('.');
file.Data(transpose(file.Data == comma)) = point;
end
Afterwards a simple fscanf(fid, '%g;', [472, Inf]) will import the data efficiently.
By the way, all decimal places are "000000" only. This means that storing only the integer part would be ways better. Storing the data in binary format would even better again. So teh main problem is that a really inefficient file format has been chosen. and you cannot blame the import of MATLAB.
With the original data:
tic;
Comma2Dot('test2.csv');
% Emulatre DLMREAD:
fid = fopen('test2.csv');
C = textscan(fid, '', -1, 'Delimiter', ';', 'EndOfLine', '\r\n', ...
'CollectOutput', 1);
fclose(fid);
data = C{1};
toc
% Elapsed time is 0.120229 seconds.
Now try:
% Write data in binary format:
fid = fopen('TestData.bin', 'W');
% Number of dimensions and size
fwrite(fid, [ndims(data), size(data)], 'uint64');
fwrite(fid, data, 'uint16');
fclose(fid);
tic;
fid = fopen('TestData.bin', 'r');
nDimsData = fread(fid, 1, 'uint64');
sizeData = fread(fid, [1, nDimsData], 'uint64');
data = fread(fid, sizeData, 'uint16');
fclose(fid);
toc
% Elapsed time is 0.013736 seconds.
The timings might be unfair, because reading data, which have been written to disk directly before, will be taken from the disk cache. But the accelerateion is expected: Compare the file sizes of 2'632 kB for the text file and 442 kB for the binary file.
So the actual optimization is not to improve the Matlab code, but to use a smart format to store the file.
8 Kommentare
Jan
am 16 Apr. 2019
Bearbeitet: Jan
am 16 Apr. 2019
"It does not work" is a lean explanation and does not allow to understand, what the problem is. Please post the details.
I've used the posted methods successfully and the timings show, that it will be much faster, if you use a proper file format instead of a text files with commas and meaingless zeros as decimal places. I've explained thois exhaustively already and do not know, how I can help you now.
Weitere Antworten (3)
Siehe auch
Kategorien
Mehr zu File Operations finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!