Are there any faster alternatives to readlines?
26 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Rahim Zaman
am 6 Feb. 2025
Beantwortet: Steve Eddins
am 7 Feb. 2025
I have a MATLAB script that uses readlines to get the input from two separate short text files, the first one a single time and the second one multiple times in a for loop. I am using the profiler tool to optimize the runtime of my script and readlines currently takes 46% of the time (18 s) for my script to run. Are there any faster alternatives to readlines to shorten the runtime?
1 Kommentar
Steve Eddins
am 6 Feb. 2025
Can you share more details with us? Why is the second file being read more than once? What is the size of the file being read in the loop, and how many text lines does it contain? How many times does the loop execute? Are you able to post a sample file?
Akzeptierte Antwort
Steve Eddins
am 7 Feb. 2025
Try this:
- Read the file using fileread.
- Convert to string.
- Call split.
Using Walter's idea for a sample text file:
filename = fullfile(matlabroot,"license_agreement.txt");
chars = fileread(filename);
text = string(chars);
lines = split(text,newline);
whos lines
lines(1:5)
See my comment under Walter's example for detailed timing comparisons.
0 Kommentare
Weitere Antworten (1)
Walter Roberson
am 6 Feb. 2025
For the purposes of the below test, I will assume that it is important that the text be split into lines, but that it is not important whether those lines are represented as a string array or as a cell array of character vectors.
filename = fullfile(matlabroot,"license_agreement.txt");
tic; S1 = readlines(filename); t1 = toc; whos S1
tic; S2 = fileread(filename); S2a = regexp(S2, '\r?\n', 'split'); t2 = toc; whos S2a
tic; fid = fopen(filename); S3 = fread(fid, [1 inf], '*char'); fclose(fid); S3a = regexp(S3, '\r?\n', 'split'); t3 = toc; whos S3a
tic; fid = fopen(filename); S4 = textscan(fid, '%s', 'Delimiter', '\n'); fclose(fid); S4a = S4{1}; t4 = toc; whos S4a
format long g
[t1; t2; t3; t4]
So fastest is fread() followed by splitting. Second fastest is textscan(). Third fastest is fileread() followed by splitting. Slowest by a noticable amount is readlines.
tic; S3b = string(S3a); toc
If string representation is necessary, converting from cell array of character vector to string takes a small but measureable time.
Note: textscan() is handling end-of-file slightly differently than the alternatives. The issue comes about because the file ends in a newline. textscan() eats the final newline and then looks for more content and does not find it, and declares that the file has finished. The alternatives on the other hand treat the newline as a separator and split at the newline, and so end up with a final empty string.
1 Kommentar
Steve Eddins
am 7 Feb. 2025
Walter, I think your readlines result may be suffering from first-time measurement effects.
I took your very nice collection of methods, put them into functions, and timed them using timeit. Looks like readlines is slower, by roughly 2-3x. The other three methods you suggested are all in the same neighborhood and faster than readlines.
Another variation, fileread_string_split_method, looks like it may be faster than the others. The steps:
- Read the file using fileread.
- Convert to string.
- Call split.
I know that the dev team who worked on string and its methods put a lot of effort into optimizing things.
filename = fullfile(matlabroot,"license_agreement.txt");
f_readlines_method = @() readlines_method(filename);
f_fileread_regexp_method = @() fileread_regexp_method(filename);
f_fread_regexp_method = @() fread_regexp_method(filename);
f_textscan_method = @() textscan_method(filename);
f_fileread_string_split_method = @() fileread_string_split_method(filename);
t_readlines_method = timeit(f_readlines_method)
t_fileread_regexp_method = timeit(f_fileread_regexp_method)
t_fread_regexp_method = timeit(f_fread_regexp_method)
t_textscan_method = timeit(f_textscan_method)
t_fileread_string_split_method = timeit(f_fileread_string_split_method)
function out = readlines_method(filename)
out = readlines(filename);
end
function out = fileread_regexp_method(filename)
out = regexp(fileread(filename), '\r?\n', 'split');
end
function out = fread_regexp_method(filename)
fid = fopen(filename);
chars = fread(fid, [1 inf], '*char');
fclose(fid);
out = regexp(chars, '\r?\n', 'split');
end
function out = textscan_method(filename)
fid = fopen(filename);
out_cell = textscan(fid, '%s', 'Delimiter', '\n');
fclose(fid);
out = out_cell{1};
end
function out = fileread_string_split_method(filename)
out = split(string(fileread(filename)),newline);
end
Siehe auch
Kategorien
Mehr zu Variables finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!