Filter löschen
Filter löschen

Issue with textscan for Matlab 2015b

1 Ansicht (letzte 30 Tage)
Thomas
Thomas am 9 Sep. 2016
I need to read various string and float data from a .csv file. So what I did was use low level i/o commands to read the file line by line. This worked perfectly in Matlab 2014b with no issues. But now, the script hangs after a few lines and the memory usage explodes out of control to well over 50GB. I included a snippet of the code below.
while(feof(Input_File_ID) == 0)
% Do not read next line if current line is Header
if Counter == 0 || Record_Type ~= '2'
aline = fgetl(Input_File_ID); % Get line
Record_Type = aline(1);
end
if Record_Type == '2' % New record detected
% Counter
if mod(Counter, 100) == 0
disp(['Number of records processed: ', num2str(Counter)]);
end
Counter = Counter + 1 % Counts the number of records
% ------------------------------------------------------
% Header Record: Record Type 2
% ------------------------------------------------------
aline_split = textscan(aline, '%d%f%d%d%s%s%s%s%d%s%s%s%s\n', 'delimiter', ',');
Header.rectype = aline_split{1};
BigStruct(Counter).Header = Header;
end
end
If I take out the line with 'Header.rectype' and 'BigStruct', the code just reads through the file. However, if I insert those lines, the memory usage explodes. This issue never happened in Matlab 2014b and only appeared in 2015b and beyond. Some other details: the script doesn't always hang in the same place. Sometimes its after record 44, sometimes after record 161. When it does hang and the memory usage starts to climb, I'm unable to terminate the script without force quitting Matlab. Any help would be very much appreciated! Thanks!

Antworten (0)

Kategorien

Mehr zu Large Files and Big Data finden Sie in Help Center und File Exchange

Produkte

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by