Reading a text file: Skip lines in the middle of data

Hello,

I am trying to import data into an array from a text file that looks something like this:

 Header lines
Header lines
Header lines
Var1  Var2  Var3  Var4
1      2     3     4
1      2     3     4
1      2     3     4
1      2     3     4
1      2     3     4
mid data header lines
mid data header lines
Var1  Var2  Var3  Var4
1      2     3     4
1      2     3     4 
1      2     3     4
1      2     3     4
1      2     3     4
mid data header lines
mid data header lines
Var1  Var2  Var3  Var4
1      2     3     4
1      2     3     4
... etc until nData

I want to grab all of Var2 into one 1xndata array. The mid data header lines occurs every 5 lines (the real file is every 50 lines, I wanted to spare you). The header is no problem I can skip it and read Var2 up to the first mid data header. I cant figure out how to skip the mid data to continue the fscanf.

[Rik]

You could read the whole file as text and then remove lines as you need to.

[Ben Morrow]

Note: I can do this seeking byte by byte in the text file and can accomplish this if I can get fscanf to only scan one value and not the whole line: it might be important to note I am using MATLAB 2009

[Guillaume]

it might be important to note I am using MATLAB 2009

That's unfortunate. In R2018a, you only need three lines of code to read the whole file, using the DataLine property of the readtable options.

As suggested by Rik, I would simply read the whole file in one go (with fileread), split it into lines (IIRC, strsplit didn't exist in R2009, so use regexp) then get rid of the offending lines and convert the rest.

Answer 1

[Ben Morrow]

Thanks guys! I tried what you were all saying and came up with this, it gets the job done! I can now just call each data row as needed :

 %get the whole file
  file = fileread(filename);
  %change into line by line
  lines = regexp(file, '\n', 'split');

%extract data lines
k = headerlines + 1;
i = 1;

while k < length(lines)
    datalines(i) = lines(k);
    if mod(i,50) ==0
      k = k + middleLines ;
    end

      k = k +1;
      i = i +1;  
  end

%change data strings into numbers
for i = 1:1:length(datalines)
   numberscell(i) = textscan(char(datalines(i)), '%f');
end
%change cell into array
for i = 1:1:length(numberscell)
    numbers(i,:) = cell2mat(numberscell(i));
end

Answer 2

[Ameer Hamza]

Try fgetl(). Read each line one by one and then decide which line to process or ignore. Also after reading the line, use textscan() to extract numeric values. For example, if you detect a line is valid for processing i.e. contains number then you can use

numbers = textscan(line, '%f') % line is char array
numberToKeep = numbers(2) % you only need second number.