How to extract values from rows and convert into a single column for several files

Hi, I have several NUM_*.dat files as the two attached. I need to extract rows (excluding some specific values) from that file and convert them into a combined column of values. For instance, from the first file, I need to get a single column which in order includes the values from the file starting in row 2, but excluding the first value of the even rows (row2,4,6...). This process should be done for several NUM_*.dat files, so that the final result is a file which contains as many columns as NUM_*.dat files. Can I get some help to do this in an efficient way? Thanks

[Jan]

What have you tried so far? Which specific problem do you have? It would be inefficient, if the readers guess, if creating a loop to adjust the file names belongs to the problem or not. So please post the existing code.

[Isma_gp]

I've tried textscan, but only the first row of values is imported.

Answer 1

[Ameer Hamza]

The following code will work. It will read every odd line, remove the first number and add it to a column. Since we don't know in advance how many values will be in each file, therefore I used a cell array. If you know the number of elements in each file in advance, consider preallocating the memory

files = dir('NUM*.txt');

matrix = cell(1, numel(files));

for i=1:numel(files)
  f = fopen(files(i).name);

    while true
        line = fgetl(f);  % to ignore the odd lines
        line = fgetl(f);
        if line == -1
            break
        end
        line_ = textscan(line, '%f');
        matrix{i} = [matrix{i}; line_{:}(2:end)];
        line = fgetl(f);  % to ignore every third empty line
    end

    fclose(f);
end

You can access the values like this: matrix{1} will give you column for the first file, matrix{2} for the second file and so on.

[Isma_gp]

Thanks for your help. I would like to read all lines but the first one (odd lines values should also be included in the column, just the first line is not required). For odd lines, all values should be included, also the first value. Thanks!

[Ameer Hamza]

Problem description in such words can be vague. Let's see some concrete example.

3.75 11.25 18.75 26.25 33.75 41.25 48.75 56.25 63.75 71.25 78.75 86.25 93.75 101.25 108.75 116.25 123.75 131.25 138.75 146.25 153.75 161.25 168.75 176.25 183.75 191.25 198.75 206.25 213.75 221.25 228.75 236.25 243.75 251.25 258.75 266.25 273.75 281.25 288.75 296.25 303.75 311.25 318.75 326.25 333.75 341.25 348.75 356.25 
0.1835 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 &

0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000  0.000000 0.000000
0.2023 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 &

These are first few lines. What do you want your column to look like from this dataset? You don't need to write the entire column, just an example that from which element to start the column and which elements to skip.

[Isma_gp]

From these data set, we would get a column with 192 values (all zeroes, since these first rows have only zeroes). I'm attaching how the start of the rows that need to go into the column would look like. Thanks

[Isma_gp]

The even lines also have a & symbol at the end of the row. The symbol would need to be removed for the column.

[Ameer Hamza]

Are you looking for something like this

files = dir('NUM*.txt');

matrix = cell(1, numel(files));

for i=1:numel(files)
  f = fopen(files(i).name);
  line = fgetl(f); % ignore first line
  lineCount = 2;
  while true
      line = fgetl(f);
      if line == -1
          break
      elseif ~isempty(line)
          line_ = textscan(line, '%f');
          if mod(lineCount, 2) == 0
              matrix{i} = [matrix{i}; line_{:}(2:end)];
          else
              matrix{i} = [matrix{i}; line_{:}];
          end
          lineCount = lineCount + 1;
      end
  end

    fclose(f);
end

[Isma_gp]

That's right. Thanks so much for your help!!

[Ameer Hamza]

You are welcome.