Finding the number of datapoints in a text file

I have multiples text files each with a header, 200 data points, and a footer. I'm trying to figure out a way of making a script capable of finding where the datapoints start and stop in each file, and load those data points into a workspace table.

I've attached a text file to make it easier to understand what I'm working with.

Answer 1

[Rik]

It looks like you can look for the line with '[data]' on it, and continue until the first line that doesn't start with a digit or a minus sign.

The code below takes the scenic route, but it gets there in the end. If you have a very large number of files you may want to consider a more efficient method.

A=readfile('https://www.mathworks.com/matlabcentral/answers/uploaded_files/348641/IN01_SCALED.txt');
n_start=0;n_end=-1;
for n=1:numel(A)
    if strcmpi(A{n},'[data]')
        n_start=n+1;
    end
    if n_start>0 && n>=n_start
        if ~any(A{n}(1)=='-0123456789')
            n_end=n-1;
            break
        end
    end
end
B=A(n_start:n_end);
B=cellfun(@(x) cellfun(@str2double,strsplit(x,',')),B,'UniformOutput',0);
B=cell2mat(B);

Let's stretch the definition of what an earlier release is. I made the above code compatible with just about any Matlab release. I didn't test it, but it should work with releases as far back as R13 (version 6.5). For the modern code I used the solution by Star Strider.

UseFallbackMethod=true;try if ~verLessThan('matlab','9.6'),UseFallbackMethod=false;end,catch,end
if UseFallbackMethod
    A=readfile('https://www.mathworks.com/matlabcentral/answers/uploaded_files/348641/IN01_SCALED.txt');
    n_start=0;n_end=-1;
    for n=1:numel(A)
        if strcmpi(A{n},'[data]'),n_start=n+1;end
        if n_start>0 && n>=n_start && ~any(A{n}(1)=='-0123456789')
            n_end=n-1;break
        end
    end
    B=A(n_start:n_end);
    %the loop below is equivalent to:
    %B=cellfun(@(x) cellfun(@str2double,strsplit(x,',')),B,'UniformOutput',0);
    for n=1:numel(B)
        s=strfind(B{n},',');
        split=zeros(1,numel(s)+1);
        start_index=[s numel(B{n})+1];
        stop_index=[0 s];
        for m=1:numel(start_index)
            str=B{n}((stop_index(m)+1):(start_index(m)-1));
            split(m)=str2double(str);
        end
        B{n}=split;
    end
    B=cell2mat(B);
else
    B = readmatrix('IN01_SCALED.txt');
    nonan = ~isnan(B);
    B = [B(nonan(:,1),1) B(nonan(:,2),2)];
end

[Natalie Cannon]

Very thorough. Exactly what I needed. Thanks for your help!

Answer 2

[Star Strider]

Another option:

A1 = readmatrix('IN01_SCALED.txt');
nonan = ~isnan(A1);
A1 = [A1(nonan(:,1),1) A1(nonan(:,2),2)];

This actually takes care of all the header lines itself, and produces the (201x2) array in ‘A1’. The only drawback is that the readmatrix function was introduced in R2019a, so this will not work for earlier releases.

[Rik]

I took the liberty of merging your answer with mine. I guess this shows what kind of decluttering you can do with a more modern release.

[Star Strider]

That works for me!

[Natalie Cannon]

This is great, thank you so much for your input.