Reading an array of 800 X 1280 data points for a file and compare to a criteria (a number) and report (write) any numbers lower than criteria into an excel file

Hello MathLab community,

I have a folder of several txt files, each contains of 800 X 1280 data points. I want to develop a code to start reading each txt files and compare each number to an arbitary value as criteria (for example 400) and if the number is lower than that, report all those numbers to an excel file row (row 1) , and then move to next txt file and do the same (row 2, 3 etc).

At the end of it, I would have an excel file includes several rows (the number of the txt files in the reading folder) and each has numbers below my criteria (depends on the number of the values lower than the criteria ,here is 400 as an example).

It would be great to have each file name on the first cell of each row.

My Question is what would be the fine way to combine the above concerns in an effcient code to run?

[Rik]

You forgot to ask a question.

Have a read here and here. It will greatly improve your chances of getting an answer.

[Aramis]

Thanks for noting Rik. I added my question then

[Rik]

You may have added a question, but you evidently didn't read both links. What did you try? Which part is causing you issues to implement? You have a list of steps ready to implement. What exactly do you need help with?

[Aramis]

Hi Rik,

I manifestly have read links while ago when posting my first question in the community.

I tried a few of ma calculations which went well, but the problem is how to make a loop to scan a line (including 1280 data values) and then jump to the next line of data ( 800 lines in total).

Below is my attemp so far, I appreciate your help to make it working

projectdir = 'C:\Users\test';

filenameList = dir( fullfile(projectdir, '*.txt')) ;

finalArray = [];

num_files = length(filenameList);

fullnames = fullfile({filenameList.folder}, {filenameList.name});

[~, basenames, ~] = fileparts(fullnames);

for filenameIndex = 1:num_files

filename = fullnames{filenameIndex};

inData = importdata(filename,'\t',8);

inData = inData.data;

outArray = Calc1(inData);

outArray2 = Calc2(outArray);

finalArray(:, filenameIndex) = outArray2;

end

T = array2table(finalArray, 'VariableNames', basenames);

T = array2table(finalArray','RowNames', basenames);

writetable(T, 'test.xlsx',"WriteRowNames",true);

[Rik]

Your code is missing comments, so it is unclear what it is attempting to do. There are two functions (Calc1 and Calc2) or variables that are very important, but are not described. Is that where the criterion is applied?

From your code I can't deduce what you want to happen.

Can you attach a few files (3 perhaps) and include code that will parse each how you need them? It doesn't have to be real data, but a small example that can run from within the Answers editor will help enormously in understanding what you need.

[Peter Perkins]

Whenever you hear ("excel file", think tables and timetables. In your case, you would have a table with one variable for the file name and one var for the numbers you want. But each file would have a different number of numbers of interest, so store those in a cell array var:

>> t = table(["A";"B";"C"],{1;1:2;1:3})
t =
  3×2 table
    Var1      Var2   
    ____    _________
    "A"     {[    1]}
    "B"     {[  1 2]}
    "C"     {[1 2 3]}

It is perhaps not well known, but writetable expands cell arrays containing vectors:

>> writetable(t)
>> type t.txt

Var1,Var2_1,Var2_2,Var2_3
A,1,,
B,1,2,
C,1,2,3

Answer 1

[Divyanshu]

You can have a look at the below demo script which uses three demo text files test1, test2 & test3. And criteria that must be satisfied is “less than 5”. And we have taken T as a cell array. Its each row has two elements:

1. Name of the file
2. The vector of numbers in the file which satisfied the criteria.

The reason to take a cell array is because the number of elements in the vector are not fixed and would be different for each file.

Finally, we created a table from this cell array and then used writetable function to produce required excel sheet. Here is the code snippet:

fullNameList = ["test1.txt","test2.txt","test3.txt"]
T = {};
for k=1:length(fullNameList)
    M = readmatrix(fullNameList(k));
    res = [];
    for i=1:2
        for j=1:6
            if M(i,j) < 5
                res(end+1) = M(i,j);
            end
        end
    end
    T(end+1,:)  = {res};
end
out = table(fullNameList',T)
writetable(out,'myData.xls','Sheet',1)

Refer the following documentation for better understanding of writetable function:

Write table to file - MATLAB writetable (mathworks.com)