Load .dat files from subfolders with different names

Hello everyone,

I have Folder1 which contains a number of subfolders with different names (subfolder_1,subfolder_th etc). Each subfolder has 7 .mat files with the same name (data1,data2,....data7). So i've got two questions:

1. Is it feasible somehow to open one subfolder at a time in a for loop in order to process tha data?

2. Is there a way to load for example data1 from all subfolders and then do the same thing for the other data?

*i guess the main problem is the subfolder's name detection and storing.

[Stephen23]

"Is it feasible somehow to open one subfolder at a time in a for loop in order to process tha data?"

You do not need to "open" a folder in order to access any files in that folder. Although you could "open" the folder in some external app (e.g. Windows Explorer), that makes absolutely no difference to accessing the files from MATLAB. All you need is to use an absolute/relative filename:

https://www.mathworks.com/help/matlab/matlab_env/specify-file-names.html

https://www.mathworks.com/help/matlab/ref/fullfile.html

"i guess the main problem is the subfolder's name detection and storing."

I don't see why that should be any problem, you can just use DIR to get the main folder contents or SPRINTF to genrate the subfolder names, just as the documentation shows:

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

What have you tried so far?

https://www.mathworks.com/help/matlab/ref/dir.html

[ibt]

Dear, I want to store the files in each subfolder in an array, but I don't see a way to make matlab read each file in each subfolder.

[Stephen23]

"but I don't see a way to make matlab read each file in each subfolder."

• outer loop that loops over the subfolders
• inner loop that loops over the files
• FULLFILE to construct the filenames.
• pick a suitable function to import your file data: https://www.mathworks.com/help/matlab/standard-file-formats.html

[ibt]

This I have so far, I don't know how to start a loop correctly so that it reads the files of interest to me.

Base = 'D:\Date';  
List = dir(fullfile(Base, '*.*));
List = List([List.isdir]);
SubFolder = {List.name};
SubFolder(ismember(SubFolder, {'.', '..'})) = [];

Answer 1

[Stephen23]

This should get you started. Adapt to suit your data files, folder names, etc.:

P = 'absolute/relative path to where the files are saved';
n1 = 19; % number of sub-directories
n2 = 7;  % number of mat-files.
C = cell(n1,n2);
for k1 = 1:n1
    D = sprintf('subfolder_%d',k1);
    for k2 = 1:n2
        F = sprintf('data%d.mat',k2);
        C{k1,k2} = load(fullfile(P,D,F));
    end
end

https://www.mathworks.com/help/matlab/matlab_prog/access-data-in-a-cell-array.html

[ibt]

Dear, this code does not work for me.

[Stephen23]

"this code does not work for me."

I cannot help debug your code if you do not show what you tried and explain what happened when you ran it.

Did the code run without error? Did you get some results, but not the results you expected? Did the code throw any warnings? Did the code throw an error (if so, what is the complete error message?) Did the code run but not return anything? How did you check this?

If you tell us nothing, we cannot help you debug. If you tell us nothing, it slows down your work.

Answer 2

[per isakson]

Yes to both questions. With help of the code snippets below I guess you can put together a script.

sad = dir( fullfile('Folder1','**','data*.mat') );  % assuming the names starts with "data"
ffs_list = fullfile( {sad.folder}, {sad.name} );    % full file specifier 
for cac = ffs_list  % cac is a scalar cell array
    ffs = cac{1};
    % do stuff 
end

or maybe better in you case

sad = dir( fullfile('Folder1','**','data*.mat') );    % sad is a 1D struc array
for jj = 1 : length(sad)
    sad(jj).name
    sad(jj).folder
    % do stuff 
end

"I guess the main problem is the subfolder's name detection" The '**' in the dir-statement does that.

The statements

isa = ismember( {sad.name}, 'data1.mat' );
sad_data1 = sad( isa );

creates a subset containing all the files named, 'data1.mat'

The statements (assuming that the subfolder names are legal Matlab names)

cac=regexp( string({sad.folder}), '(?<=\\)\w+$', 'match' );
unique_subfolder_names = unique( horzcat( cac{:} ) );

creates a string array of unique subfolder names. To sort this array you might need Natural-Order Filename Sort by @Stephen Cobeldick

And see Auto fetch input files to process from a specified folder

[Stephen23]

Using the recursive search is a good idea, it also makes storing the data easy in the same structure:

sad(jj).data = ... whatever

Note that using DIR by itself will not return folder names subfolder 1 ... 19 in alphanumeric order.

https://www.mathworks.com/matlabcentral/answers/834733-store-txt-files-from-different-subfolders

[per isakson]

"Note that using DIR by itself will not return folder names subfolder 1 ... 19 in" indeed it does not.