How to automate running the same scripts in multiple folders

I have 100 different images that I would like to analyze with the same 5 scripts. Currently I have each image in its own folder, and all of the folders in the same directory. Each subfolder also contains the 5 scripts used to analyze the images, in addition to the image itself.

In order to automate the analysis I wrote a short loop that navigates to each subfolder, runs each the 5 scripts, navigates to the next subfolders, and so on and so forth (see below).

The problem I am running into is that this approach requires saving a .mat file into each subfolder and then reloading after running the scripts in order to maintain the memory of which folders it needs to navigate to. This is a problem because the .mat file is sort of big and if I ever need to apply this script to thousands or hundreds of thousands of images it is going to add up to quite a bit of space.

Is there any way to structure the loop so that the .mat file doesn't need to be saved? Or perhaps a completely different way to approach this problem? Any feedback would be appreciated.

g=uigetdir('','Select Input-folder'); 
cd(g);
list = dir;
list = list([list.isdir]); 
list = list(~ismember({list.name},{'.' '..'}));
l=length(list);
for i=1:l
    dest = list(i).name ;
    cd(dest);
    save datafile.mat;
    temporal_1 ;
    temporal_2 ;
    temporal_4 ;
    temporal_5 ;
    clear ;
    load datafile.mat
    cd(g); 
end

[Matt Gaidica]

It's unclear what's being saved and loaded. Can you create a MAT-file with a table in your project root with a column for the path and a flag for whether you analyzed it?

Answer 1

[Jan]

Do not use scripts, but functions for reliable code. The workspaces of the scripts influence eachother and you see, that you need a clear command already to avoid some problems. With function there is no need for clearing variables and therefore you can avoid saving and loading the mat files.

Do not store the scripts or functions in the same folder as the data. As far as I understand, the scripts are identical? Then save them in a folder, which is included in Matlab's path. Instead of changing the current directory using cd(), prefer to provide the files with a full path:

folder = uigetdir('','Select Input-folder'); 
list = dir(folder);
list = list([list.isdir]); 
list = list(~ismember({list.name}, {'.' '..'}));

for i = 1:length(list)
    subfolder = fullfile(folder, list(i).name);
    
    temporal_1(subfolder);
    ...
end

Modify temporal_1 such, that it accepts an input an processes the included files with their absolute path using fullfile.

Changing the current directoy with cd() is a fragile method, because any GUI or TIMER callback or can do this also. Generations of programmers produced code, which was susceptible for this serious problem. Using absolute paths in general is one of the points needed for parallelization of the code: If you have the Parallel Computing Toolbox, changing "for" to "parfor" can run the code in multiple threads. This might be useful if you process hundrets of thousands of files.

[Zac Swider]

Thank you for this answer! I should have qualified my question with the disclaimer that I am a suuuper novice matlab programmer, so I will need to figure out how to turn these temporal scripts (which somebody else wrote) into functions. Still, I understand enough to see the virtues of organizing the code this way.

I will report back as soon as I can test, thanks again.