Code for looping through a folder and sub-folders within that folder to get .csv files and move them to the main path

So I have a main folder on the desktop called 'Jasper_190319'.

Within this folder there are 5 sub-folders, called:

'Jasper_190319_104907', 'Jasper_190319_113628', 'Jasper_190319_125141', 'Jasper_190319_131031' and 'Jasper_190319_133135'.

These folders may or may not contain .csv files. I want help writing a code that will select the main folder, and loop through the sub-folders, find the .csv files (if present) and move them to the main path (which is where the main folder is stored).

Could anyone help me with this? Will really appreciate it!!!

EDIT: I also have another question, if I wanted to read the csv files into a table on the main path, how would I do that?

Answer 1

[Mohammad Sami]

You can use the dir function to list the folders and files in any folder.

You can use the movefile function to move the files to a new location.

It would go something like this

mainpath = uigetdir; % open gui to select main path
mainfileandfolders = dir(mainpath);
subfolders = mainfileandfolders([mainfileandfolders.isdir]); % find the subfolders
csvfiles = {};
for i = 1:length(subfolders)
    subpath = fullfile(mainpath,subfolders(i).name); % path to subfolder
    csvfiles{i} = dir(fullfile(subpath,'*.csv')); % list the csvfile in subfolder
end
csvfiles = vertcat(csvfiles{:}); % concatenate all csvfiles
for i = 1:length(csvfiles)
    origpath = fullfile(csvfiles(i).folder,csvfiles(i).name);
    [status,message,messageId] = movefile(origpath,mainpath,'restricted','f') % see doc movefile
end

[Zuha Yousuf]

Hi thanks so much for responding! When I run this, it gives me an error with respect to movfile, saying that there are too many input arguments. Is there any way to fix this?

[Walter Roberson]

Probably

    [status,message,messageId] = movefile(origpath,mainpath) % see doc movefile

You could also add 'f' to the end of the options, to mean "force", but in my opinion in a situation like this, if there is a problem with the move, you would prefer to know about it rather than to trigger a potentially wrong response.

[Zuha Yousuf]

Thank you all so much! That worked. I have another question, if I wanted to read the csv files into a table on the main path, how would I do that?

[Walter Roberson]

tables do not exist on a path, tables exist in memory.

[Zuha Yousuf]

So there isn't a way to create an empty table in the main folder that populates with the data from the csv files? Just making sure.

[Walter Roberson]

Table objects are not files, they do not exist in any directory.

It *is* possible to combine contents of multiple files into one file including possibly xlsx files. This is easier if each file is to be written to a separate sheet, or if all of the files have the same columns in the same order and you just want to append them all without marking the ends of the files.

[Mohammad Sami]

I think a workaround can be to load all the csv files in memory, concatenate them and then write it back to the mainfolder as a new csv file.

csvtables = {};
for i = 1:length(csvfiles)
    origpath = fullfile(csvfiles(i).folder,csvfiles(i).name);
    csvtables{i} = readtable(origpath);
end
mergedtable = vertcat(csvtables{:}); % this only works if all csv files have same columns and order and format
writetable(mergedtable,fullfile(mainpath,'mergedtable.csv')); % this will create a mergedtable.csv file on your main path

[Walter Roberson]

I suspect that there might not be a header on the csv files; if that is the case, then the readtable() call should have 'readvariablenames', false added to the call.

[Ayesha Batool]

@muhammad Sami

in your code above where do you give the path to the main folder:

mainpath = uigetdir; % open gui to select main path
mainfileandfolders = dir(mainpath);
subfolders = mainfileandfolders([mainfileandfolders.isdir]); % find the subfolders

My main folder is

D:\PhD\Research\Experimental Design\Experiment 1\DATA\Analysis\Nystrom\100514\DetectionResults\