How to read text files from different sub folders in a folder ?

180 views (last 30 days)
Luffy
Luffy on 30 Sep 2015
Commented: Image Analyst on 25 Mar 2021
I have a collection of .txt files which are an output of a simulation software.The software puts the text files in different sub-folders within a folder.Is it possible to read all of those ?
For example:- In a single folder named top, there are 500 sub-folders(each subfolder is of format TC_SYS_01,TC_SYS_02,.....TC_SYS_500) with a text-file in each of them.I want to read those text files from each of those folders & do some operations.
I'm thinking of changing the simulation software code itself so that it outputs txt files within a folder(no sub-folders) & use this code:
F = dir('*.txt');
for i = 1:length(F)
text = fileread(F(i).name) ;
% Do further operations
end
but can i do it without changing the simulation software code and this : http://in.mathworks.com/matlabcentral/newsreader/view_thread/301929 doesn't help me

Accepted Answer

Image Analyst
Image Analyst on 30 Sep 2015
Not sure which code you used in that link, but you can use genpath() to generate a list of all subfolders, then use a loop to recurse into each of them. In the innermost for loop is where you'll process your text file. Try it - just copy and paste and it should work. Here is the code:
% Start with a folder and get a list of all subfolders.
% Finds and prints names of all text files in
% that folder and all of its subfolders.
% Similar to imageSet() function in the Computer Vision System Toolbox: http://www.mathworks.com/help/vision/ref/imageset-class.html
clc; % Clear the command window.
workspace; % Make sure the workspace panel is showing.
format long g;
format compact;
% Define a starting folder wherever you want
start_path = fullfile(matlabroot, '\toolbox');
% Ask user to confirm or change.
topLevelFolder = uigetdir(start_path);
if topLevelFolder == 0
return;
end
% Get list of all subfolders.
allSubFolders = genpath(topLevelFolder);
% Parse into a cell array.
remain = allSubFolders;
listOfFolderNames = {};
while true
[singleSubFolder, remain] = strtok(remain, ';');
if isempty(singleSubFolder)
break;
end
listOfFolderNames = [listOfFolderNames singleSubFolder];
end
numberOfFolders = length(listOfFolderNames)
% Process all text files in those folders.
for k = 1 : numberOfFolders
% Get this folder and print it out.
thisFolder = listOfFolderNames{k};
fprintf('Processing folder %s\n', thisFolder);
% Get filenames of all TXT files.
filePattern = sprintf('%s/*.txt', thisFolder);
baseFileNames = dir(filePattern);
numberOfFiles = length(baseFileNames);
% Now we have a list of all text files in this folder.
if numberOfFiles >= 1
% Go through all those text files.
for f = 1 : numberOfFiles
fullFileName = fullfile(thisFolder, baseFileNames(f).name);
fprintf(' Processing text file %s\n', fullFileName);
end
else
fprintf(' Folder %s has no text files in it.\n', thisFolder);
end
end
  6 Comments
Image Analyst
Image Analyst on 25 Mar 2021
You could certainly put a call to your function inside the loop over f
[output] = extract_data(fullFileName); % Do some custom operations in this function.
You might also want to look into the imageDatastore() or dir() with two asterisks to have just one loop rather than a pair of nested loops.

Sign in to comment.

More Answers (3)

Meade
Meade on 2 Mar 2017
You can try the code below. It will do the same thing as Image Analyst's post, but is a little cleaner if you don't need any of the nested folder information, i.e. you only want the file paths.
mainFolder = uigetdir(); % Select your Main folder
[~,message,~] = fileattrib([mainFolder,'\*']);
fprintf('\n There are %i total files & folders.\n',numel(message));
allExts = cellfun(@(s) s(end-2:end),{message.Name},'uni',0);% Get file ext
TXTidx = ismember(allExts,'txt');% Search extensions for "CSV" at the end
TXT_filepaths = {message(TXTidx).Name}; % Use idx of TXTs to list paths.
fprintf('There are %i files with *.txt file ext.\n',numel(TXT_filepaths));
for ii = 1:numel(TXT_filepaths)
% Do import here
end
  2 Comments
Stephen
Stephen on 16 Sep 2019
"the code runs well but reads the extension after the frist "
There is nothing in this code that detects the period character, so it doesn't even know where the first period character is.
Note that the code is buggy as it incorrectly assumes that all file extensions have three characters: this can easily be fixed using fileparts.
"any advice is appreciated!"
Write simpler, more robust code using dir, fullfile, and fileparts.

Sign in to comment.


KSSV
KSSV on 30 Sep 2015
Hi
You can get the names of the sub folders inside a folder using the following piece of code.
dirinfo = dir(pathfolder);
dirinfo(~[dirinfo.isdir]) = []; %remove non-directories
dirinfo(1:2) = []; % Remove the first two fields as they are . and ..
dirinfo is a structure with all the information. You can run a loop along this structure, go to the specific folder and then you can select the text files in it.
Cheers
Srinivas
  2 Comments

Sign in to comment.


Thorsten
Thorsten on 30 Sep 2015
filenames = getAllFiles('.', '*.txt', 1);
and then loop through the filenames and work on them; you do not need to cd to the directories.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by