Directory Issue - not finding subfolder

Hi,

(Note: all subfolders and the subfolders' subfolders are on the search path).

I'm currently mapping and extracting data from different folders. I've mapped the present working directory, and its subfolders'. However, when i try to map the subfolders subfolders using dir, i get a message saying that the specific folders are not found. However, if i click the subfolder and attempt to run dir to map the subsubfolder, it works. I need to run this code for a large scale of folders, so clicking each subfolder and running dir by hand is not feasible.

eg NSW -> 2018 -> April 2018 -> file i need to extract.

dir('2018') works, but dir('April 2018') does not work, unless i click into the 2018 folder.

If i change the current directory to 2018 (ie cd 2018), then dir('April 2018') works. However, i need to change the cd back to the parent NSW directory if i use this method, which i haven't been able to figure out how to do.

Any advice on how to approach this issue is appreciated. Apologies if this question has been asked before, but i could not find an answer looking at previous posts.

If i've been unclear, i will clarify any area.

Cheers,

Kiran

[Stephen23]

"Note: all subfolders and the subfolders' subfolders are on the search path"

Do NOT do this!

The MATLAB Search Path should only be for code.

In general folders of data should not be on the MATLAB Search Path.

[Guillaume]

As Stephen says don't cd or modify matlab search path. It's unnecessary, slow and dangerous.I would recommend that you always use absolute path.

You talk about mapping and it's not clear what you mean by that. One thing that matlab has been known to have trouble with is synchronised folder (e.g. dropbox or onedrive) and networked folder. Is this what you mean by mapping?

For local folder, if it exists, dir will find it as long as you provide the correct path.

Answer 1

[Stephen23]

"Any advice on how to approach this issue is appreciated."

• Do not use cd.
• Do not add data folders to the Search Path.
• Use absolute or relative file paths.
• Construct the file paths using fullfile.

Either use dir or sprintf to suit your requirements, e.g. (untested, but should get you started):

P = 'path of the root directory'
S = dir(fullfile(P,'*'));
Q = setdiff({S([S.isdir]).name},{'.','..'});
for ii = 1:numel(Q) % loop over subfolders.
    S = dir(fullfile(P,Q{ii},'*'));
    R = setdiff({S([S.isdir]).name},{'.','..'});
    for jj = 1:numel(R) % loop over subsubfolders.
        S = dir(fullfile(P,Q{ii},R{jj},'*.txt'));
        for kk = 1:numel(S) % loop over files.
            F = fullfile(P,Q{ii},R{jj},S(kk).name); % Filename
            ... process your file
        end
    end
end

See also:

https://www.mathworks.com/help/matlab/import_export/process-a-sequence-of-files.html

and many threads that discuss looping over subfolders, e.g.:

https://www.mathworks.com/matlabcentral/answers/437494-how-to-loop-through-all-files-in-subfolders-in-a-main-folder

https://www.mathworks.com/matlabcentral/answers/293919-how-to-loop-through-subfolders-and-apply-a-particular-function#answer_375509

https://www.mathworks.com/matlabcentral/answers/359493-cannot-iterate-over-subdirectories-from-data-structure

etc.

And also:

https://www.mathworks.com/matlabcentral/answers/376645-how-to-use-a-function-that-is-not-in-the-same-folder-as-your-current-folder

[Kiran Eswaran]

Thank you for your prompt response, i will try this!!

[Walter Roberson]

Mathworks puts a fair number data folders on the search path. That is the reason why you can

imshow('cameraman.tif')

without having to use an auxiliary function to locate the file or directory.

[Kiran Eswaran]

Adapted it slightly and works an absolute charm, you are a legend Stephen, very clear code and learnt along the way, really appreciate your help mate!

Answer 2

[Steven Lord]

Since you're using release R2016b, you can use the functionality added to dir in that release to recursively search through folders and subfolders. Let me find all MAT-files in subdirectories of the matlab directory in the toolbox directory under matlabroot.

>> cd(matlabroot)
>> cd toolbox/matlab
>> D = dir('**/*.mat');

Because I'm using release R2019a (what I have open right now) the specific numbers may be different from the results you receive in your installation of release R2016b, but for my installation item 43 in the struct returned by dir is one of the MAT-files used by some of the MATLAB demos.

>> D(43)
ans = 
  struct with fields:

       name: 'usborder.mat'
     folder: 'C:\Program Files\MATLAB\R2019a\toolbox\matlab\demos'
       date: '14-Mar-2004 10:32:22'
      bytes: 1735
      isdir: 0
    datenum: 7.3202e+05

If I want to work with it, I can build the full file name and pass that into functions like whos, load, etc.

>> theFullName = fullfile(D(43).folder, D(43).name);
>> whos('-file', theFullName)
>> data = load(theFullName);

No changing directory required except in the setup before I called dir, and even that I could have avoided using fullfile:

>> Q = fullfile(matlabroot, 'toolbox', 'matlab', '**/*.mat');
>> D2 = dir(Q);
>> isequal(D, D2) % true

[Kiran Eswaran]

Awesome, cheers for your prompt response, i will try this!!