How to load multiple .mat files containing timetables into the workspace and concatenate them vertically

Hi there,

I'm trying to load ~100 .mat files into the workspace and combine them into a single timetable.

Each file contains a timetable from a specific date and I'm simply trying to combine them together. However I'm struggling to not name them dynamically.

clearvars
matFiles = dir('*.mat'); 
numfiles = length(matFiles);
mydata = cell(1, numfiles);

for k = 1:numfiles 
  mydata{k} = load(matFiles(k).name); 
end

This produces a 1x100 cell array, where each cell is a 1x1 struct containing a timetable. However when they are loaded in, the struct fieldname is taken from the filename which means they are all different and I cannot use vertcat. (I.e. I get 'Names of fields in structure arrays being concatenated do not match' error). Do I have to rename the structs and if so how or have I gone about this all wrong?

What is the correct way to process this sequence of files?

Kind regards,

Oliver

[Oliver Higbee]

I have also tried the wiki (https://matlab.fandom.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F) using the code below but get the following error. 'Subscripted assignment between dissimilar structures'.

% Get a list of all mat files in the current folder, or subfolders of it.
fds = fileDatastore('*.mat', 'ReadFcn', @importdata);
fullFileNames = fds.Files
numFiles = length(fullFileNames)

out = struct();
% Loop over all files reading them in and plotting them.
for k = 1 : numFiles
    fprintf('Now reading file %s\n', fullFileNames{k});
    out(k) = load(fullFileNames{k});
end

[Ive J]

I don't get it, do you have timetables or structs? Because they're not of the same class. The error you mentioned is for concatenation of structs and not tables. So assuming you have structs, one way would be to get rid of inconsistent field names and directly access to the content of that field:

V = ({});
for i = 1:n
    fname = fieldnames(mystruct);
    V{i} = mystruct.(fname{1}); % file has only one field
end
V = vertcat(V{:});

[Christopher McCausland]

Hi @Oliver Higbee,

I would suggest that you attach a couple of data files. This should make it a lot easier to help you.

Christopher

Answer 1

[Stephen23]

"Do I have to rename the structs..."

No, you do not need to.

But in future when designing data you should keep the structure names the same. Some beginners think that having different names is easier to work with... in fact when looping over multiple files using exactly the same variable name/s makes processing the data simpler, more efficient, and much more robust.

"What is the correct way to process this sequence of files? "

Assuming exactly one variable saved in each mat file:

P = 'absolute or relative path to where the files are saved';
S = dir(fullfile(P,'*.mat'));
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    C = struct2cell(load(F));
    S(k).data = C{1};
end
T = vertcat(S.data)

Answer 2

[Rik]

If you use struct2cell (either inside your loop or in a cellfun) you can extract the data itself. Then you should be able to use vertcat(data{:}).