How to load multiple *.mat files in a directory into individual 271x1 vectors. Not structures. Need them named dynamically. Don't crucify me.

Ok, so yes, I know, DONT USE DYNAMIC VARIABLES!!!!

I totally understand why this would not be good in functions and scripts and all that. In fact, if I try to even think about implementing something like that, my head hurts and says "nope, just stop".

However in this situation I need dynamic variables, unless someone else can think of a better way.

I am processing field data from gas concentration measurements. The important data I need to look at is the concentration curve. For my two day survey I have say one vector that's about 40000x1 in size. This gets broken down into 300x1 pieces for the observations at different sites. So, I end up with about 72 individual 300x1 vectors. Each comes from a different location (eg ccf1 ccf1 cc1 cc2 st1 st2 etc etc etc etc). I have written each 300x1 vector to a *.mat file named after the site (eg ccf12.mat) .

Now, I want to look at each using cftool and play with curve fittings. However, cftool only takes in vectors. You can't access structures or tables (otherwise I would def go that route). SO I am trying to import all the *.mat files in the directory and name the output variables accordingly. However they keep ending up being structures. I'm just using a simple loop:

a=dir('*mat');
for i = 1:length(a)
    b=load(a(i).name);
    eval([char(a(i).name) '= [b.co2_umolm3(:)]']) %%%trying to not have it output a structure by calling the variable itself
   clear b
end

All the variables are loaded, with their fancy dynamic names. But all are structures with a variable mat, like cc14.mat would be the data. Then if I want to add to the loop a step to try and change the structure to a vector I encounter the annoying aspect of dynamic variables in that I have to continue to use eval.

This isn't something that will be in a code or used often. It's just for some data exploration and curve fitting testing. Hence I am not concerned about using dynamic variables. But this structure thing is driving me nutts. Any suggestions (Aside from DONT USE DYNAMIC VARIABLES).

Cheers

[Stephen23]

But why do you need every vector to have a different name? cftool simply accepts x, y, (z, w) inputs, and presumably you are only going to call it on one set of data at once, in which case what is stopping you from simply assigning whichever set of data you currently want to fit to the same variables:

x = use indexing/fieldname to select imported data from cell/structure/whatever array
y = ditto
cftool(x,y)

Why do you want to make it much more complex than that?

"However, cftool only takes in vectors."

Which can be stored in the fields of a structure or the cells of a cell array, or in a table or in an ND array, etc.

So far everything you have described could be achieved using basic indexing and/or fieldnames, Don't get too fixated on one approach: take a step back and think about what you are actually trying to achieve here, and then ask about that: http://xyproblem.info/

[Walter Roberson]

cftool only accepts simple variables for x when it populates the variable name list, but you can pass values to cftool when you invoke it.

[Jordan Mertes]

That's a fair point.

[Jordan Mertes]

Ultimately, Stephen I think you are right. The amount of time I've already spent trying to get this to work could have just been used doing what you suggest.

Another example of why dynamic variables are daft.

Thanks for the input.

Answer 1

[Steven Lord]

How do you distinguish the X and Y data stored in each MAT-file? Or do you have a 271-by-N variable stored in each MAT-file, with one of the columns being your Y data and the rest being the X data?

>> data = load('census.mat')

data = 

  struct with fields:

    cdate: [21×1 double]
      pop: [21×1 double]

>> xdata = data.cdate;
>> ydata = data.pop;
% Now work with xdata and ydata

If you don't know the names of your X and Y data variables but can distinguish them when you see them:

data = load('census.mat');
variablesFromFile = fieldnames(data);
for whichVariable = 1:numel(variablesFromFile)
    name = variablesFromFile{whichVariable};
    V = data.(name);
    if isXData(V) % You need to write this, maybe with name being an additional input
        xdata = V;
    elseif isYData(V) % You need to write this
        ydata = V;
    else
        % how do you want to handle the other data in the file? Ignore it?
    end
end
% Now work with xdata and ydata

Answer 2

[Mathieu NOE]

Hi

though I am not really an expert in this kind of field, I did long time ago something in that area . don't know, but maybe in my code below you would see how to proceed

Good luck !

% eval(['load ' Filenameshort '.MAT']);
MyStruct = load([Filenameshort '.MAT']);

fn  = fieldnames(MyStruct);
VariableName = char(fn);

%%%%%%%%%%

% % si le nom du fichier .mat n'a pas été renommé apres acquisition alors le nom de la structure est idem nom du fichier
% if isempty(VariableName)
% 	VariableName = lower(Filenameshort);
% end
% 
% Initialisation
    VarName = [];
    ind_comma = [];
    ind_bracket = [];
    ind_value = [];
    

% Axe temporel
%time = long_capture.X.Data;
time = eval(['MyStruct.' VariableName '.X.Data']); % temps en secondes
nsamples = length(time);

% capture des data
commandeIM = zeros(nb_outputs*nb_harmo,nsamples);
commandeRE = zeros(nb_outputs*nb_harmo,nsamples);
residuelIM = zeros(nb_inputs*nb_harmo,nsamples);
residuelRE = zeros(nb_inputs*nb_harmo,nsamples);
freq = zeros(1,nsamples);

% calcul du nombre de variables
nb_var = length(eval(['MyStruct.' VariableName '.Y']));

for ci = 1:nb_var
    VarName = eval(['MyStruct.' VariableName '.Y(' num2str(ci) ').Name']);
    ind_bracket1 = findstr(VarName ,'[');   % détetction bracket "ouverture"
    ind_comma = findstr(VarName ,',');      % détetction virgule
    ind_bracket2 = findstr(VarName ,']');   % détetction bracket "fermeture"
    ind_value1 = str2num(VarName(1,ind_bracket1+1:ind_comma-1));
    % cas particulier nb_inputs = 1 alors ind_value1 =  [], à remplacer par 1
    if isempty(ind_value1), ind_value1 =1;end
    
    ind_value2 = str2num(VarName(1,ind_comma+1:ind_bracket2-1));
    % cas particulier nb_outputs = 1 alors ind_value2 =  [], à remplacer par 1
    if isempty(ind_value2), ind_value2 =1;end
    
    ind_value = max(ind_value1,ind_value2); % ex [1,10] ou [10,1] => 10
%     ind_value_table1 = [ind_value_table1; ind_value];% debug only
    
    if ~isempty(findstr(VarName,'commandeIM'))
        commandeIM(ind_value,:) = eval(['MyStruct.' VariableName '.Y(' num2str(ci) ').Data']);
    elseif ~isempty(findstr(VarName,'commandeRE'))
        commandeRE(ind_value,:) = eval(['MyStruct.' VariableName '.Y(' num2str(ci) ').Data']);
    elseif ~isempty(findstr(VarName,'freq'))
        freq =  eval(['MyStruct.' VariableName '.Y(' num2str(ci) ').Data']);
    elseif ~isempty(findstr(VarName,'residuelIM'))
        residuelIM(ind_value,:) = eval(['MyStruct.' VariableName '.Y(' num2str(ci) ').Data']);
    elseif ~isempty(findstr(VarName,'residuelRE'))
        residuelRE(ind_value,:) = eval(['MyStruct.' VariableName '.Y(' num2str(ci) ').Data']);
    end

end

residuel = residuelRE+j*residuelIM;
commande = commandeRE+j*commandeIM;
freq = double(freq);

[Jordan Mertes]

Thans for the script. I will have a play with this.