Modify invalid field names in .mat file using code

Hi, I have exported a .mat file from a program called Wintax4 and some of the field names are starting with _xxx which is not allowed by .mat file. This means I can't load my .mat file using data = load(data.mat) because it throws an error for invalid field name. How can I go about automatically going through this .mat file and just removing the underscore from all field names that start with underscore?

An example when I use data = Load('data.mat') I get the following error

"Error using load

Invalid field name: '_ADU_Odometer'."

There are about 10-15 field names using this underscore as the first letter and I have tons of these datalogs so manually editing them is not an option.

Thanks in advance

[Dyuman Joshi]

Please attach some of the .mat files. Use the paperclip button to do so.

[Andreas Zetterlund]

The file is too big to upload to compress it.

Answer 1

[dpb]

unzip ExampleFile
dir

.                ..               ExampleFile.mat  ExampleFile.zip  

whos -file ExampleFile _*

  Name                              Size              Bytes  Class     Attributes

  _ADU_Odometer                 26087x1              208696  double              
  _ADU_ResettableOdometer       26087x1              208696  double              
  _ADU_TrackBeacon             260861x1             2086888  double              
  _ADU_TrackDistance            52173x1              417384  double              
  _ADU_TrackLap                 52173x1              417384  double              
  _ADU_TrackLapTime            260861x1             2086888  double              
  _GPS_Latitude                 52173x1              417384  double              
  _GPS_Longitude                52173x1              417384  double              
  _GPS_Speed                    52173x1              417384  double              

%S=load('ExampleFile') % _ADU_Odometer
fid=fopen('ExampleFile.mat','r');
[filename,~,~,encoding] = fopen(fid);
f=fread(fid,'*char*1').';
%whos f
%ix=strfind(f,'_ADU_')
%f(ix(1):ix(1)+20)
f=strrep(f,'_ADU_','xADU_');
%f(ix(1):ix(1)+20)
fid=fclose(fid);
fid=fopen('NewFile.mat','w','n',encoding);
fwrite(fid,f);
fid=fclose(fid);
whos -file NewFile x*

  Name                              Size              Bytes  Class     Attributes

  xADU_Odometer                 26087x1              208696  double              
  xADU_ResettableOdometer       26087x1              208696  double              
  xADU_TrackBeacon             260861x1             2086888  double              
  xADU_TrackDistance            52173x1              417384  double              
  xADU_TrackLap                 52173x1              417384  double              
  xADU_TrackLapTime            260861x1             2086888  double              

S=load('NewFile','xA*')

S = struct with fields:
              xADU_Odometer: [26087×1 double]
    xADU_ResettableOdometer: [26087×1 double]
           xADU_TrackBeacon: [260861×1 double]
         xADU_TrackDistance: [52173×1 double]
              xADU_TrackLap: [52173×1 double]
          xADU_TrackLapTime: [260861×1 double]

Well, it does seem to work! I wasn't sure if there were some other details in the header that might have to be munged on as well besides just changing the names.

You'll have to do the other variables besides _ADU_*, too, of course, before you write the new one out; I was just seeing if it would work or not.

[Andreas Zetterlund]

Thank you for your reply. If I would like to try replace both the _ADU and _GPS how would I do this? I tried adding just "f=strrep(f,'_GPS_','GPS_');" underneeth but this makes the matlab file corrupt.

[Walter Roberson]

You have to replace with exactly the same size for the file to remain valid. For example

f=strrep(f,'_GPS_','GPS__');

[James Tursa]

True. Follow-up, the reason for this is that all of the variable and field name lengths are embedded within the file. Technically, if one could find where the length for a particular name is recorded in the file you could change it and then change the name's length in your rewrite, but it is much easier to just keep the length the same.

[dpb]

@Walter Roberson already pointed out the problem, I'll just note the difference in

f=strrep(f,'_GPS_','GPS_');

and what I did above in

f=strrep(f,'_ADU_','xADU_');

is that I simply substituted an x in place of the leading underscore; I didn't remove it. That would ensure no two variable names would end up the same and let one have a decent shot of finding all that had been modified by searching for the leading 'x' to be able to check had a set of names wanted...they could then be turned into a set of "real" variables as desired for actual use.

Answer 2

[Walter Roberson]

Use @James Tursa https://www.mathworks.com/matlabcentral/fileexchange/42274-loadfixnames-loads-a-mat-file-fixing-invalid-names?s_tid=srchtitle which is designed for these kind of cases.

[James Tursa]

That FEX submission is obsolete because MATLAB changed the mex-mat interface some years ago. It used to be it would load whatever name was in the file, then it could be fixed in the mex routine before sending it on to MATLAB. But the interface now checks the name for validity and will not even load the file if it finds a problem. Rewriting the file ala dpb is the only way I know how to do this with later versions of MATLAB, and that would only work as long as the file is not compressed I would guess.

[Walter Roberson]

I tried it earlier this year and it seemed to work, except that for the particular file I was using it on, it told me that there were too many variables to fix up.

On that particular file, I found that the valid variablenames out of it would load properly and without message if I used load as a command, but if I used load as a function it would warn about invalid names.

It might make a difference that it was struct field names rather than the names of the variables themselves.

[dpb]

I guess that would depend upon whether it compresses the content of the whole file or just compresses the data within each variable before writing it.