How to efficiently write complex structures to a file using memmapfile?

I am writing a code that reads complex structures from files using memmapfile, then you can modify the data, and last you write the data again to disk with memmapfile. I am struggling with the last part.

As a example, I read the first part of a file using:

filename = 'original.bin';
dataFormat = {'uint32',1,"Source_GRIB_code";
    'uint32',1,"Target_GRIB_code"};
Nrepeat = 100;
offset = 0;
m = memmapfile(filename, 'Format', dataFormat, 'offset', offset, 'Repeat', Nrepeat);
tdata = m.Data;

tdata is a 100×1 struct array with fields:

Source_GRIB_code

Target_GRIB_code

After I modify tdata, I want to write it to another file:

filename = 'modified.bin';
dataFormat = {'uint32',1,"Source_GRIB_code";
    'uint32',1,"Target_GRIB_code"};
Nrepeat = 100;
offset = 0;
m2 = memmapfile(filename, 'Format', dataFormat, 'offset', offset, 'Repeat', Nrepeat, 'Writable', true);

I expected to be able to execute:

m2.Data = modified_tdata;

but I get:

Error using memmapfile/hParseStructDataSubsasgn
When the Data field of a memmapfile object is a structure, it may not be replaced by assignment.

Error in memmapfile/hDoDataSubsasgn (line 492)
            [valid, s, newval] = hParseStructDataSubsasgn(obj, s, newval);

Error in indexing (line 749)
                hDoDataSubsasgn(obj, s, newval);
 

I can not even do:

m2.Data(1) = modified_tdata(1);

I get the same error.

I also tried :

ttt = num2cell(uint32(vertcat(modified_tdata.Source_GRIB_code)));
[m2.Data.Source_GRIB_code] = ttt{:};

But I get:

Error using indexing
A subscripting operation on the Data field attempted to create a comma-separated list. The memmapfile class does not support the use of comma-separated lists
when subscripting.

Since it is going to be really slow, I want to avoid doing a for loop that does something like:

for ii=1:Nrepeat
    m2.Data(ii).Source_GRIB_code = modified_tdata(ii).Source_GRIB_code;
    m2.Data(ii).Target_GRIB_code = modified_tdata(ii).Target_GRIB_code;
end

Is there an alternative to the for loop?

Thank you

Answer 1

[Raul Onrubia]

I ended up using fwrite. I reordered the data and casted it to uint8, and then writting it to disk using fwrite. I also tried the same approach but using memmapfile to store the uint8 data instead fwrite, but it was slower.

Answer 2

[Srija Kethiri]

Hi Raul,

I understand that you want to write the modified data from one memory mapped file to another.

The reason for the following error is “Error using memmapfile/hParseStructDataSubsasgn When the Data field of a memmapfile object is a structure, it may not be replaced by assignment.”

After the creation of “memmapfile” object, it doesn’t support changing the value of its “Data” property.

The reason for the following error is “Error using indexing. A subscripting operation on the Data field attempted to create a comma-separated list. The memmapfile class does not support the use of comma-separated lists when subscripting.” is,

In “memmapfile” object accessing a nonscalar structure array as “m.Data.fieldname” is not supported. A nonscalar structure array can only be accessed as m.Data(index).fieldname.

The alternative way of writing the data into “memmapfile” is, you can modify the data in the “modified.bin” file and then memory map that file.

To know more about “memmapfile” function, refer to the following documentation:

https://in.mathworks.com/help/matlab/ref/memmapfile.html

Hope this helps!

[Raul Onrubia]

Thanks. The memmapfile documentation for writing is not quite clear, I must say.