Why is saving a large matrix to a MAT file very slow in MATLAB 7.7 (R2008b)?

I have a single huge (4 GB) variable of type double. I want to save it to a MAT file and reload it at many different places in my application to use it. I am experiencing that it takes a very long time to write it to a MAT file.

Here is what I am trying to do:

%---------USING SAVE/LOAD---------

>> a=rand(5e9/8,1);

>> tic,save -v7.3 largefile.mat a,toc

Elapsed time is 370.750804 seconds.

>> clear a

>> tic,load largefile.mat,toc

Elapsed time is 55.672785 seconds.

When I compare this with FWRITE I get significantly improved performance.

%--------USING FWRITE/FREAD----------

>> tic, fid=fopen('bigfile','wb'), count=fwrite(fid,a,'double'),fclose(fid),toc

fid =

       3

count =

     625000000

Elapsed time is 68.695837 seconds.

>>  tic, clear a, fid=fopen('bigfile','rb'), a=fread(fid,'double');

fclose(fid), toc

fid =

       3

Elapsed time is 16.190452 seconds.

Answer 1

[MathWorks Support Team]

Saving to a MAT file can be slower than saving to a raw bit binary file.

This is because MAT file is a structured and compressed file format. Structured means that (in comparison to a "raw" stream of bits) the MAT file describes its own contents (e.g. whos -file), allows growth (e.g. adding other variables), access to sub-parts (e.g. load one variable out of many), names its data (e.g. so LOAD can restore a workspace) and other features supported by the MAT API. Compression means you can trade (file) space against time. All these tasks add overhead while saving to a MAT file as compared to writing to a raw binary file.

If none of these features are important, then you can certainly get much better performance by just streaming the raw bits to a binary file using FWRITE.