Can't save workspace of 3 month calculation - URGENT

Hi,

I did some scientific simulation on a university server which took more than 3 months. The result worked out great. I wanted to save the workspace so I could continue analyzing on a local machine.... but I can't figure it out!!

I tried

save('data.mat');

which returned

Warning: Variable 'image' cannot be saved to a MAT-file whose
version is older than 7.3.
To save this variable, use the -v7.3 switch.
Skipping... 
Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ]
is not serializable 
Error using save
Can't write file C:\data.mat.

then I read about v7.3 option and tried it like this:

save('data.mat','-v7.3');

which gave me

Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ]
is not serializable 
Error using save
Can't write file C:\data.mat.

Now I'm at a loss. I really need this data and I can't affort to have to run the 105 days simulation again! Please help me save this workspace somehow. Estimated size is just one GB.

kind regards, Arnold

[Walter Roberson]

http://www.mathworks.com/matlabcentral/answers/29922-why-your-question-is-not-urgent-or-an-emergency

Answer 1

[arnold]

I found a solution. There were some variables which were of class 'java.awt'. I deleted them since they don't contain anything I need.... and now it works.

Panic-attack over :)

I wonder why I have to set the parameter 'v7.3' manually in the save command, why isn't it standard?

thanks

[per isakson]

"I wonder why I have to set the parameter 'v7.3'" &nbsp The answer is in the documentation.

[David]

where, can you provide a link? Why did you add this comment?

[Rodrigo Bezerra]

Here is a suggestion to change the MATLAB configuration to save the MAT-Files using version 7.3 without setting the parameter 'v7.3'. By default MATLAB uses version 7 to save MAT-Files.

https://www.mathworks.com/help/matlab/import_export/load-parts-of-variables-from-mat-files.html

"To change your preferences to save new files in Version 7.3 format, access the Environment section on the Home tab, and click Preferences. Select MATLAB > General > MAT-Files. This preference is not available in MATLAB Online™."

Answer 2

[Image Analyst]

Sounds like perhaps you unwisely chose to have a variable called "image", which is the same as a built in function name. Perhaps save() is getting confused as to which image you want to save. Try renaming that variable. Also try saving things more frequently than every 3 months. At least try saving it after the first iteration so that if something goes wrong you don't have to wait 105 days to find out.

It also sounds like maybe you're trying to save new version MATLAB things into an old version .mat file, which I guess has a different format. Try deleting the existing file first and then saving it so that it creates a brand new one.

Answer 3

[arnold]

Thanks for your response,

of course I usually save things more frequently, but the core code of this does not provide any possibility to do so, it is provided to our reasearch group by an external partner and all the temporary files are binary. :) Anyways. I renamed "image" to "image2" and tried to save the workspace again using:

save('data.mat','-v7.3');

and again:

Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ] is
not serializable 
Error using save
Can't write file C:\data.mat.

and no, there was no 'data.mat' in the root directory before. However, it always does write a file with some content. The variable 'image2' is not contained though. Checked via:

whos('-file','data.mat')

So renaming it did not solve the problem. Here's some more info on the 'image2' variable:

whos('image2')
  Name        Size                 Bytes  Class     Attributes

    image2      1x1             2997904144  struct

>> image2

image2 = 

                       visible: [4096x4096x3 uint8]
                       invalid: [4096x4096 logical]
                     heightmap: [4096x4096 double]
                         peaks: [4096x4096 logical]
                      confocal: [4096x4096 double]
                          mask: [4096x4096 double]
                       indices: [4096x4096 double]
                      pyramids: [4096x4096 double]
              indices_original: [5785x5785 double]
                       borders: [4096x4096 double]
                       sizemap: [4096x4096 double]
                   sizemap_tmp: [4096x4096 double]
               pyramid_density: [723x723 double]
        pyramids_superposition: [401x401 double]
                      distance: [4096x4096 single]
                   sizemap_rgb: [4096x4096x3 double]
               sizemap_overlay: [4096x4096x3 uint8]
          sizemap_diameter_rgb: [4096x4096x3 double]
      sizemap_diameter_overlay: [4096x4096x3 uint8]
                     peaks_new: [5785x5785 double]
                       valleys: [5785x5785 double]
                   fft_borders: [2705x2705 double]

Strangely enough, trying to save 'image2' on its own seems to work:

>> save('image2.mat','image2','-v7.3')
>> whos('-file','image2.mat')
  Name        Size                 Bytes  Class     Attributes

    image2      1x1             2997904144  struct

I'd feel better having a way to properly save the entire workspace though.

Regards

Arnold

[per isakson]

"Strangely enough" &nbsp why strange? The warning indicated that it should work with the -v7.3 switch: "To save this variable, use the -v7.3 switch."

"Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ] is not serializable" &nbsp is most likely a different problem. Did you use java.awt.Robot? Could there be an instance of that Java class in the work space?

Proposal: Save a few variables at a time until you spotted the variable that causes the problem.

[arnold]

I have solved it by doing exactly that. there were several variables of class 'java.awt.Robot'. Cleared them and it worked. I'll can't fully check the code since part of it is pcode.

I find it strange that the standard behavior for the save command is not the most general one (allowing for large variables to be saved), why would it be one which is less general? So from now on to all times we should have to use the -v7.3 extension in order to save our ever growing workspaces? I find that counterintuitive and would much rather have an optional 32bit switch or something.

Answer 4

[arnold]

ehhh... why is any *.mat I save with '-v7.3' WAY bigger than without that option?

[Walter Roberson]

-v7.3 saves in HDF files, a standard format that has more overhead than the older format.

My understanding is that HDF files can sometimes have compression for components, but I do not know the circumstances under which that applies. See the discussion here and note that MATLAB might have changed which compression it does automatically since that time.

Answer 5

[David]

So... I'm getting this same error, trying to save a struct variable with no ROBOTS in it. I'm surprised that no one can answer the actual question here, which is why version 7.3 has anything to do with saving a brand new mat file with a brand new version of matlab.

Sadly, my 90 hour data run already closed, and all my attempts at intermediate saves errored out with this same -v7.3 garbage.

I don't want any lectures about intermediate saves, or how I should've made sure it was working before a 90 hour run. Just an answer to the question please.

[per isakson]

• "this same error" &nbsp Please describe the error in some detail. I don't understand what you refer to.
• "same -v7.3 garbage" &nbsp What do you mean?
• "surprised that no one can answer the actual question" &nbsp What is the actual question?

[Walter Roberson]

You need -v7.3 to save any variable that contains an object, or to save a variable whose storage size exceeds 2 Gb.

What are the class() of the elements of your struct variable?

As to why -v7.3 is not the default: -v7.3 results in considerably larger files, and the access to data can be quite a bit slower. In early implementations, there were reports of save() requiring 2+ hours for data that took about 5 minutes to save() with -v7 . Performance improvements have been made, but the time and storage are still a fair bit more.

Most people do not need the extended facilities at all, and even then not most of the time. If a facility is only needed (say) 5% of the time, then does it make sense to penalize performance notably the other 95% of the time?

The people who do need the facility regularly, the people who routinely run with large data sets or with saving objects: those people test their code.

Answer 6

[Akhilesh Thakur]

Warning: java.awt.Robot[ autoDelay = 0, autoWaitForIdle = false ] is not serializable

Error using save

Can't write file

These are the errors I got today. But even using v7.3 as it says use for larger files > 2GB this didn't do any effect. I found out that there are timing issues with save. If you are calling save option again and again to update your file, there is an internal error of saving it. Even I am running a huge program that gives me result after 3 days. I found out i was using save command after every iteration of my program. This won't work. I added a functionality where I can manually save the file using flag or keypress. I realised that save command failed after calling it 19 times. I dont know what happens exactly when it is exactly called at 19th time. But the point is whenever you want to save file use a mechanism that will use this command very less frequently. So even if someone wants to save the file and the program is running, You can do a keypress So i did if( handles.keypress == 'p') save('results.mat');

So the conclusion is avoid calling save multiple times. This method worked for me.

Hope this helps.

[Akhilesh Thakur]

One more change

I checked only saving the data and filtering out other data such as small variables and handles, so I am only saving required variables in the result like this

save example.mat results

This works for me and never fails. Try filtering out only data which you want. Without handles and other info it is working. I guess it's handles which is messing with java.awt. This is the way where save worked for me for 192 hours of continuous calculations.

Hope this works for everyone. I spent almost 4 days debugging this.

Answer 7

[Mech Princess]

So the problem is that there are some variables that are communicating with java. So once it comes to that variable, it stops saving and gives the error

not serializable 
Error using save

My workspace has a lot of variables both serializable and not. Does anyone know how to save ignoring the variables not serializable without having to delete it manually or name them one by one?

Thanks