Parfor Loop Error Using imwrite

Hi there,

I am trying to use a parfor loop which includes code which generates a RGB image and then saves it using the "imwrite" function within the parfor loop. It runs for a certain period of time and then randomly gives me the following error:

"PNG library failed: Could not open file..

Error in imwrite (line 566)

feval(fmt_s.write, data, map, filename, paramPairs{:});

Error in HEATMAP_SCRIPT (line 65)

parfor i = 2:N"

The error seems to occur at random, sometimes occuring at 20% completion of the parfor loop or sometimes 30% etc. Any suggestions as to why this may be happening and how I can get around this?

Thanks

[Walter Roberson]

Is it possible that you are writing to the same file in different workers? And that you are using MS Windows, which automatically locks files against multiple writing?

[Sanjeev Uthishtran]

I am using MS Windows, however I am writing to separate files as seen below:

"FinImage" is the RGB image I want to save and each worker is creating one of these images for different timepoints in a timeseries where the timepoint is specified by "i".

"imwrite(FinImage,output_folder+"\Frame"+num2str(i)+".png")"

[Venkat Siddarth Reddy]

Can you share the code to better understand the scenario?

[Walter Roberson]

The known causes of that problem are:

• having two processes (or threads) writing to the exact same file name at the same time.
• running out of open file descriptors within one process, because the process happens to want to write to more simultaneous files than there are available descriptors
• running out of open file descriptors within one process, because the program "leaks" open files (fails to close files after it finishes with them)

We can rule out the first two possibilities in your particular situation. At the moment we cannot rule out the possibility that imwrite() is "leaking" open files.

[Sanjeev Uthishtran]

Hi Walter,

You are correct. As far as I understand, my problem aligns with the third point you mentioned.

If I understand correctly, the "imwrite" function in some cases is not able to close the file. Is there any way to get the file descriptor while using "imwrite" so I can forcefully close the file as soon as it has finished being written.

Thanks

[Walter Roberson]

Within the parfor process you could try

fclose('all')

Or you could use

fopen('all')

and fish through the return values to determine the files to fclose

[Sanjeev Uthishtran]

I still get the same error unfortunately. It seems to be occuring at random times.

Answer 1

[Shivani]

Hello @Sanjeev Uthishtran

From the context provided, it looks like the error encountered is due to an issue with file access or creation. This problem can arise due to several reasons, and its seemingly random occurrence might be influenced by factors such as file naming, directory access permissions, or system-specific limitations.

You can try out the following troubleshooting steps to resolve this issue:

• Ensure each iteration of the parfor loop generates a unique file name for the PNG files. If two iterations attempt to write to the same file simultaneously, it could cause the error you're seeing.
• Verify that the MATLAB process has write permissions for the directory where you're attempting to save the PNG files.
• Check if there's sufficient disk space where you're trying to save the images. Running out of disk space during the parfor loop execution could lead to file creation failures.
• Some filesystems have limitations on the number of files in a directory or the total volume of data. Ensure you're not hitting such a limit, which could vary depending on your operating system and filesystem configuration.