AttachedFiles in parfor on cluster

Hi,

I am using MATLAB Distributed Computing Server (currently names MATLAB Parallel Server?) on four machines with a total of 64 workers. During the day, several processes will run, not simulataneously, on that cluster. Up until now we have been closing and opening the pool for every run, but due to some issues with opening the pool sometimes, we want to leave the pool running during the day and only reset them once a day.

I am a bit confused about the way AttachedFiles is used as argument in the parpool function.

• If AutoAddClientPath is true, does it make any sense to include any of the files on the path manually with the AttachedFiles argument?
• When I am running different exe's on this cluster, using the same pool for both processes, where one exe's is built from different m-files as the other, does the AutoAddClientPath deal with that correctly? Both exe's do have the same folders/files included in their path at time of compilation.

Hope the question makes any sense. If you need clarification, I would be happy to elaborate.

Best,

Robert

Answer 1

[Edric Ellis]

Firstly, AutoAddClientPath modifies the path on the workers to match the path on the client. This works only when the client and the workers can see the same code at the same location. (On Windows for example, this means that the code must be available on a UNC share). If the code is found via that mechanism, there's no need to attach it explictly.

As well as explicitly adding files using the AttachedFiles parameter, note that the parpool infrastructure attempts to attach files automatically when they are needed (when this happens, you'll see a message like "Analyzing and transferring files to the workers ...done"). You can see what has been attached using the listAutoAttachedFiles method of the pool, like so:

>> listAutoAttachedFiles(gcp())
Files automatically added to the parallel pool:
  pfeg.m

I don't know of any particular problems using the setup you describe.