Worker unable to find mex file

Hi,

I am making some tests with Matlab Parallel Server. I am trying the interactive mode, and I have a function with a parfor. Inside this parfor, there is a mex file. The remote cluster I use has Linux, and the mex files have been created in Linux. The client where I am running Matlab is Windows.

When I try running the file, I get the following error:

 Worker unable to find file.
 

and the name of the mex file. I was reading that the best is to have a networked file share which the worker machines can access. I have placed the mex file in this networked file share, but I get the same error. When I tried running mex files in a parfor directly under Linux, I did not have this problem. Do I need to do something specific when I am running from a Windows client?

Answer 1

[Raymond Norris]

I'm assuming you are using parpool? How are you setting the worker's path on the Linux cluster to find the MEX-file? One option is to use pctRunOnAll.

[Maria]

Hi,

sorry for my late reply, I simplified the code to minimize the data transfer.

I am now trying the batch mode. I get a similar error even if I use the local cluster.

Error: The parallel job was cancelled because the task with ID 1 terminated abnormally for the following reason:

The source code (/var/lib/mjs/cluster-node-01.matrix.local_worker1_mlworker_log/matlab/filedependencies/3/a/tpbda8df2e_f22f_4d32_b58c_e978d377f047/internal_compute_H_9int_sym.m) for the parfor-loop that is trying to execute on the worker could not be found.

[Raymond Norris]

Putting the pieces together (from your other posts)

c = parcluster('MatlabCluster');
job = batch(c,@compute_H_matrix,1,{large data inputs},'Pool',31);

Nested in compute_H_matrix is a call to internal_compute_H_9int_sym, correct?

Is internal_compute_H_9int_sym called directly? Or is it possibly that internal_compute_H_9int_sym is referenced either via a function pointer or eval statement? For example

fcn = str2func('internal_compute_H_9int_sym');
feval(fcn)
eval('internal_compute_H_9int_sym')

In both cases (certainly with eval), MATLAB doesn't know to attach internal_compute_H_9int_sym to the batch job. If you have cases like this, you'll need to explicilty add internal_compute_H_9int_sym to the batch job. See the batch doc, but here's a simple example

job = batch(c,@compute_H_matrix,1,{large data inputs},'Pool',31,'AttachedFiles',{'internal_compute_H_9int_sym'});

[Maria]

Thank you for your help.

All functions are called directly.

I could run the batch mode on the local cluster by adding this

addpath C:\Users\mardel\Documents\folder2Add

and it worked.

However, on the remote cluster, it does not work. I tried with

addpath /data_matlab/maria/folder2Add

where I copied everything on a shared folder, but I get the same error. I also tried adding the main function as you suggested, but that file could be found either.

[Maria]

We are digging into the folders of the nodes, and it seems that, automatically, the necessary files are copied into the var/lib/etcetc folder of the nodes (no matter how I use the addpath or 'AdditionalPaths', the nodes keep on looking in this var/lib/... folder).

All necessary files are copied, but not the mex files. If I do not use the mex files but the original Matlab files, then is fine.

[Maria]

So, fixed it!

I did like this

job1 = batch(c,@myFunction,1,{inputs },'Pool',N-1,'CurrentFolder','.','AutoAddClientPath',false,'AdditionalPaths',ap,'AttachedFiles',{'file1.mexa64' 'file2.mexa64'});

The fields

'CurrentFolder','.','AutoAddClientPath',false,

Are there because the client is windows but the cluster is in Linux, I wanted to be sure that the paths were not messed up.

The 'AdditionalPaths' uses

ap = {'/data_matlab/maria/myfolder/'};

And this

'AttachedFiles',{'file1.mexa64' 'file2.mexa64'}

solved the issue with the mex.

[Raymond Norris]

Correct. That's because on your client, the client is looking for and attaching file1.dll. It's unaware that MATLAB will be running on Linux. You're approach to attach the mexa64 is the right way.