MATLAB Answers

Is it possible to run .mex functions inside a threaded worker on a cluster?

9 views (last 30 days)
Nicole on 22 Sep 2016
Commented: Michelle Wu on 27 Sep 2016
I'm trying to run a script on a cluster using the 'parpool' function. One of the functions I try to call within the parfor command is in the form of a .mex file. I've tried attaching this file to the pool using:
parpool('local', 4, 'IdleTimeout', Inf,'AttachedFiles',{'svmtrainLS.mexw64','svmpredictLS.mexw64'})
The above successfully completes, and outputs:
Analyzing and transferring files to the workers ...done.
Then, when the function is called, I get an error stating that the function is unrecognized.
An UndefinedFunction error was thrown on the workers for 'svmtrainLS'. This might be because the file containing 'svmtrainLS' is
not accessible on the workers. Use addAttachedFiles(pool, files) to specify the required files to be attached. See the
documentation for 'parallel.Pool/addAttachedFiles' for more details.
Caused by:
Undefined function 'svmtrainLS' for input arguments of type 'double'.
I'm wondering if it's possible to run .mex functions inside a threaded worker on a cluster?


Edric Ellis
Edric Ellis on 23 Sep 2016
It certainly should be possible to do this. Usually with the 'local' cluster type, you don't need to transfer anything to the workers - providing they're on the path of the client. After you've opened the pool, could you try running:
which svmtrainLS
spmd, which svmtrainLS, end
This should show you first where the client is finding svmtrainLS, and then where each worker is finding it (or not). If the client cannot find svmtrainLS, then you need to call addpath there.
Michelle Wu
Michelle Wu on 27 Sep 2016
What you are trying to achieve is definitely possible.
The error message "Undefined function 'svmtrainLS' for input arguments of type 'double'." might indicate that there could be another MEX file with the same name but different arguments shadowing the MEX file that you need to transfer to the workers. So execute the command:
>> which -all svmtrainLS
and check if the actual MEX file is being shadowed by another file. If it is, move the actual MEX file ahead of the other file in the MATLAB path and see if that resolves the issue.
You can check if the location of the MEX file is accessible by the MATLAB client using the "which" command. If the file is located on a shared file system, you can move it to a local file system and add that path using the "addpath" command and see if the MATLAB client and the workers can access that file. If you encounter further issues, you can reach out to the Technical Support at MathWorks.

Sign in to comment.

Answers (0)

Translated by