Run Matlab parallel on multi-nodes of a remote cluster with SGE scheduler

Hi,

I am using a local cluster and at the moment I am able to run matlab parallel codes on a single node (using parpool). However, the calculations are heavy and running them on a single node made lots of problem for other users. I want to distribute my workers over several nodes. The problem is we can not use the Matlab job scheduler and should use SGE (Sun Grid Engine) for scheduling jobs. I already defined a generic cluster and validated it. I used the instruction in this link and tried to run the Matlab code by qsub test.sh. But I got this error:

Error in communicatingsge (line 21)

submit(pjob);

Caused by:

Error using communicatingSubmitFcn (line 112)

Submit failed with the following message:

Unable to run job: denied: host "compute-0-0.local" is not a submit host

Exiting.

Could anyone help me please to solve this? unfortunatelly the support team are not familiar with Matlab and can not help me. I would appriciate if there are other suggestions regarding how to do this. If this is helpful I should say that the cluster has both MPI and OpenMP.

Thank you!

Answer 1

[Alvaro]

I cannot access the link you gave but this might work:

https://www.mathworks.com/matlabcentral/answers/326641-how-to-run-matlab-on-a-sun-grid-engine-cluster-to-use-parallel-optimization#answer_256132

In particular, look at the linked plugin and the comments in that answer.