How much parallelism can I get out of 32 workers?

I'm looking at setting up the Matlab Parallel Server on a cluster I work with. My institution has a 32-worker license. The cluster has 40 CPU cores per node.

How much parallelism and performance can I get out of such a setup? If I ran one worker per CPU core then I won't even be able to saturate a node, rendering the parallel server pointless as there will be no inter-node communication. If I ran one or two workers per node, is it possible to use multiple threads on each worker? If so, what parallel constructs will and won't work with this setup?

Many thanks in advance.

Answer 1

[Edric Ellis]

If your workloads are amenable to MATLAB's intrinsic multi-threading, then you could run 32 multithreaded workers, each of which has 40 CPU threads. To set this up, you'd need to set up the NumThreads property of your cluster profile. Something like this:

c = parcluster('myCluster')
c.NumThreads = 40;
parpool(c)
parfor ii = 1:10000
    out(ii) = max(max(rand(1000) * rand(1000))); % or something.
end

Of course, the other side of this is to ensure that your cluster is correctly configured to launch one worker process per host. The details of that depend heavily on your cluster (e.g. is it running MJS, SLURM, ...).