The assignment of workers to cores

Hello, we are planning to buy an MDSC of 128 workers. In running a parallel job of 128 workers, what is the minimum number of cores should there be in the hardware platform that I am going to use? How many workers can I assign to each core of the machine at the maximum ? Would appreciate your answer. Thanks

Answer 1

[Kojiro Saito]

It is possible to run more workers than the number of CPU cores, for example, 128 workers on 1 CPU core, but it would not make the performance better.

Please see this MDCS System Requirments. It says "Maximum of 1 MATLAB worker per CPU core is recommended." So, assume you want to obtain 128 MDCS workers, recommended CPU cores are more than 128.

Answer 2

[Halil Eyyuboglu]

Dear Saito ; Thanks for the reply, there is also the thread issue (mostly two threads per core). The people who sell workstations say that since the threads within the core are independent physical entities, it should be possible to assign one worker to a thread or alternatively two workers to a core (without degrading the performance). I would like to receive your kind comment on this issue as well. Thanks.

[Kojiro Saito]

Yes, it is possible to assign two workers to one core. Also, you can tune the number of threads on each worker. For example, assigning 2 threads on one worker and each worker to each core would fully utilize two threads per core of your future machines.

Here is a screen shot of NumThreads configuration. By default it is set to 1.

But as Note in this doc says, setting NumWorkers x NumThreads larger than CPU cores might not improve the performance. So, I recommend you to get a trial of MDCS from here.

[wonderkismet]

How to do it programmaly?

[Kojiro Saito]

You can change NumThreads by editing parcluster object. Here is an example.

c = parcluster('myMjsProfile'); % myMjsProfile is a cluster profile name

% Change NumThreads from 1 to 2
c.NumThreads = 2;

% Optional. You can save the cluster profile
saveProfile(c)

% Open parpool using this cluster profile
p = parpool(c);

% Do parfor job
parfor n=1:c.NumWorkers
% ...
% ...
end