How to set up a Matlab parallel cluster for thread-based environment
11 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Maria
am 16 Jun. 2021
Kommentiert: Raymond Norris
am 16 Jun. 2021
Hi,
I am starting to explore Matlab Parallel functionalities, and, I have to say, I am a bit confused about the process-based vs. thread-based environment.
First question: I have 2 clusters, namely, the local cluster and the "MatlabCluster" (remote cluster with 8 nodes, 32 workers). If I use
poop = parpool('MatlabCluster');
the default environment is the "process-based" environment. Correct? Can I use the remote cluster in a "thread-based" environment? If I do
pool = parpool('thread');
only the local cluster switches to 'thread'. Can I do the same with the remote cluster?
Second question: I am experimenting with distributed arrays. However, if I start the 'MatlabCluster' (remote cluster), I get few errors and the last error message is
No workers are available for FevalQueue execution
This happens for the line of code that uses distrubuted arrays. I read that FevalQueue is not supported in "thread-based environment". Does this error mean that, by default, the remote cluster is starting as "thread-based"? (which would contradict my first hypotesis?).
0 Kommentare
Akzeptierte Antwort
Raymond Norris
am 16 Jun. 2021
The thread-based pool only runs on the same machine as the MATLAB client, similar to a local process-based pool. However, unlike the local pool, the threaded pool has a fixed startup size, which is the value returned by maxNumCompThreads. If you wanted a different number of workers started with a threads pool, you have to set it first in maxNumCompThreads. For example:
% Let's assume you have 8 physical cores, but only want to start a threaded
% pool of 2 workers.
old_threads = maxNumCompThreads(2);
parpool("threads");
Keep in mind that setting maxNumCompThreads, in addition to effecting the number of workers started, may have an effect on your other MATLAB code.
You'll need to post a bit more (code, errors) to decipher the FevalQueue error.
2 Kommentare
Raymond Norris
am 16 Jun. 2021
To get the memory on the Linux nodes, run
free -mth
This will give you the free & used memory.
Weitere Antworten (0)
Siehe auch
Kategorien
Mehr zu Parallel Computing Fundamentals finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!