How do MATLAB workers/PC cores divide the work in a parallelized optimization inside a parfor loop?

For example:

parfor i = 1:20
    options = optimoptions('ga','UseParallel',true,'UseVectorized',false);
    x = gamultiobj(ObjectiveFunction,[],options);
end

Let's say there are 10 workers, and we are running the optimization algorithm with parallelization within a parallelized parfor loop.

Although my sample is not large, I have noticed that this is faster than using a simple for loop.

According to MATLAB docs, a parfor inside a parfor does not work. Yet this combination (which in the end is a parfor inside a parfor, I guess) does work.

Thus, my questions are:

• How do workers divide the work? I notice that the first 10 loop cases are started at the same time, but do the workers then stop their loop iteration and help out whichever started the gamultiobj first?
• Assuming "n" workers, do the parfor and for approaches deliver the same performance when num of cases >> n?

Answer 1

[Raymond Norris]

Hi Taro,

Do you want to run gamultiobj 20 (in this example) times? If so, then parfor will run quicker than for. However, the parellel for loop you've written will negate the parallel for loop that is running in the optimization algorthim.

I suspect you simple need to call

    options = optimoptions('ga','UseParallel',true,'UseVectorized',false);
    x = gamultiobj(ObjectiveFunction,[],options);

And let the inner parallel for loop do its work.

Raymond

[Mitsu]

Hi Raymond, thanks for your answer.

I am indeed running many cases of the optimizer.

If the outer parfor loop negates the parfor(s) in the optimizer function, then that answers my question.

Regarding the second question at the end, though... in terms of total time to run all the cases, does it make more sense to use an outer parfor loop, or to use a normal for loop and make use of the parfor that lies inside the optimizer function?

[Raymond Norris]

Good question, Taro. Ideally, you want the outter loop to be given the most amount of work. Therefore, generally speaking, the outter loop would be re-written as a parfor. But take the following example

p = parpool(2);
parfor idx = 1:2
    do some work
    parfor jdx = 1:1000
        do some other work
    end
end
p.delete

If I'm working on a 8- or 16-core machine, I might re-write the inner for loop as a parallel for loop. That's because even with 8 or 16 cores, I'm only using 2 cores for the parallel pool (threading aside). It might make more sense then to dedicate the cores to the inner loop instead of the outer loop.

Best to run this in the Profiler to see how it works for you.

Raymond

[Mitsu]

Got it. I take that as a general rule, if the outer loop has more cases than number of workers, it makes sense to use an outer parfor loop; otherwise keep only the inner parfor (inside the optimization).

Thanks!