why does parfor execute loops in a random order?

I can't see any reason why, and if my PC crashes during a 20,000 loop iteration, it's a pain to figure out which iterations are still to be done.

Cheers Mike

Answer 1

[Jason Ross]

Just to be clear -- parfor loops are of independent of iteration order, and do not guarantee deterministic results. It's not random.

The functional reason for this is if iteration 2 finishes before iteration 1 then iteration 3 can start work, and so on, since each iteration is independent of the other.

[Michael]

I don't understand why they can't just go 1,2,3,4,...? If they are independent then what prevents this? I've got 8 workers and the first one they try is iteration 49 or something.

[Walter Roberson]

The choice of order for parfor() loops is best treated as being dependent on the exact version being used, and upon the number of iterations, and potentially upon information gathered from analyzing the loop. It is left up to MATLAB to do them in any order it feels is most efficient.

There are cases in which the most efficient order is to give each worker a "chunk" of iterations, with the chunk separation chosen so that the different iterations are not hitting the same cache line in the scheduling processor's memory. Whether or not parfor currently takes that level of detail into account, it could.

Different processors will potentially operate at slightly different speeds, or might get interrupted differently for devices they control, or might have "better luck" as to which parts of memory are in the operating system disk cache. So the order that iterations finish in is indeterminate, and the order of queueing for more work is indeterminate.

You should not be manually trying to figure out which iterations are still to be done: you should be building a "checkpoint" system.

[Michael]

Thanks.

I suppose a vector of zeros (to be switched to 1's on completion) would automatically register which iterations are completed. It's not something I thought about before!

[Jason Ross]

You might also want to investigate using the job/task interface, which gives you more control over the iterations and tracking down errors. A simple example:

c=parcluster();
alloutputs = [];
for ii=1:10
    job = c.createJob;
    for jj=1:10
        createTask(job,@rand, 1, {3,3});
    end
    job.submit;
    job.wait;
    outputs = job.fetchOutputs;
    alloutputs = [alloutputs ; outputs];
    disp(ii);
    job.destroy;
end

Note that things like preallocation, fancy formatting and error checking have been left out. Things that will likely be interesting to you:

• The job object contains the Task ID of errors. You could use this information to know what failed.
• You will likely need to figure out what work for you in terms of iteration display, number of tasks per job, dealing with the returns, etc.