parallelization during parallelization in recursive function
8 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Everyone good time of day!
Not so long ago, I began to deal with parallel computing toolbox. I wrote a rather large program, which I will not bring here, which, in fact, is sorting out options for meanings. Since there can be a lot and a lot of options, I decided not to keep a list of options, but just compare a couple of options on each iteration. This is complicated, so I found a way out in double recursion - one recursive function is called, which calls itself up to some point, after which it calls a second recursive function, which, in turn, also refers to itself several times. Within the body of each recursive function there are two for loops.
Forgive me my English (I do not know him well), but I will continue :D
This algorithm works for quite some time, but I managed to get acceleration almost 10 times compared to the original version with sorting the entire list. Nevertheless, I would like to learn to take advantage of some of the advantages of modern processors, such as multi-core.
I introduced one parfor cycle into the "upper recursion" body instead of the usual for, which accelerated the program three times. However, for 60% of the program time, only one out of six core is occupied, which, obviously, cannot but be depressed. I noticed that despite the fact that the recursive function is called several times, and the new parfor cycle is inside another parfor cycle, internal parfor cycles do not lead to the inclusion of new processor cores.
The essence of my question is how you can organize "parallelization during another parallelization."
parfor inside another parfor within the same function turns out to be an incorrect syntax, and if you transfer the internal cycle to a separate function, then it simply ceases to work as parfor and is replaced by the usual for.
I tried to put the whole body of the recursive function ("upper") in the cycle "spmd ... end "(without any parfor), however, this leads to an incredibly large busy amount of RAM (MatLab closes without finishing work, since there is not enough memory).
I also tried to call a recursive function using parfevalOnAll, but the program just works indefinitely, although the processor is fully loaded and the program should complete in a minute or two (I think).
I ask for help with understanding not only how to properly organize parallelization, but also what mistakes I make (from the description of what I am doing). Unfortunately, the documentation describes everything rather vaguely. I will be immensely grateful for any help!
Below I will give the basic template of my algorithm, which I am trying to parallel as much as possible.
% enter all x
variant = PEREBOR(x1, x2, .., xN);
function [variant] = PEREBOR(x1, x2, .., xN)
% some preliminary one-time calculations
for k = left:1:k_right
variants = nchoosek(..);
if isempty(variants) ~= 1
for i = 1:1:size(variants,1)
if recursion_depth > 1
[variant] = PEREBOR(x1, x2, .., xN);
else
[variant] = PEREBOR_2(x2, x4, x7);
end
end
else
if recursion_depth > 1
[variant] = PEREBOR(x1, x2, .., xN);
else
[variant] = PEREBOR_2(x2, x4, x7);
end
end
end
function [variant] = PEREBOR_2(x2, x4, x7)
% some preliminary one-time calculations
for k = left:1:k_right
variants = nchoosek(..);
if isempty(variants) ~= 1
for i = 1:1:size(variants,1)
if recursion_depth > 1
[variant] = PEREBOR_2(x2, x4, x7);
else
if ...
variant = something...
else
variant = [];
continue
end
end
end
else
if recursion_depth > 1
[variant] = PEREBOR_2(x2, x4, x7);
else
if ...
variant = something...
else
variant = [];
continue
end
end
end
end
0 Kommentare
Antworten (0)
Siehe auch
Kategorien
Mehr zu Performance and Memory finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!