Is there a parallel version of splitapply()?
5 Ansichten (letzte 30 Tage)
Ältere Kommentare anzeigen
Simon
am 15 Sep. 2023
Kommentiert: Edric Ellis
am 20 Sep. 2023
Group-based computation is naturally sutiable for parallel computation. I wonder why Matlab has not yet a built-in parfor-splitapply? In Github, there is a repository called Matlab-FunUtils, which has cmap.m. It's like a parfor version of arrayfun. I couldn't find a similar function for splitapply. I have been trying to modify Matlab's splitapply, but the codes are very difficult to understand. Any suggesstoins? Should I write from scratch?
0 Kommentare
Akzeptierte Antwort
Edric Ellis
am 15 Sep. 2023
tall arrays support parallel execution of splitapply. Would that work for your case?
6 Kommentare
Edric Ellis
am 20 Sep. 2023
Yes, typically this sort of algorithm cannot work in a nested way. It isn't quite the same restriction as nested parfor (in fact, with thread-pools, you can get genuine 2-level parallelism with parfor, but there's a separate restriction that you need to "hide" the inner parfor inside a function).
As to your original question - does it work to make the groups based on the combinations of variables, as per the 2nd syntax of findgroups ? I.e. something like:
load patients
G = findgroups(Smoker, Age >= 40)
This divides the data into 4 groups for each combination of Smoker and Age >= 40.
Weitere Antworten (1)
Bruno Luong
am 15 Sep. 2023
To me there are 2 reasons:
- Not everyone have parallel toolbox
- The user function can be multi-threaded and already use efficiently the CPU cores, add parallel on top will then likely reduce the performance. The parallel computation should never be applied automatically anywhere. It should be judged case by case.
Siehe auch
Kategorien
Mehr zu Preprocessing Data finden Sie in Help Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!