Simpler MEX Multi-Threading w/ a Persistent Thread Pool

Version 1.1.2 (12,9 KB) von oreoman
Further speed up your MEX files by making it easier to add in support for multi-threading and eliminate multi-threading overhead.
72 Downloads
Aktualisiert 5. Jul 2021
I can't believe this actually worked.
For smaller problems that can't be vectorized, properly setting up multi-threading within a MEX file can create enough overhead that the single-threaded solution is often faster. Using a thread pool eliminates that overhead. The pool is created once, and then can be re-used in multiple calls to your MEX function. The thread pool is then closed out and cleared from memory by typing 'clear all' or 'clear mex' in the command line (or closing MATLAB).
Depending on the problem, you can potentially see a speedup of up to 'maxNumCompThreads', though that's probably not likely. In my very basic testing, I'm seeing ~2x speedup (with four threads) over many iterations of a small problem. For the real-world problem I created this for, I'm seeing a ~60% speedup compared to standard MEX multi-threading, saving me a couple of days in total simulation time.
In addition, adding multi-threading to a MEX file is now much easier, as you only need to learn how to use one function, AddThreadPoolJob. You then just have to remember to close your MEX file with SynchronizeThreads so that you don't get awesome race conditions.
See the included madd_threadpool.c example and compare to madd.c that uses regular multi-threading to figure out what's going on. There's also more documentation in threadpool.h. To see how single- thread vs. multi-thread vs. thread pooled vs. vectorized code compares, compile things and check out threadpool_timings.m.
NOTE: properly vectorized MATLAB code will always be faster than this. Only use this stuff if you can't vectorize your code but it still lends itself to straightforward parallelization, e.g. integrating recurrence relations in multiple dimensions.

Zitieren als

oreoman (2024). Simpler MEX Multi-Threading w/ a Persistent Thread Pool (https://github.com/michael-nix/MATLAB-MEX-Threadpool), GitHub. Abgerufen .

Kompatibilität der MATLAB-Version
Erstellt mit R2019a
Kompatibel mit R2018a und späteren Versionen
Plattform-Kompatibilität
Windows macOS Linux
Kategorien
Mehr zu Write C Functions Callable from MATLAB (MEX Files) finden Sie in Help Center und MATLAB Answers

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Versionen, die den GitHub-Standardzweig verwenden, können nicht heruntergeladen werden

Version Veröffentlicht Versionshinweise
1.1.2

Simple bug fix in madd_threadpool.c

1.1.1

Connected to GitHub...

1.1.0

Added a few helper functions to make my life easier, and make it possible to use the same thread pool among multiple MEX files. Not properly documented 'cause I'm lazy.

1.0.0

Um Probleme in diesem GitHub Add-On anzuzeigen oder zu melden, besuchen Sie das GitHub Repository.
Um Probleme in diesem GitHub Add-On anzuzeigen oder zu melden, besuchen Sie das GitHub Repository.