Filter löschen
Filter löschen

Avoid memory copy in thread-based environment (parfeval)

8 Ansichten (letzte 30 Tage)
Is it possible to avoid memory copy of the input data to each individual worker when using a thread based environment (thus with shared memory)? Here a very simple example on what I'm intended to do:
function compMat()
m = rand(1000000000, 1);
c1 = f1(m);
c2 = f2(m);
fprintf("%f %f\n", c1, c2);
p1 = parfeval(backgroundPool, @f1, 1, m);
p2 = parfeval(backgroundPool, @f2, 1, m);
c1 = fetchOutputs(p1);
c2 = fetchOutputs(p2);
fprintf("%f %f\n", c1, c2);
function x = f1(m)
x = mean(sin(m));
function x = f2(m)
x = mean(cos(m));
The input data is quite large, but the two operations f1 and f2 on this data don't need a copy of the data. Is it possible to reimplement the above example such that all the time there is only one instance of the input vector in memory and the operations f1 and f2 run in parallel on it?

Akzeptierte Antwort

Edric Ellis
Edric Ellis am 9 Nov. 2023
In this case, thread-based workers already avoid copying the large array m to the workers. See this section in the doc. Note that when you call sin(m) that will create a new temporary array of the same size as m for the output.
Perhaps somewhat counter-intuitively for MATLAB, you might be better off using a for loop (rather than vectorised operations as you have in your example code) to operate on the large array m to avoid making same-sized temporary arrays during the computation.
  1 Kommentar
Thomas Witkowski
Thomas Witkowski am 13 Nov. 2023
Bearbeitet: Thomas Witkowski am 15 Nov. 2023
Thanks, I've completly overseen that sin(m) / cos(m) creates a temporary array. I've check the memory behavior with a for loop working on m. The memory consumption is than as expected in multi threaded environment.

Melden Sie sich an, um zu kommentieren.

Weitere Antworten (0)


Mehr zu Parallel for-Loops (parfor) finden Sie in Help Center und File Exchange




Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by