bug parfor: analyzing and transferring files to worker: not done: blocked

47 Ansichten (letzte 30 Tage)
Hello there,
I am a running a working script with parfor on matlab, after a while (~1200 iteration on parfor), matlab block without appareant reasons while producing the following message: "analyzing and transferring files to workers...". I am running the parfor on matlab 2023a, on local parfor only. I did validate the cluster of parfor before. The code is working correctly as the 1200 files were done. I tried with other very similar files and it did block also similarly. There is largely enough memory space on my PC. I tried to restart matlab, and also my PC, but this did not solve the problem. I also tried to find solution online, unsuccesfully. Anyone would have an idea ? many thanks for your attention.
  2 Kommentare
Ganesh
Ganesh am 17 Jan. 2024
In order for us to reproduce this issue, it would be a helpful if you share your code files.
I would also request you to share the stack trace of the error you are encountering.
student student
student student am 17 Jan. 2024
Bearbeitet: Walter Roberson am 7 Jul. 2024
Thanks Ganesh,
The code is working, so there is no error message.
The code is very classic, and as following:
---------------------------------------------------------------------------------------------------
a=1:1:10000;
parfor amF=1:length(a)
functAA(a,amF)
end
---------------------------------------------------------------------------------------------------
So, I guess it is more low level, matlab bug on parfor
thanks

Melden Sie sich an, um zu kommentieren.

Antworten (2)

Edric Ellis
Edric Ellis am 17 Jan. 2024
The "analyzing and transferring files to workers" message occurs only when the workers encounter a failure to find some code that they need. If you're using the "local" cluster type, then you should basically never see this. You can diagnose what got transferred by using listAutoAttachedFiles after your parfor loop, like this:
parfor i = 1:n
doStuff()
end
% Assuming "analyzing and transferring..." message has occurred
listAutoAttachedFiles(gcp())
  3 Kommentare
Steven Lord
Steven Lord am 17 Jan. 2024
Without seeing the body of your functAA function, I suspect it's probably going to be difficult or impossible to offer any specific suggestions.
student student
student student am 22 Jan. 2024
thank you for your responses. what I suspect here is that the parfor display: "analyzing and transferring files to workers..." as it cannot find some unproduced data into the loop: so it should stop and report the problem. but instead it display this message. thanks

Melden Sie sich an, um zu kommentieren.


Felix
Felix am 7 Jul. 2024
Ok. This is a bit late but hope it still helps. Also this may help people like me who encountered the same problem.
The behavior of stucking on 'analyzing and transferring files to worker ...' is most likely caused by error thrown by whatever functions called within the parfor loop somewhere in the middle of the parfor iterations.
When this happens, for reasons, instead of actually reporting the error, MATLAB will stuck on the message 'analyzing and transferring files to worker ...'.
The fix of this problem is obviously to resolve the error within the parfor loop. To see the error message, the workaround for MATLAB 2024a/2023b (I didn`t check other releases) is using thread based parpool, i.e., pool = parpool('Threads') before the parpool.
I hope this helps.

Kategorien

Mehr zu Parallel for-Loops (parfor) finden Sie in Help Center und File Exchange

Produkte


Version

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by