Parallel Computing Toolbox: Underperformance AND Cannot cancel or destroy a job that was not created by this Local cluster.

13 Ansichten (letzte 30 Tage)
I am running Win Server 2012 R2 with 2014A with PCT. I have two questions.
Question 1:
I keep getting the below warning,
Warning: Cannot cancel or destroy a job that was not created by this Local cluster.
> In Local.cancelOrDestroyJob at 34
In Local.Local>Local.hDestroyJob at 237
In CJSJobMethods>CJSJobMethods.destroyOneJob at 70
In CJSCommunicatingJob>CJSCommunicatingJob.destroyOneJob at 94
In Job.Job>Job.delete at 1037
In my_mFile at 39
Starting parallel pool (parpool) using the 'local' profile ... connected to 32 workers.
My code looks like,
myCluster = parcluster('local');
delete(myCluster.Jobs);
delete(gcp);
parpool('local',32);
A = length(myFiles);
parfor a = 1: A
%do stuff
end
The reason I included "delete(myCluster.Jobs);" is that without it I get another warning, as below. I should say I only have one MATLAB IDE running at anyone time on my machine.
Warning: Found 1 pre-existing communicating job(s) created by pool that are running. You can use 'delete(myCluster.Jobs)' to remove
all jobs created with profile local. To create 'myCluster' use 'myCluster = parcluster('local')'.
What code do I need to change/include to make sure I do not get any warning?
Question 2:
I have 32 vCPU (vCPU = half a physical core, ie half of a shared core, or one hyperthread), (Intel Xeon E5-2680 v2) on my machine. The task I am trying to run is very demanding (each task takes c. 3mins to run). Hence I want to make sure the machine is maxed out.
In the MATLAB IDE I go to HOME, parallel, manage cluster profiles, edit and change "Number of Workers to start on your local machine" to 32.
I start the job. Task manager shows the total CPU at 40-50%. Going into resource monitor shows the individual matlab.exe taking up 0.2% CPU on average.
Is this right? Normally when I use PCT (on a normal/low spec machine) and don't set myCluster, parpool(local,32) etc) I get CPU useage of 100%.
When I remove the two lines:
myCluster = parcluster('local');
delete(myCluster.Jobs);
and then run the job, I see CPU now averages at >90%. No idea why...
thanks
  2 Kommentare
Darwin Brochero
Darwin Brochero am 17 Dez. 2014
I think that warning appears due to PARFOR loops unsuccessfully closed in other passed sessions. But you can rename (or deleted ?) configuration file that saves history of pass parfor evaluations (in my GNU/LINUX> .matlab/local_cluster_jobs/R2014a/matlab_metadata.mat)
Darwin Brochero
Darwin Brochero am 18 Dez. 2014
oups!
Renaming file gives an error in new parpool sessions, so in my case I save original configuration file with some prefix (orig-matlab_metadata.mat), and when I have this warning, I copy this file as "matlab_metadata.mat" to avoid this problem.
...That's still as a not technical fix, I know, but it's works.

Melden Sie sich an, um zu kommentieren.

Antworten (1)

Sean de Wolski
Sean de Wolski am 18 Dez. 2014
Bearbeitet: Sean de Wolski am 18 Dez. 2014
The only time I have seen that is also when something failed in a previous MATLAB session (hang/out of memory/etc.)
You should be able to clear them out in the job monitor.

Kategorien

Mehr zu Parallel Computing Fundamentals finden Sie in Help Center und File Exchange

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by