Unrecognized function or variable in parfor

Hi.
I am a beginner in MATLAB software and I use MATLAB online .The problem is in initializing a variable called 'estModel' in a parallel loop . I want to use parallel processing because the calculations take a long time to complete but this problem occurs when I use 'threads' in parpool instead of 'local' . In 'local' mode this problem does not exist and the code is executed correctly and the results are correct . But in the case of 'threads', the variable 'estModel' is not initialize in the parfor loop , and give this error : Unrecognized function or variable 'estModel' . In 'local' mode the number of workers are 2 but in 'threads' mode the number of workers are 8 . So I use 'threads' instead of 'local' . I do not know about parpool structure completely . Is there a problem in this part ( parpool : local or threads or ... and chooses 'threads' based on the number of workers ) ? Finally the three variables 'kh', 'model' and 'DATA' are completely initializing before parallel loop and the variable 'model' uses the arima function .
Can anyone help me to solve this problem?
part of code :
poolobj=parpool('threads');
parfor i=1:kh
estModel(i)=estimate(model(i),DATA);
end
delete(poolobj);

Answer 1

[Raymond Norris]

I don't see how you're getting the error. Can you provide a bit more context? For example, when I run something similar (I think?), I get a different error

kh = 2;
for i = 1:kh
    model(i) = arima(2,0,1); %#ok<SAGROW> 
end
DGP = arima('AR',{0.5,-0.3},'MA',0.2,'Constant',0,'Variance',0.1);
rng(5);
T = 500;
DATA = simulate(DGP,T);
poolObj = parpool("threads");
parfor i = 1:kh
    estModel(i) = estimate(model(i),DATA);
end

Is this not what you're seeing?

Error using optimoptions
Use of function matlabpath is not supported on a thread-based worker.

Error in arima/estimate (line 253)
default = optimoptions('fmincon');

Related documentation

[An Engineer]

complete code :

clear all;
close all;
clc;
DATA=xlsread('test');
sz=length(DATA);
p=20;
q=10;
D=2;
kh=0;
for k=1:D
    for j=1:q
        for i=1:p
            if i+j+k-2<=sz
                kh=kh+1;
                model(kh)=arima(i-1,k-1,j-1);
            end
        end
    end
end
poolobj=parpool('threads');
parfor i=1:kh
    estModel(i)=estimate(model(i),DATA);
end
delete(poolobj);

The excel file called 'test' is an arbitrary time series that is a column vector with 500 datas.

[Walter Roberson]

arima is not one of the functions supported by threads.

https://www.mathworks.com/help/matlab/referencelist.html?type=function&capability=threadsupport

arima/estimate calls upon fmincon() which also is not supported.

[An Engineer]

Hi.

Thanks for your reply

You are right, It seems that 'threads' dont support 'arima' and 'estimate'.

Is there another way that can be replaced to reduce code execution time?

[Walter Roberson]

Perhaps if you edit the configuration for your local pool, you might be able to use more workers.

Note: it is not recommended to use hyperthreading for mathematical computation -- but it can work well to use hyperthreading for I/O workers.

[An Engineer]

Unfortunately, in 'local' mode, the maximum number of workers is 2 and it can not be used more.

[Walter Roberson]

https://www.mathworks.com/help/parallel-computing/discover-clusters-and-use-cluster-profiles.html

Edit NumWorkers (number of workers to start on your local machine)

The default setting is the number of physical cores, but you can change that. You can, for example, double that so that hyperthreaded cores are used as well (assuming that you have hyperthreading enabled.)

You could, if you wanted, ask for 50 workers (but more than 64 can be a problem on Windows.) You would be leaving it up to the operating system to schedule all those workers on the number of cores (and hyperthreads) actually available.

But remember:

• On the great majority of CPUs, hyperthreading does not provide additional compute capacity. hyperthreading is more a fast-switch technology, so that as soon as a worker volunteers to give up control of a core (to wait for disk or an interrupt or for a user to respond or voluntary pause()), then the new worker gets activated quite quickly. Hyperthreading does not add any additional Arithmetic Logic Units or instruction decoders or so on.
• On some quite new CPUs, hyperthreading is able to dispatch instructions to the Integer Arithmetic Logic Unit while the main thread is busy doing floating point tasks. Using this effectively requires some very fine tuning.
• If you have more workers than you have effective available cores. then the operating system might try to schedule them "fairly" -- which might mean that it rolls a worker out and replaces it with another worker. This takes time lowers the overall efficiency compared to running one task to completion and then starting another

[Raymond Norris]

Hi @An Engineer. Support for local clusters running on MATLAB Online was recently unintentionally added and has since been removed. You might consider running remote clusters with Cloud Center. To do so, you'll need an AWS account as well as a MATLAB Parallel Server license. Contact your Account Rep or Technical Support for more information.