Variable in parfor cannot be classified, error not shown in editor

Hey everyone I've been looking at parfor and going through some previously given answers and documentation but I'm having a hard time to figure out what my mistake is.

Error: The variable CFxL_Norm in a parfor cannot be classified.

I do assume that CFxLconst_Norm will give the same error. In my script I first assign 2 empty cell-matrix of size (XX,10) and each cell will contain a matrix (501x401x401), except in the last column the cells will contain 1 simple number.

normalisePerc=[0.2, 0.5, 0.8, 1, 0.01, 0.05, 0.1, 0.3, 0.4, 0.6, 0.7, 0.9]; %values can be changed later on
Lnlp=length(normalisePerc); %Length of the normalise percentage array
CFxL_Norm=cell(Lnlp,8+2); % CFxLight400nm + normaliseperc
CFxLconst_Norm=cell(Lnlp,8+2); % CFxLight400nm + normaliseperc

some other code and then this

A=Ef_Norm{9}; %%% assign variable as temporary for parfor, throwaway afterwards
%Lnlp=length(normalisePerc);  %Same, to not use length but a fixed constant
parfor kk=1:Lnlp
    fprintf('number %i of %i cycles for normalising \n', kk, Lnlp);
    NLP=normalisePerc(kk); %get value from array
    NLPinv=1/NLP; %allows to use multiplication in a few places for an increase in speed
    
    D=A.*NLPinv;
    D(D > 1) = 2./(1+1./D(D > 1));
    for tt=1:8
        fprintf('number %i of %i steps for normalising \n', tt+(kk-1)*8, Lnlp*8);
        tic
        B=Ef_Norm{tt}.*NLPinv;  %Gives a warning about overhead but not causing problems
        B(B > 1) = 2./(1+1./B(B > 1));
        for qq=1:lyq2 %Would have prefered to use parfor for this loop, lyq2=401. Matlab recommends outer loops only
            CFs=Cafluo_Norm{tt}(:,:,qq)'; %Gives a warning about overhead but not causing problems
            CFxL_Norm{kk,tt}(:,:,qq)=CFs.*B(:,:,qq);
            CFxLconst_Norm{kk,tt}(:,:,qq)=CFs.*D(:,:,qq);
        end
        fprintf('Time needed for this step was: \n');
        toc
    end
    tt=9;
    for qq=1:lyq2 %Also would have prefered to use parfor for this loop but Matlab recommends outer loops only.
        tmp1=Cafluo_Norm{tt}(:,:,qq)'.*D(:,:,qq);
        CFxL_Norm{kk,tt}(:,:,qq)=tmp1;
        CFxLconst_Norm{kk,tt}(:,:,qq)=tmp1;
    end
end
tt=10;
tmpnlp=num2cell(normalisePerc');
CFxL_Norm(:,10)=tmpnlp;
CFxLconst_Norm(:,tt)=tmpnlp;

clear tt kk NLP A B D NLPinv qq CFs tmp1 tmpnlp
fprintf('\n \n \n');

I don't see any of the problems regarding things, like addressing the same cell at the same time or even multiple times. I am also pretty sure the variables are independent of each other. Thus I don't really understand why I'm still getting this error and what I'm doing wrong.

Answer 1

[Edric Ellis]

The code you've shown there isn't complete enough for us to attempt to run and see the error you're encountering. It would be helpful if you could simplify the code to the point where we can try and run it as a for loop before trying a parfor loop.

That said, I think you're hitting one of the restrictions of assigning into sliced variables with nested for loops inside parfor. This doc page has the details. I think the relevant restriction is:

• Required (static): If you use a nested for-loop to index into a sliced array, you cannot use that array elsewhere in the parfor-loop.

In your code, you've got multiple cases where you're assigning into the sliced outputs. I would recommend trying to change things so that you have only a single assignment statement into your sliced outputs.

[Bob photonics]

I've run the code in a for loop before (messier code though) and that works, the whole reason for going for parfor is that I'm already running this on a server to offload and not interfere with other work (64 cores). It's still a bit slow as it takes an hour or so to run my code, especially if I want to to do some more extensive runs, right now matlab mostly only uses 1 core out of the 64, it doesn't need to use all 64, but using at least 8 or 16 would be great. Still the code I have is a bit difficult to allow it to be run as my data is fairly large and it would take a long time to run and is dependent on previous data and other scripts.

Considering that both Ef_Norm and Cafluo_Norm are both cell-arrays with 9 cells each, each cell containing a matrix each of size 501x401x401. The code I posted in my questions takes about an 10 minutes to run on the server after optimisations. 96*~6seconds (before 96*~25seconds) this excludes the time it takes to load the cell-array data into matlab which is probably another few minutes. Double-precision numbers, few zero values and some NaN values, but mostly filled matrices (less then 1/3 is zero/NaN).

Now I'm having a bit of trouble with understanding sliced variables and how to address them.

Looking more closely the parts that I wanted to turn into parfor loops seem to be causing the issue

parfor qq=1:lyq2
        CFs=Cafluo_Norm{tt}(:,:,qq)'; %Gives a warning about overhead
        CFxL_Norm{kk,tt}(:,:,qq)=CFs.*B(:,:,qq); %Now actually says: Valid indices are restricted in parfor loops
        CFxLconst_Norm{kk,tt}(:,:,qq)=CFs.*D(:,:,qq); %Now actually says: Valid indices are restricted in parfor loops
end

and this part

parfor qq=1:lyq2
    tmp1=Cafluo_Norm{tt}(:,:,qq)'.*D(:,:,qq);
    CFxL_Norm{kk,tt}(:,:,qq)=tmp1; %Now actually says: Valid indices are restricted in parfor loops
    CFxLconst_Norm{kk,tt}(:,:,qq)=tmp1; %Now actually says: Valid indices are restricted in parfor loops
end

Which seems to coincide with slicing problems, nonetheless I don't understand the slicing part well and how to solve it. All of it is expressed relatively abstract (which is good but hard to understand sometimes) and I haven't been able to match one of the examples to my code/problem.

[Bob photonics]

I tried a few changes based on what I understood about slicing but it didn't resolve the problem gives me the same error again:

Error: The variable CFxL_Norm in a parfor cannot be classified.

See Parallel for Loops in MATLAB, "Overview".

A=Ef_Norm{9}; %%% assign variable as temporary for parfor, throwaway afterwards
parfor kk=1:Lnlp
    fprintf('number %i of %i cycles for normalising \n', kk, Lnlp);
    NLP=normalisePerc(kk); %get value from array
    NLPinv=1/NLP; %allows to use multiplication in a few places for an increase in speed
    
    D=A.*NLPinv;
    D(D > 1) = 2./(1+1./D(D > 1));
    for tt=1:8
        fprintf('number %i of %i steps for normalising \n', tt+(kk-1)*8, Lnlp*8);
        tic
        B=Ef_Norm{tt}.*NLPinv;  %Gives a warning about overhead
        B(B > 1) = 2./(1+1./B(B > 1));
        AAA=zeros(lzq2,lxq2,lyq2); %CHANGE TO ADD THIS, temporary variable
        AAA2=zeros(lzq2,lxq2,lyq2); %CHANGE TO ADD THIS, temporary variable
        for qq=1:lyq2
            CFs=Cafluo_Norm{tt}(:,:,qq)'; %Gives a warning about overhead
            AAA(:,:,qq)=CFs.*B(:,:,qq); %Introduced to remove slicing and indexing issues
            AAA2(:,:,qq)=CFs.*D(:,:,qq); %Introduced to remove slicing and indexing issues
        end
        CFxL_Norm{kk,tt}=AAA; %Assign here to actual variable
        CFxLconst_Norm{kk,tt}=AAA2; %Assign here to actual variable
        fprintf('Time needed for this step was: \n');
        toc
    end
    tt=9;
    AAA3=zeros(lzq2,lxq2,lyq2); %Same changes repeated as before
    AAA4=zeros(lzq2,lxq2,lyq2); %same change
    for qq=1:lyq2
        tmp1=Cafluo_Norm{tt}(:,:,qq)'.*D(:,:,qq);
        AAA3(:,:,qq)=tmp1; %same change as before
        AAA4(:,:,qq)=tmp1; %same change
    end
    CFxL_Norm{kk,tt}=AAA3; %same change
    CFxLconst_Norm{kk,tt}=AA4; %Same change
end
tt=10;
tmpnlp=num2cell(normalisePerc');
CFxL_Norm(:,10)=tmpnlp;
CFxLconst_Norm(:,tt)=tmpnlp;

[Walter Roberson]

In the below, I have made the relevant changes only for CFxL_Norm, without looking at the other variables. You probably need to make similar changes for CFxLconst_Norm .

The key here is to use temporary variables to accumulate all partial results, and then have a single assignment statement at the end that changes everything having to do with the variable indexed by the parfor index.

A=Ef_Norm{9}; %%% assign variable as temporary for parfor, throwaway afterwards
parfor kk=1:Lnlp
    fprintf('number %i of %i cycles for normalising \n', kk, Lnlp);
    NLP=normalisePerc(kk); %get value from array
    NLPinv=1/NLP; %allows to use multiplication in a few places for an increase in speed
    
    D=A.*NLPinv;
    D(D > 1) = 2./(1+1./D(D > 1));
    AAAt = cell(1,9);
    for tt=1:8
        fprintf('number %i of %i steps for normalising \n', tt+(kk-1)*8, Lnlp*8);
        tic
        B=Ef_Norm{tt}.*NLPinv;  %Gives a warning about overhead
        B(B > 1) = 2./(1+1./B(B > 1));
        AAA=zeros(lzq2,lxq2,lyq2); %CHANGE TO ADD THIS, temporary variable
        AAA2=zeros(lzq2,lxq2,lyq2); %CHANGE TO ADD THIS, temporary variable
        for qq=1:lyq2
            CFs=Cafluo_Norm{tt}(:,:,qq)'; %Gives a warning about overhead
            AAA(:,:,qq)=CFs.*B(:,:,qq); %Introduced to remove slicing and indexing issues
            AAA2(:,:,qq)=CFs.*D(:,:,qq); %Introduced to remove slicing and indexing issues
        end
        AAAt{tt}=AAA; %Assign here to actual variable
        CFxLconst_Norm{kk,tt}=AAA2; %Assign here to actual variable
        fprintf('Time needed for this step was: \n');
        toc
    end
    tt=9;
    AAA3=zeros(lzq2,lxq2,lyq2); %Same changes repeated as before
    AAA4=zeros(lzq2,lxq2,lyq2); %same change
    for qq=1:lyq2
        tmp1=Cafluo_Norm{tt}(:,:,qq)'.*D(:,:,qq);
        AAA3(:,:,qq)=tmp1; %same change as before
        AAA4(:,:,qq)=tmp1; %same change
    end
    AAAt{tt}=AAA3; %same change
    CFxLconst_Norm{kk,tt}=AA4; %Same change
    
    CFxL_Norm(kk,:) = AAAt;
end

tt=10;
tmpnlp=num2cell(normalisePerc');
CFxl_Norm(:,tt)=tmpnlp;
CFxLconst_Norm(:,tt)=tmpnlp;

[Bob photonics]

@walter Robison

Sorry it looks like my reply didn't get posted.

Nonetheless thank you very much as this absolutely solved my problem.