UNDEFINED VARIABLE OR FUNCTION

Hello--

I am using the parallel toolbox which cause me to see this error message:

     An UndefinedFunction error was thrown on the workers for 'Q'.  This might be because the file containing
     'Q' is not accessible on the workers.  Use addAttachedFiles(pool, files) to specify the required files to
      be attached.  See the documentation for 'parallel.Pool/addAttachedFiles' for more details.

      Caused by:
      Undefined function or variable 'Q'.

The structure of my code is as following:(You can see how I defined Q)

DS=[is a given 1*N matrix]
parfor s = 1 :size(1,DS)
   % with some commands I end up having a new matrix called V
     for j = 3:2:( 2*(size(V,1))
      if  ((V(j,1) - V((j-2),1)) ~=0)
        vicinity_atom = [(V(j,:)), ( V((j-2),:)) 0 0 0 0 0 0 0 0 0 0 0 0 0 0];
         reduced_vicinity_atom = (sortrows((unique(vicinity_atom(vicinity_atom ~= 0),'stable')).')).' ;
         n = size (((find (reduced_vicinity_atom <(nh+1))).'), 1);
         m = size (((find (reduced_vicinity_atom >(nh))).'), 1);
         Q = zeros ( 501  ,( (1.5*(n+m))+ 6));  %This is where I defined matrix Q!
         Q(1:28 , ( (1.5*(n+m))+ 6)) =  vicinity_atom' ;
         %Here I am proposing some calculations to fill the matrix Q with appropriate values

        tttt = k + (1*285000);
        output =  sprintf('%d%s%d.csv',tttt,'evolution',atom);
        dlmwrite(output, Q, 'delimiter', ',', 'precision', 9);
         end
         end

      end

Thus, in the parfor loop there is a for loop in which a matrix Q is generated for the identifiers which satisfy the if conditions. As you see I saw such an error which would not be happened if I change parfor to for. Can anyone explain the reason and suggest a way to tackle this issue?

Thanks

Answer 1

[Walter Roberson]

The code has no way of proving that ((V(j,1) - V((j-2),1)) ~=0) will be true exactly once. parfor needs it to be obvious that all variables are well defined.

If there are more than one ((V(j,1) - V((j-2),1)) ~=0) then you are overwriting Q: are you sure you want to do that?

If you are certain that there is exactly one matching case perhaps you should use find() instead of a loop to locate it. Whether you do that or not, consider dlmwrite'ing within the section that defines Q.

[Homayoon]

No, there are several matching case with the condition determined in the if clause! Each time a new Q has to be defined and as you can see the dimension of matrix Q is also a variable which depends on m and n. For every matching case matrix Q has to be written as the output! But the crucial point is that a new matrix Q has to be defined each time a matching case is found.

[Homayoon]

well let me correct a mistake I made in the body of the question

[Homayoon]

Sorry for the mistake, Actually my code has 1500 lines and Obviously I could not copy all of it here! In my actual version I was dlmwrite'ing within the section that Q was defined. BUT the problem is that I see that error message. I am so grateful if you can help to figure what is going on with the parfor loop.

[Walter Roberson]

Sorry, my crystal ball is out of operation today. I will need to see the code.

[Homayoon]

Ok i will send it to you then, Is there any way to send it directly to you sir?

[Walter Roberson]

You can attach it here.

[Stephen23]

@Homayoon: upload your code here: click the paperclip button, then both Choose file and Attach file buttons.

[Homayoon]

Hello-- The code is attached to the question now. Please be advised that I could not attach the input file ask.txt due to the limitations! I keep receiving that error. what is wrong with the logic behind the code?

[Walter Roberson]

You have

for k = ks:(kmax-1)

so k is always at least as big as ks and can be equal to ks.

Then further down you have

for j = 3:2:( 2*(k - ks +1))

The first iteration there, k will equal ks, so k-ks would be 0; add 1 to that to get 1, multiply by 2 to get 2, so the "for j" loop is 3:2:2 which is not going to have any iterations. But when j has no iterations, Q will not be given a value.

[Homayoon]

Thank you for the correction, but it did not help out. Again I keep seeing the same error. Please be advised, as stated earlier the code runs perfectly and produces desirable outputs if I change parfor to for. Another issue is that actually when I changed the size of my input file i.e. ask.txt to a smaller file, the code ran even with parfor without giving this error!! I am kind of confused and I hope you could help me to figure if there is any logical error in the body of the code or if there is some limitation with the parfor!!!! Thanks

[Homayoon]

By the way, I can give you the input file if there is anyway to do that!!! Thanks for you helps

[Walter Roberson]

To best model the behavior of parfor without using parfor itself, right after the for loop statement that will later be converted to parfor, add "clear" statements that clear every variable that you assign to in the loop (except for the output variable indexed only by the loop variable and constants). This will eliminate the possibility that there is a path that fails to compute that variable, leaving the variable unchanged from a previous "for" iteration.

[Homayoon]

so basically you meant that I cannot use parfor the way is currently used in my code? The only reason to use parfor is actually speeding up my calculations which takes 4 days with a for loop!!!!

[Homayoon]

And my big question remains unanswered. Why does the code work with small input files (even using parfor) and why does it work with for loop but does not with parfor and large input files?

[Homayoon]

Would you please specifically determine in what line of my code I have to add 'clear' statements? I am confused and totally lost Thanks

[Walter Roberson]

"Please be advised, as stated earlier the code runs perfectly and produces desirable outputs if I change parfor to for. "

I express my doubts about that. I think that some of your paths do not initialize some variables. Each parfor worker is a separate process, and tasks can be assigned to different workers at arbitrary "parfor" indices, so if any variable is not defined in one path then the variable might not exist in the worker (or might, depending what the worker has happened to have done previously.) The way to debug that behavior while using "for" is to add a whole bunch of "clear" and see if the code bombs on the larger file.

Or you could use the debugger on the parfor version, perhaps putting in a breakpoint conditional on that array not existing, and when it fires work backwards to figure out why. I have not attempted to debug parallel processes so I do not know what the limitations of doing so are.