SPMD and FFT causes lock condition?

Is the FFT function compatible with SPMD execution?

I am finding that if I create a worker pool (parpool) of more than 1 worker, operating on chunks of data from a distributed variable, the FFT command appears to deadlock the workers. They are obviously running full blast (based on process utilization), but they never complete. If I run the same code with only 1 worker, or remove the SPMD block altogether, the FFT works fine.

In my application I have an electronic signal load as an array. I want to run an FFT filter over that waveform, taking it one FFT sized chunk at a time. I want to pass the same waveform data to each worker in the SPMD block and have each of them work on a section of the signal. So, I have converted the waveform to a CODISTRIBUTED array and let the workers figure out indexing into the array based on their LABINDEX value.

Here is some simple code that mimics what I am trying to do and shows that the workers get "stuck" when they execute the FFT commands:

% delete any existing parallel pools
delete (gcp ('nocreate'));

% create a pool with the given number of workers
iNumWorkers = 2;
pool = parpool (iNumWorkers);

% create enough data to spread out to the workers
iFFTSize = 1024;
x = rand (iNumWorkers * iFFTSize, 1) - 0.5;

spmd
    % create codistributed version of the raw data
    X = codistributed (x);

    % figure out the range of data this worker should use
    iStart = (labindex - 1) * iFFTSize + 1;
    iEnd = iStart + iFFTSize - 1;
    fprintf ('Range = %d to %d\n', iStart, iEnd);

    % grab our chunk of data and run FFT on it
    t = X (iStart : iEnd, 1);
    T = fft (t);
end

If you comment out the FFT line, or set iNumWorkers to 1, the code works fine.

So, am I doing something wrong, or is the FFT command not compatible with SPMD?

Answer 1

[Edric Ellis]

In this case, the problem is arising because each worker is indexing into the codistributed array X using different indices. Generally, all method calls on codistributed arrays should have identical arguments on each worker. This means that the array t is in a weird state, and I think that's causing the problems. You can calculate the fft of X in one of two ways. The simplest is:

spmd
  T = fft(X);
end

... but that's not quite what you were doing in your example. If you want to work with the local parts explicitly, you can do:

spmd
  LP = getLocalPart(X);
  LPf = fft(LP);
  T = codistributed.build(LPf, getCodistributor(X));
end

Answer 2

[lbender]

Thank you, Edric, that was the answer. Using getLocalPart () allows the workers to run without deadlocking over control of the codistributed array.