Can I use parfor within parfeval in MatlabR2019b and if yes, how?

I'm encountering an issue with my MATLAB R2019b GUI project. I'm trying to display two sets of 4DCT (Four-Dimensional Computed Tomography) images simultaneously. Each set contains 10 CT volumes. While displaying the first case(two sets), I want to asynchronously load additional sets in the background. My approach involves using parfeval(@select_data, x, data{idx}) for each 4DCT, where select_data is a function utilizing parfor to read the 10 DICOM volumes.

When I run select_data independently, it performs well, utilizing all available workers efficiently (e.g., 12 workers). However, when I use parfeval, only one additional worker is utilized, leading to slower loading times (about 30-40 seconds per 4DCT). Also it causes lag in the main thread, even though I expect multiple workers to be employed for the loading process and the rest available for my main.

Here are the server specs I have access to:

CPU(s): 24

Thread(s) per core: 2

Core(s) per socket: 6

Socket(s): 2

Memory: 70GB

I've attempted to limit the number of workers to 8 in the parpool, leaving 4 cores free to handle the main GUI thread, but the lag persists.

It's possible that the issue lies with the usage of parfor within parfeval, which might not be leveraging all available workers as expected?

[Damian Pietrus]

You can't currently nest parallel functions like parfeval and parfor together. For example, when you have a parfor loop that calls another function that contains a parfor, the inner function runs in serial. This should explain why select_data runs well independently but not when called from parfeval.

Are you only calling parfeval twice (one for each set), or more than that? Am I correct in understanding that you're using parfeval so that you can display both images simultaneously, and that running select_data independently only produces one image?

[max_hid]

Alright, I had a feeling that might be the case. While I don't necessarly need a solution for speeding up subsequent cases, the current performance issue is troubling.

I've implemented parfeval to load additional volumes in the background. This way, when I move to the next case (each comprising two 4DCTs, with ten CTs each), I aim to eliminate any loading delays. Moreover, I need to ensure that my GUI remains responsive during the background loading. For instance, users should still be able to scroll through slices and switch phases seamlessly while working with the initially opened case.

So my procedure is:

1. Load in the first case (vol_left, vol_right) using select_folders()
2. Asynchronously load more cases in the background (one parfeval for each 4DCT)
3. Wait for all futures to finish loading and process the outputs

[Edric Ellis]

Damian is correct that by default it is not possible to get nested layers of parallelism. However, if you're willing to to perform some additional setup, and some additional code refactoring, it can be done. (One problem is that you're not allowed to place parfor directly in the body of another parfor - but you can perfectly well hide it inside a function). Here's an untested sketch of how you might make this work.

c = parcluster('Processes');
c.NumThreads = 4; % Required to allow us to create thread pools on the process workers
p = parpool(c, 4);
% Get each process worker to build its own thread pool
parfevalOnAll(p, @parpool, 0, 'Threads', 4);
parfor i = 1:N
    out(i) = myfcn(i);
end
function out = myfcn(in)
out = 0;
% This will run in parallel using the thread pool assigned to this process
% worker
parfor i = 1:in
    out = out + 1;
end
end

[max_hid]

@Damian Pietrus & @Edric Ellis Thank you again. I've managed to implement several different options, but none have been satisfying so far. All of them worked for loading the volumes in the background (some faster, some slower). Nevertheless, the problem arises when my main thread lags during scrolling through the currently opened volume. The lag seems to occur during the fetching of the output, indicating a possible resource blocking issue. I've attempted to mitigate this by using two different cell arrays to prevent blocking when fetching the output, but the lag persists. Could it be because different variables within the workspace cannot be accessed at the same time? For instance, my main thread accessing axVolumes and a worker accessing left_Volumes. Is there a feature within the Parallel Toolbox that could solve this problem for me? I've included some reduced code from my project to illustrate how my variables are assigned. I read about ValueStore[which is not available in 2019b, I think], would that be a solution?

Additionally, I've tried an option using a parcluster and used jobs/tasks (which were way slower than parfeval, almost too slow for my use case). During the running job, there was zero lag, but then at fetchOutput, my main thread was completely frozen.

Update: I went through more documentation an saw that fetchOutputs will block the client until done. Which is probably the reason for my problem. So I though about usinge DataQueue

"parfeval does not block MATLAB, so you can continue working while computations take place. The workers compute in parallel and send intermediate results through the DataQueue as soon as they become available.

If you want to block MATLAB until parfeval completes, use the wait function on the future objects. Using the wait function is useful when subsequent code depends on the completion of parfeval."

If I understand this correct, I can receive my output from the workers without blocking my main client if I use the DataQueue without the wait() function?

[axVolumes{1}, generalInfos{1}, ~, ~, ~]      = select_folders(first_case_l, false);
[axVolumes{1+1}, generalInfos{1+1}, ~, ~, orientation{1}]  = select_folders(first_case_r, false);
if cases >= 2
    % Loop over each case and load them asynchronously
    futures(numel(all_volumes), 1) = parallel.FevalFuture;
    for idx = 3:numel(all_volumes)
        % Start loading the current case in the background
        futures(idx) = parfeval(@select_folders, 5, all_volumes{idx}, false);
    end

    % Wait for all futures to finish loading and process the outputs
    for idx = 3:numel(all_volumes)
        % Determine whether to store in left_Volumes or right_Volumes
        if mod(idx, 2) == 1
            left_case = (idx + 1) / 2;
            [left_Volumes{left_case}, generalInfos{idx}, ~, ~, orientation{idx}] = fetchOutputs(futures(idx));
            disp(['Volume: ', int2str(idx), ' is finished loading']);
        else
            right_case = idx / 2;
            [right_Volumes{right_case}, generalInfos{idx}, ~, ~, orientation{idx}] = fetchOutputs(futures(idx));
            disp(['Volume: ', int2str(idx), ' is finished loading']);
            disp(['Case: ', int2str(right_case), ' is finished loading']);
        end
    end

end

display_slice(axViewer1, axVolumes{1}, current_slice);
display_slice(axViewer2, axVolumes{2}, current_slice);

[Edric Ellis]

I'm still not 100% sure I understand where the problem is in your code. Here are some thoughts:

• Only use fetchOutputs if you need the output of a specific Future "right now". This will block for it to complete
• You might be able to use fetchNext if you want to retrieve outputs in the order that they complete. However, this will only change the ordering of when you get the results, it will not change how long it takes until the final result is available.
• There are various ways that you can trigger actions on completion of Futures. For instance, you could use afterAll to trigger an action when all elements of your futures array has finished. This approach leaves your client completely idle while the workers are busy.

[max_hid]

Big Update:

I think I managed to find a way to make it work like intented by only fetching in the moment I switch my case, I need to validate this further next Wednesday and will post my solution afterwards and also update on Stackoverflow.

Until then thank you very much, have a great weekend