Call to inv() function seems to have (undesired) impact on Thread pool or maxNumCompThreads()

I tried to parallelize parts of my code via parpool("Threads"). I also use maxNumCompThreads to limit the maximum CPU utilization. I use a parfor loop which works as expected, meaning that the defined number of cores corresponds to the total cpu utilization shown in the windows taks manager (more or less).

However, if a call to the inv() function appears somewhere in the code before the thread pool is started, then the cpu utilization of the thread pool is unexpectedly higher, although the number of cores and also maxNumCompThreads is not changed. This happens reproducible, until matlab is restarted (and inv() is not called).

To obtain the unexpected behavior the input to inv() must exceed a certain size: inv(rand(10)) --> nothing happens, but inv(rand(1000)) --> CPU utilization of the following parfor loop is unexpectedly high.

A simple script to reproduce the described behavior (in matlab2023b):

maxNumCompThreads(12);
nCores = 12;

%% random parallel code
fprintf("Before inv function call:\n");
pp = parpool("Threads", nCores);
for j = 1:3
    tic;
    parfor (i = 1:100)
        A = rand(1000) / rand(1000);
    end
    toc
    pause(2);
end
delete(pp);


%% matrix inverse
Minv = inv(rand(5000));
pause(5);


%% same random parallel code as before --> CPU Utilization goes up to 100%
fprintf("\n\nAfter inv function call:\n");
pp = parpool("Threads", nCores);
for j = 1:3
    tic;
    parfor (i = 1:100)
        A = rand(1000) / rand(1000);
    end
    toc
    pause(2);
end
delete(pp);

On a 56-core machine, the first parallel block runs with < 20% CPU utilization, while the second block has ~50%.

I get the following output:

Before inv function call:

Starting parallel pool (parpool) using the 'Threads' profile ...

Connected to parallel pool with 12 workers.

Elapsed time is 5.852217 seconds.

Elapsed time is 2.475874 seconds.

Elapsed time is 2.447292 seconds.

Parallel pool using the 'Threads' profile is shutting down.

After inv function call:

Starting parallel pool (parpool) using the 'Threads' profile ...

Connected to parallel pool with 12 workers.

Elapsed time is 23.414892 seconds.

Elapsed time is 24.350276 seconds.

Elapsed time is 23.297744 seconds.

Parallel pool using the 'Threads' profile is shutting down.

The increased core utilization for Thread pools stays present until matlab is closed an restarted. With parpool("Processes") I did not observe this behavior.

Am I missing anything here?

[Walter Roberson]

Are you sure you need inv() ? It is more robust to use A\B iinstead of inv(A)*B .

If you need the inv() result multiple times, then consider using decomposition

[Jan]

Hi Walter, I don't use inv() explicitly, but it is called by modalreal().

Answer 1

[Christine Tobler]

I asked around internally, and there was an issue that looks very similar in R2023b that has been fixed in R2024a. If you have R2024a available, could you check if things work as expected there?

[Jan]

Hi Christine,

I have a matlab2024a installation only on my laptop with 4 cores. There the picture is not as clear, but it indeed seems that the issue is solved in 2024a.

Also, if maxNumCompThreads is set to 1 BEFORE the inv() call, the problem does not seem to appear. However, the performance of inv() will probably decrease in that case.

Answer 2

[MULI]

Hi Jan,

I understand that you are encountering unexpected CPU utilization behaviour when using thread-based parallel pools in MATLAB. Here are some potential reasons:

1.Thread Scheduling and Usage:

Matrix Inversion Impact: The inv(rand(5000)) operation is very demanding on the CPU. When MATLAB performs this task, it may change how it uses your CPU cores to maximize performance.

Aggressive Thread Use: After such a heavy computation, MATLAB might start using threads more aggressively, sometimes ignoring the limit set by maxNumCompThreads.

2. Resource Contention:

Shared Resources: The matrix inversion uses a lot of shared resources (like CPU and memory). This can lead to "resource contention," where different parts of your code compete for the same resources.

Impact on parfor Loop: This contention can cause the parfor loop to use more CPU and take longer to execute.

Usage of ‘parpool’ eliminates this because,

• ‘parpool’ creates separate process instances rather than threads.
• Each process has its own memory space and resources, so changes in one process (like the matrix inversion) do not affect the others.
• This isolation prevents the unexpected CPU utilization increase.

You can refer to this documentation to know more about the "parpool" function:

https://www.mathworks.com/help/releases/R2023b/parallel-computing/parpool.html

[Jan]

Hi, thank you for your informative reply.

Aggressive Thread Use: After such a heavy computation, MATLAB might start using threads more aggressively, sometimes ignoring the limit set by maxNumCompThreads.

That's exactly the point I suspected. My follow up question would be, is there the possibility to restore the default "internal" settings for parallel threads (after the call to inv())?

Maybe some background information: In my simulation inv() is called internally by a toolbox function. Then a lot of computations are carried out (~1000 seconds, therefore I do not think that resource contention is the reason for the observed CPU utilization), including a parallel processes block. After that, the parallel thread pool is started, which is obviously influenced by the preceeding call of inv(). This part cannot be parallelized via "processes" because it invloves a large amount of data which makes the parallelization inefficient.

[Jan]

Follow up: On my machine the behavior change is observable if inv() is computed of a matrix of size >= 257.

Probably the algorithm to compute the inverse changes at this size.

[John D'Errico]

@Jan - I recall these codes all do not use multi-threading until the size grows to a point where a gain would be seen. Below that point, you would get a loss of performance, due to the extra overhead required. So I'd not be at all surprised to see it happen just beyond 256, a power of 2.