Main Content

Benchmark A\b by Using GPU Coder

This example shows how to benchmark solving a linear system by generating GPU code. Use matrix left division, also known as mldivide or the backslash operator (\), to solve the system of linear equations A*x = b for x (that is, compute x = A\b).

Prerequisites

  • CUDA® enabled NVIDIA® GPU with compute capability 3.5 or higher.

  • NVIDIA CUDA toolkit and driver.

  • Environment variables for the compilers and libraries. For information on the supported versions of the compilers and libraries, see Third-Party Hardware. For setting up the environment variables, see Setting Up the Prerequisite Products.

Verify GPU Environment

To verify that the compilers and libraries necessary for running this example are set up correctly, use the coder.checkGpuInstall function.

envCfg = coder.gpuEnvConfig('host');
envCfg.BasicCodegen = 1;
envCfg.Quiet = 1;
coder.checkGpuInstall(envCfg);

Determine the Maximum Data Size

Choose the appropriate matrix size for the computations by specifying the amount of system memory in GB available to the CPU and the GPU. The default value is based only on the amount of memory available on the GPU. You can specify a value that is appropriate for your system.

g = gpuDevice;
maxMemory = 0.25*g.AvailableMemory/1024^3;

The Benchmarking Function

This example benchmarks matrix left division (\) including the cost of transferring data between the CPU and GPU, to get a clear view of the total application time when using GPU Coder™. The application time profiling must not include the time to create sample input data. The genData.m function separates generation of test data from the entry-point function that solves the linear system.

type getData.m
function [A, b] = getData(n, clz)

%   Copyright 2017-2019 The MathWorks, Inc.

    fprintf('Creating a matrix of size %d-by-%d.\n', n, n);
    A = rand(n, n, clz) + 100*eye(n, n, clz);
    b = rand(n, 1, clz);
end

The Backslash Entry-Point Function

The backslash.m entry-point function encapsulates the (\) operation for which you want to generate code.

type backslash.m
function [x] = backslash(A,b)
%#codegen

%   Copyright 2017-2019 The MathWorks, Inc.

    coder.gpu.kernelfun();
    x = A\b;
end

Generate the GPU Code

Create a function to generate the GPU MEX function based on the particular input data size.

type genGpuCode.m
function [] = genGpuCode(A, b)

%   Copyright 2017-2019 The MathWorks, Inc.

    cfg = coder.gpuConfig('mex');
    evalc('codegen -config cfg -args {A,b} backslash');
end

Choose a Problem Size

The performance of the parallel algorithms that solve a linear system depends greatly on the matrix size. This example compares the performance of the algorithm for different matrix sizes.

% Declare the matrix sizes to be a multiple of 1024.
sizeLimit = inf;
if ispc
    sizeLimit = double(intmax('int32'));
end
maxSizeSingle = min(floor(sqrt(maxMemory*1024^3/4)),floor(sqrt(sizeLimit/4)));
maxSizeDouble = min(floor(sqrt(maxMemory*1024^3/8)),floor(sqrt(sizeLimit/8)));
step = 1024;
if maxSizeDouble/step >= 10
    step = step*floor(maxSizeDouble/(5*step));
end
sizeSingle = 1024:step:maxSizeSingle;
sizeDouble = 1024:step:maxSizeDouble;
numReps = 5;

Compare Performance: Speedup

Use the total elapsed time as a measure of performance because that enables you to compare the performance of the algorithm for different matrix sizes. Given a matrix size, the benchmarking function creates the matrix A and the right-side b once, and then solves A\b a few times to get an accurate measure of the time it takes.

type benchFcnMat.m
function time = benchFcnMat(A, b, reps)

%   Copyright 2017-2019 The MathWorks, Inc.

    time = inf;
    % Solve the linear system a few times and take the best run
    for itr = 1:reps
        tic;
        matX = backslash(A, b);
        tcurr = toc;
        time = min(tcurr, time);
    end
end

Create a different function for GPU code execution that invokes the generated GPU MEX function.

type benchFcnGpu.m
function time = benchFcnGpu(A, b, reps)

%   Copyright 2017-2019 The MathWorks, Inc.

    time = inf;
    gpuX = backslash_mex(A, b);
    for itr = 1:reps
        tic;
        gpuX = backslash_mex(A, b);
        tcurr = toc;
        time = min(tcurr, time);
    end
end

Execute the Benchmarks

When you execute the benchmarks, the computations can take a long time to complete. Print some intermediate status information as you complete the benchmarking for each matrix size. Encapsulate the loop over all the matrix sizes in a function to benchmark single- and double-precision computations.

Actual execution times can vary across different hardware configurations. This benchmarking was done by using MATLAB R2020a on a machine with a 6 core, 3.5GHz Intel® Xeon® CPU and an NVIDIA TITAN Xp GPU.

type executeBenchmarks.m
function [timeCPU, timeGPU] = executeBenchmarks(clz, sizes, reps)

%   Copyright 2017-2019 The MathWorks, Inc.

    fprintf(['Starting benchmarks with %d different %s-precision ' ...
         'matrices of sizes\nranging from %d-by-%d to %d-by-%d.\n'], ...
            length(sizes), clz, sizes(1), sizes(1), sizes(end), ...
            sizes(end));
    timeGPU = zeros(size(sizes));
    timeCPU = zeros(size(sizes));   
    for i = 1:length(sizes)
        n = sizes(i);
        fprintf('Size : %d\n', n);
        [A, b] = getData(n, clz);
        genGpuCode(A, b);
        timeCPU(i) = benchFcnMat(A, b, reps);
        fprintf('Time on CPU: %f sec\n', timeCPU(i));
        timeGPU(i) = benchFcnGpu(A, b, reps);
        fprintf('Time on GPU: %f sec\n', timeGPU(i));
        fprintf('\n');
    end
end

Execute the benchmarks in single and double precision.

[cpu, gpu] = executeBenchmarks('single', sizeSingle, numReps);
results.sizeSingle = sizeSingle;
results.timeSingleCPU = cpu;
results.timeSingleGPU = gpu;
[cpu, gpu] = executeBenchmarks('double', sizeDouble, numReps);
results.sizeDouble = sizeDouble;
results.timeDoubleCPU = cpu;
results.timeDoubleGPU = gpu;
Starting benchmarks with 9 different single-precision matrices of sizes
ranging from 1024-by-1024 to 25600-by-25600.
Size : 1024
Creating a matrix of size 1024-by-1024.
Time on CPU: 0.010894 sec
Time on GPU: 0.012735 sec

Size : 4096
Creating a matrix of size 4096-by-4096.
Time on CPU: 0.256912 sec
Time on GPU: 0.056594 sec

Size : 7168
Creating a matrix of size 7168-by-7168.
Time on CPU: 0.859825 sec
Time on GPU: 0.138257 sec

Size : 10240
Creating a matrix of size 10240-by-10240.
Time on CPU: 2.190538 sec
Time on GPU: 0.276720 sec

Size : 13312
Creating a matrix of size 13312-by-13312.
Time on CPU: 4.265689 sec
Time on GPU: 0.481420 sec

Size : 16384
Creating a matrix of size 16384-by-16384.
Time on CPU: 8.167554 sec
Time on GPU: 0.771961 sec

Size : 19456
Creating a matrix of size 19456-by-19456.
Time on CPU: 12.814798 sec
Time on GPU: 1.150009 sec

Size : 22528
Creating a matrix of size 22528-by-22528.
Time on CPU: 19.060859 sec
Time on GPU: 1.671082 sec

Size : 25600
Creating a matrix of size 25600-by-25600.
Time on CPU: 27.237500 sec
Time on GPU: 2.318981 sec

Starting benchmarks with 6 different double-precision matrices of sizes
ranging from 1024-by-1024 to 16384-by-16384.
Size : 1024
Creating a matrix of size 1024-by-1024.
Time on CPU: 0.025851 sec
Time on GPU: 0.021405 sec

Size : 4096
Creating a matrix of size 4096-by-4096.
Time on CPU: 0.435886 sec
Time on GPU: 0.175009 sec

Size : 7168
Creating a matrix of size 7168-by-7168.
Time on CPU: 1.734958 sec
Time on GPU: 0.765464 sec

Size : 10240
Creating a matrix of size 10240-by-10240.
Time on CPU: 4.240617 sec
Time on GPU: 2.081403 sec

Size : 13312
Creating a matrix of size 13312-by-13312.
Time on CPU: 8.611123 sec
Time on GPU: 4.415243 sec

Size : 16384
Creating a matrix of size 16384-by-16384.
Time on CPU: 19.387437 sec
Time on GPU: 8.050974 sec

Plot the Performance

Plot the results and compare the performance on the CPU and the GPU for single and double precision.

First, look at the performance of the backslash operator in single precision.

fig = figure;
ax = axes('parent', fig);
plot(ax, results.sizeSingle, results.timeSingleGPU, '-x', ...
     results.sizeSingle, results.timeSingleCPU, '-o')
grid on;
legend('GPU', 'CPU', 'Location', 'NorthWest');
title(ax, 'Single-Precision Performance')
ylabel(ax, 'Time (s)');
xlabel(ax, 'Matrix Size');
drawnow;

Now, look at the performance of the backslash operator in double precision.

fig = figure;
ax = axes('parent', fig);
plot(ax, results.sizeDouble, results.timeDoubleGPU, '-x', ...
     results.sizeDouble, results.timeDoubleCPU, '-o')
legend('GPU', 'CPU', 'Location', 'NorthWest');
grid on;
title(ax, 'Double-Precision Performance')
ylabel(ax, 'Time (s)');
xlabel(ax, 'Matrix Size');
drawnow;

Finally, look at the speedup of the backslash operator when comparing the GPU to the CPU.

speedupDouble = results.timeDoubleCPU./results.timeDoubleGPU;
speedupSingle = results.timeSingleCPU./results.timeSingleGPU;
fig = figure;
ax = axes('parent', fig);
plot(ax, results.sizeSingle, speedupSingle, '-v', ...
     results.sizeDouble, speedupDouble, '-*')
grid on;
legend('Single-precision', 'Double-precision', 'Location', 'SouthEast');
title(ax, 'Speedup of Computations on GPU Compared to CPU');
ylabel(ax, 'Speedup');
xlabel(ax, 'Matrix Size');
drawnow;