dlCodegenBench

​​dlCodegenBench is a tool that benchmarks the runtime performance of deep learning models in MATLAB® code generation workflows. ​

dlCodegenBench

dlCodegenBench is a tool that benchmarks the runtime performance of deep learning models in MATLAB® code generation workflows.

Setup

MathWorks Products (https://www.mathworks.com)

Tested in MATLAB release R2024a or newer

• MATLAB
• Deep Learning Toolbox™
• MATLAB Coder™
• GPU Coder™
• MATLAB Coder Interface for Deep Learning
• GPU Coder Interface for Deep Learning

Documentation

Syntax

dlCodegenBench(model, inputData, codegenConfig)
dlCodegenBench(model, inputData, codegenConfig1, ... codegenConfigN)
[benchmarkResults, executionTimings] = dlCodegenBench( __ )
[ __ ] = dlCodegenBench( __ , Name=Value) 

Description

dlCodegenBench is a tool that benchmarks the runtime performance of deep learning models in code generation workflows.

dlCodegenBench(model, inputData, codegenConfig) benchmarks the specified model based on the provided input data and codegen config.

dlCodegenBench(model, inputData, codegenConfig1, ... codegenConfigN) benchmarks for the model with N different codegen configs.

[benchmarkResults, executionTimings] = dlCodegenBench( __ ) returns the benchmark results table and the raw execution timing matrix using any of the previous syntaxes.

[ __ ] = dlCodegenBench( __ , Name=Value) specifies additional options using one or more name-value arguments.

Input Arguments

model - Deep learning model for benchmark

This argument can be:

name of the MAT-file containing an object from one of the following classes:

• dlnetwork
• DAGNetwork or SeriesNetwork
• yolov2ObjectDetector, yolov3ObjectDetector, yolov4ObjectDetector, ssdObjectDetector

name of the function that is on the path and returns an object of the classes mentioned above.

inputData - Input data for the deep learning model

Example values that define the size and class of the deep learning model's inputs, specified as a numeric array or a dlarray object. For a model with multiple inputs, use a cell array of multiple numeric arrays or dlarray objects.

codegenConfig - Code generation configuration objects

Code generation configuration objects, created by either coder.config or coder.gpuConfig. You could provide one or more codegenConfigs in the call.

Name-Value Arguments

BatchSizes

A scalar or array of batch size values. The batch dimension of inputData is overwritten by each BatchSizes.

Default value is 1.

Mode

The evaluation mode can be set to either of the following two options:

• 'Offline' - benchmark results is throughput, defined as batch / (average of time) in unit of samples/sec.
• 'SingleStream' - benchmark results is latency, defined as 90th percentile of time vector in unit of ms.

For 'SingleStream' mode, BatchSizes can only be set to 1.

Default value is 'Offline'.

Iterations

Execution iterations.

Default value is 100.

WarmupIterations

Warmup iterations to be excluded from benchmark results evaluation. WarmupIterations must be smaller than Iterations.

Default value is 2.

InnerIterations

Number of executions inside each iteration. Within each iteration, InnerIterations number of runs are executed. If your application is faster than 1/10 second, consider setting InnerIterations to a higher value.

Default value is 1.

RunMatlabInference

Run MATLAB inference and include this to benchmark results when this is set to true.

Default value is true.

ExtraDLFunctionArgs

A cell array with additional name-value arguments to be passed to the deep learning function.

Example: {'MiniBatchSize', 128, 'SequenceLength', 'shortest'}

Default value is {}.

Display

Display benchmark results. When set to true, the benchmark results table is printed on MATLAB Command Window and a plot for comparison is displayed.

Default value is true.

ResultsDir

Directory to save benchmark results table and raw execution timing matrix in MAT-files. When this is empty, the MAT-files are not saved.

Default value is empty.

Output Arguments

benchmarkResults - Benchmark results table

Benchmark results returned as a table, throughput when Mode is set to 'Offline' and latency when Mode is set to 'SingleStream'.

executionTimings - Raw timing data

A cell array of raw timing data, which can be used for additional analysis. The elements in the cell array follow the order of codegenConfig input argument, and the last one being MATLAB inference raw timing if RunMatlabInference is set to true. Each element is a numeric matrix, where the row stands for each Iterations and column stands for each BatchSizes.

Examples

Example 1. Benchmark pretrained squeezenet

% Load a pretrained SqueezeNet neural network
net = imagePretrainedNetwork;

% Create 3 MEX codegen configs with different target deep learning libraries
cfg1 = coder.gpuConfig('mex');
cfg1.DeepLearningConfig = coder.DeepLearningConfig('cudnn');
cfg1.IntegrityChecks = false;
cfg1.ResponsivenessChecks = false;

cfg2 = coder.gpuConfig('mex');
cfg2.DeepLearningConfig = coder.DeepLearningConfig('tensorrt');
cfg2.IntegrityChecks = false;
cfg2.ResponsivenessChecks = false;

cfg3 = coder.gpuConfig('mex');
cfg3.DeepLearningConfig = coder.DeepLearningConfig('none');
cfg3.IntegrityChecks = false;
cfg3.ResponsivenessChecks = false;

% Define network input size and class
inputs = {dlarray(ones(227,227,3,'single'), 'SSC')};
% Run dlCodegenBench
% benchmark batch size 1 and 8, and run MATLAB inference
[benchmarkResults, executionTimings] = dlCodegenBench('imagePretrainedNetwork', inputs, cfg1, cfg2, cfg3, 'BatchSizes', [1,8], 'RunMATLABInference',true);

Example 2. Benchmark a multi-input dlnetwork with 'SingleStream' mode

% Create a simple multi-input dlnetwork and save it to a MAT-file
net = dlnetwork(additionLayer(2), dlarray(1,'SSC'),dlarray(1,'SSC'));
save exampleNet.mat net

% Create library-free mex and exe codegen configs for C code
cfg1 = coder.config('mex');
cfg1.DeepLearningConfig = coder.DeepLearningConfig('none');
cfg1.IntegrityChecks = false;
cfg1.ResponsivenessChecks = false;

cfg2 = coder.config('exe');
cfg2.DeepLearningConfig = coder.DeepLearningConfig('none');
% Run dlCodegenBench with 'SingleStream' for latency
dlCodegenBench('exampleNet.mat', {dlarray(1,'SSC'),dlarray(1,'SSC')}, cfg1, cfg2, 'Mode', 'SingleStream');

License

The license is available in the license.txt file in this project.

Community Support

MATLAB Central

Copyright 2025 The MathWorks, Inc.