Operations

Develop custom deep learning functions

For most tasks, you can use built-in layers. If there is not a built-in layer that you need for your task, then you can define your own custom layer. You can specify a custom loss function using a custom output layer and define custom layers with learnable and state parameters. After defining a custom layer, you can check that the layer is valid, GPU compatible, and outputs correctly defined gradients. To learn more, see Define Custom Deep Learning Layers. For a list of supported layers, see List of Deep Learning Layers.

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For models that cannot be specified as networks of layers, you can define the model as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

Use deep learning operations to develop MATLAB^® code for custom layers, training loops, and model functions.

Funktionen

alle erweitern

Automatic Differentiation

`dlarray`	Deep learning array for customization (Seit R2019b)
`dims`	Dimension labels of `dlarray` (Seit R2019b)
`finddim`	Find dimensions with specified label (Seit R2019b)
`stripdims`	Remove `dlarray` data format (Seit R2019b)
`extractdata`	Extract data from `dlarray` (Seit R2019b)
`isdlarray`	Check if object is `dlarray` (Seit R2020b)

Deep Learning Operations

`dlconv`	Deep learning convolution (Seit R2019b)
`dltranspconv`	Deep learning transposed convolution (Seit R2019b)
`lstm`	Long short-term memory (Seit R2019b)
`gru`	Gated recurrent unit (Seit R2020a)
`attention`	Dot-product attention (Seit R2022b)
`embed`	Embed discrete data (Seit R2020b)
`fullyconnect`	Sum all weighted input data and apply a bias (Seit R2019b)
`dlode45`	Deep learning solution of nonstiff ordinary differential equation (ODE) (Seit R2021b)
`batchnorm`	Normalize data across all observations for each channel independently (Seit R2019b)
`crosschannelnorm`	Cross channel square-normalize using local responses (Seit R2020a)
`groupnorm`	Normalize data across grouped subsets of channels for each observation independently (Seit R2020b)
`instancenorm`	Normalize across each channel for each observation independently (Seit R2021a)
`layernorm`	Normalize data across all channels for each observation independently (Seit R2021a)
`avgpool`	Pool data to average values over spatial dimensions (Seit R2019b)
`maxpool`	Pool data to maximum value (Seit R2019b)
`maxunpool`	Unpool the output of a maximum pooling operation (Seit R2019b)

Activations

`relu`	Apply rectified linear unit activation (Seit R2019b)
`leakyrelu`	Apply leaky rectified linear unit activation (Seit R2019b)
`gelu`	Apply Gaussian error linear unit (GELU) activation (Seit R2022b)
`softmax`	Apply softmax activation to channel dimension (Seit R2019b)
`sigmoid`	Apply sigmoid activation (Seit R2019b)

Loss Operations

`crossentropy`	Cross-entropy loss for classification tasks (Seit R2019b)
`l1loss`	L₁ loss for regression tasks (Seit R2021b)
`l2loss`	L₂ loss for regression tasks (Seit R2021b)
`huber`	Huber loss for regression tasks (Seit R2021a)
`mse`	Half mean squared error (Seit R2019b)
`ctc`	Connectionist temporal classification (CTC) loss for unaligned sequence classification (Seit R2021a)

Function Acceleration

`dlaccelerate`	Accelerate deep learning function for custom training loops (Seit R2021a)
`AcceleratedFunction`	Accelerated deep learning function (Seit R2021a)
`clearCache`	Clear accelerated deep learning function trace cache (Seit R2021a)

Themen

Automatic Differentiation

List of Functions with dlarray Support
View the list of functions that support dlarray objects.
Automatic Differentiation Background
Learn how automatic differentiation works.
Use Automatic Differentiation In Deep Learning Toolbox
How to use automatic differentiation in deep learning.

Model Functions

Train Network Using Model Function
This example shows how to create and train a deep learning network by using functions rather than a layer graph or a dlnetwork.
Update Batch Normalization Statistics Using Model Function
This example shows how to update the network state in a network defined as a function.
Make Predictions Using Model Function
This example shows how to make predictions using a model function by splitting data into mini-batches.
Initialize Learnable Parameters for Model Function
Learn how to initialize learnable parameters for custom training loops using a model function.

Deep Learning Function Acceleration

Deep Learning Function Acceleration for Custom Training Loops
Accelerate model functions and model loss functions for custom training loops by caching and reusing traces.
Accelerate Custom Training Loop Functions
This example shows how to accelerate deep learning custom training loop and prediction functions.
Check Accelerated Deep Learning Function Outputs
This example shows how to check that the outputs of accelerated functions match the outputs of the underlying function.
Evaluate Performance of Accelerated Deep Learning Function
This example shows how to evaluate the performance gains of using an accelerated function.