Custom Training Loops

Customize deep learning training loops and loss functions for sequence and tabular data

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For networks that cannot be created using layer graphs, you can define custom networks as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

Funktionen

alle erweitern

Custom Training Loops

`dlnetwork`	Deep learning network for custom training loops (Seit R2019b)
`trainingProgressMonitor`	Monitor and plot training progress for deep learning custom training loops (Seit R2022b)
`minibatchqueue`	Create mini-batches for deep learning (Seit R2020b)
`padsequences`	Pad or truncate sequence data to same length (Seit R2021a)
`dlarray`	Deep learning array for customization (Seit R2019b)
`dlgradient`	Compute gradients for custom training loops using automatic differentiation (Seit R2019b)
`dlfeval`	Evaluate deep learning model for custom training loops (Seit R2019b)

Loss Operations

`crossentropy`	Cross-entropy loss for classification tasks (Seit R2019b)
`l1loss`	L₁ loss for regression tasks (Seit R2021b)
`l2loss`	L₂ loss for regression tasks (Seit R2021b)
`huber`	Huber loss for regression tasks (Seit R2021a)
`mse`	Half mean squared error (Seit R2019b)
`ctc`	Connectionist temporal classification (CTC) loss for unaligned sequence classification (Seit R2021a)

Deep Learning Operations

`dlconv`	Deep learning convolution (Seit R2019b)
`dltranspconv`	Deep learning transposed convolution (Seit R2019b)
`lstm`	Long short-term memory (Seit R2019b)
`gru`	Gated recurrent unit (Seit R2020a)
`attention`	Dot-product attention (Seit R2022b)
`embed`	Embed discrete data (Seit R2020b)
`fullyconnect`	Sum all weighted input data and apply a bias (Seit R2019b)
`dlode45`	Deep learning solution of nonstiff ordinary differential equation (ODE) (Seit R2021b)
`batchnorm`	Normalize data across all observations for each channel independently (Seit R2019b)
`crosschannelnorm`	Cross channel square-normalize using local responses (Seit R2020a)
`groupnorm`	Normalize data across grouped subsets of channels for each observation independently (Seit R2020b)
`instancenorm`	Normalize across each channel for each observation independently (Seit R2021a)
`layernorm`	Normalize data across all channels for each observation independently (Seit R2021a)
`avgpool`	Pool data to average values over spatial dimensions (Seit R2019b)
`maxpool`	Pool data to maximum value (Seit R2019b)
`maxunpool`	Unpool the output of a maximum pooling operation (Seit R2019b)

Activations

`relu`	Apply rectified linear unit activation (Seit R2019b)
`leakyrelu`	Apply leaky rectified linear unit activation (Seit R2019b)
`gelu`	Apply Gaussian error linear unit (GELU) activation (Seit R2022b)
`softmax`	Apply softmax activation to channel dimension (Seit R2019b)
`sigmoid`	Apply sigmoid activation (Seit R2019b)

Themen

Custom Training Loops

Train Deep Learning Model in MATLAB
Learn how to training deep learning models in MATLAB^®.
Define Custom Training Loops, Loss Functions, and Networks
Learn how to define and customize deep learning training loops, loss functions, and networks using automatic differentiation.
Train Sequence Classification Network Using Custom Training Loop
This example shows how to train a network that classifies sequences with a custom learning rate schedule.
Monitor Custom Training Loop Progress
Track and plot custom training loop progress.
Train Network with Multiple Outputs
This example shows how to train a deep learning network with multiple outputs that predict both labels and angles of rotations of handwritten digits.
Classify Videos Using Deep Learning with Custom Training Loop
This example shows how to create a network for video classification by combining a pretrained image classification model and a sequence classification network.
Train Neural ODE Network
This example shows how to train an augmented neural ordinary differential equation (ODE) network.
Solve Ordinary Differential Equation Using Neural Network
This example shows how to solve an ordinary differential equation (ODE) using a neural network.

Automatic Differentiation

Create Bidirectional LSTM (BiLSTM) Function
This example shows how to create a bidirectional long-short term memory (BiLSTM) function for custom deep learning functions. (Seit R2023b)
List of Functions with dlarray Support
View the list of functions that support dlarray objects.
Automatic Differentiation Background
Learn how automatic differentiation works.
Use Automatic Differentiation In Deep Learning Toolbox
How to use automatic differentiation in deep learning.

Enthaltene Beispiele

Solve Partial Differential Equations Using Deep Learning

Solve Burger's equation using deep learning.

Live Script öffnen

Solve Partial Differential Equation with L-BFGS Method and Deep Learning

Train a physics informed neural network (PINN) to numerically compute the solution of the Burger's equation by using the limited-memory BFGS (L-BFGS) algorithm.

Live Script öffnen