Main Content

Custom Training Loops

Customize deep learning training loops and loss functions for sequence and tabular data

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For networks that cannot be created using layer graphs, you can define custom networks as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.


alle erweitern

dlnetworkDeep learning network for custom training loops (Seit R2019b)
trainingProgressMonitorMonitor and plot training progress for deep learning custom training loops (Seit R2022b)
minibatchqueueCreate mini-batches for deep learning (Seit R2020b)
padsequencesPad or truncate sequence data to same length (Seit R2021a)
dlarrayDeep learning array for customization (Seit R2019b)
dlgradientCompute gradients for custom training loops using automatic differentiation (Seit R2019b)
dlfevalEvaluate deep learning model for custom training loops (Seit R2019b)
crossentropyCross-entropy loss for classification tasks (Seit R2019b)
l1lossL1 loss for regression tasks (Seit R2021b)
l2lossL2 loss for regression tasks (Seit R2021b)
huberHuber loss for regression tasks (Seit R2021a)
mseHalf mean squared error (Seit R2019b)
ctcConnectionist temporal classification (CTC) loss for unaligned sequence classification (Seit R2021a)
dlconvDeep learning convolution (Seit R2019b)
dltranspconvDeep learning transposed convolution (Seit R2019b)
lstmLong short-term memory (Seit R2019b)
gruGated recurrent unit (Seit R2020a)
attentionDot-product attention (Seit R2022b)
embedEmbed discrete data (Seit R2020b)
fullyconnectSum all weighted input data and apply a bias (Seit R2019b)
dlode45Deep learning solution of nonstiff ordinary differential equation (ODE) (Seit R2021b)
batchnormNormalize data across all observations for each channel independently (Seit R2019b)
crosschannelnormCross channel square-normalize using local responses (Seit R2020a)
groupnormNormalize data across grouped subsets of channels for each observation independently (Seit R2020b)
instancenormNormalize across each channel for each observation independently (Seit R2021a)
layernormNormalize data across all channels for each observation independently (Seit R2021a)
avgpoolPool data to average values over spatial dimensions (Seit R2019b)
maxpoolPool data to maximum value (Seit R2019b)
maxunpoolUnpool the output of a maximum pooling operation (Seit R2019b)
reluApply rectified linear unit activation (Seit R2019b)
leakyreluApply leaky rectified linear unit activation (Seit R2019b)
geluApply Gaussian error linear unit (GELU) activation (Seit R2022b)
softmaxApply softmax activation to channel dimension (Seit R2019b)
sigmoidApply sigmoid activation (Seit R2019b)


Custom Training Loops

Automatic Differentiation