Deep Learning Custom Layers

Define custom layers for deep learning

You can define your own custom deep learning layer for your problem. You can specify a custom loss function using a custom output layers and define custom layers with or without learnable parameters. After defining a custom layer, you can check that the layer is valid, GPU compatible, and outputs correctly defined gradients.

Functions

expand all

Custom Layers

`functionLayer`	Function layer
`checkLayer`	Check validity of custom or function layer
`setLearnRateFactor`	Set learn rate factor of layer learnable parameter
`setL2Factor`	Set L2 regularization factor of layer learnable parameter
`getLearnRateFactor`	Get learn rate factor of layer learnable parameter
`getL2Factor`	Get L2 regularization factor of layer learnable parameter
`networkDataLayout`	Deep learning network data layout for learnable parameter initialization

Network Assembly

`findPlaceholderLayers`	Find placeholder layers in network architecture imported from Keras or ONNX
`replaceLayer`	Replace layer in layer graph or network
`assembleNetwork`	Assemble deep learning network from pretrained layers
`PlaceholderLayer`	Layer replacing an unsupported Keras or ONNX layer, or unsupported functionality from `functionToLayerGraph`

Topics

Custom Layers Overview

Define Custom Deep Learning Layers
Learn how to define custom deep learning layers.
Define Custom Deep Learning Intermediate Layers
Learn how to define custom deep learning intermediate layers.
Define Custom Deep Learning Output Layers
Learn how to define custom deep learning output layers.

Custom Intermediate Layers

Define Custom Deep Learning Layer with Learnable Parameters
This example shows how to define a PReLU layer and use it in a convolutional neural network.
Define Custom Deep Learning Layer with Multiple Inputs
This example shows how to define a custom weighted addition layer and use it in a convolutional neural network.
Define Custom Deep Learning Layer with Formatted Inputs
This example shows how to define a custom layer with formatted dlarray inputs.
Define Custom Recurrent Deep Learning Layer
This example shows how to define a peephole LSTM layer and use it in a neural network.
Specify Custom Layer Backward Function
This example shows how to define a PReLU layer and specify a custom backward function.
Custom Layer Function Acceleration
Accelerate custom layer forward and predict functions by caching and reusing traces.
Define Custom Deep Learning Layer for Code Generation
This example shows how to define a PReLU layer that supports code generation.
Assemble Network from Pretrained Keras Layers
This example shows how to import the layers from a pretrained Keras network, replace the unsupported layers with custom layers, and assemble the layers into a network ready for prediction.
Replace Unsupported Keras Layer with Function Layer
This example shows how to import the layers from a pretrained Keras network, replace the unsupported layers with function layers, and assemble the layers into a network ready for prediction.

Custom Output Layers

Define Custom Classification Output Layer
This example shows how to define a custom classification output layer with sum of squares error (SSE) loss and use it in a convolutional neural network.
Define Custom Regression Output Layer
This example shows how to define a custom regression output layer with mean absolute error (MAE) loss and use it in a convolutional neural network.
Specify Custom Output Layer Backward Loss Function
This example shows how to define a custom classification output layer with sum of squares error (SSE) loss and specify a custom backward loss function.

Network Composition and Nested Layers

Deep Learning Network Composition
Define custom layers containing layer graphs.
Define Nested Deep Learning Layer
This example shows how to define a nested deep learning layer.
Train Deep Learning Network with Nested Layers
This example shows how to train a network with nested layers.

Check Layer Validity

Check Custom Layer Validity
Learn how to check the validity of custom deep learning layers.