incrementalClassificationNeuralNetwork

R2026b

Neural network classification model for incremental learning

Since R2026b

Description

The incrementalClassificationNeuralNetwork function creates an incrementalClassificationNeuralNetwork model object, which represents a neural network classification model for incremental learning.

Unlike other Statistics and Machine Learning Toolbox™ model objects, incrementalClassificationNeuralNetwork can be called directly. Also, you can specify learning options, such as performance metrics configurations and the objective solver, before fitting the model to data. After you create an incrementalClassificationNeuralNetwork object, it is prepared for incremental learning.

incrementalClassificationNeuralNetwork is best suited for incremental learning. For a traditional approach to training a neural network model for classification (such as creating a model by fitting it to data, performing cross-validation, tuning hyperparameters, and so on), see fitcnet.

Creation

You can create an incrementalClassificationNeuralNetwork model object in several ways:

• Call the function directly — Configure incremental learning options, or specify learner-specific options, by calling incrementalClassificationNeuralNetwork directly. This approach is best when you do not have data yet or you want to start incremental learning immediately.

• Convert a traditionally trained model — To initialize a model for incremental learning using the model parameters and hyperparameters of a trained model object, you can convert the traditionally trained model (ClassificationNeuralNetwork) to an incrementalClassificationNeuralNetwork model object by passing it to the incrementalLearner function.

• Call an incremental learning function — fit, updateMetrics, and updateMetricsAndFit accept a configured incrementalClassificationNeuralNetwork model object and data as input, and return an incrementalClassificationNeuralNetwork model object updated with information learned from the input model and data.

Syntax

Mdl = incrementalClassificationNeuralNetwork(Name=Value)

Description

Mdl = incrementalClassificationNeuralNetwork(Name=Value) returns a default incremental learning model object for neural network classification, Mdl, and sets properties and other options using name-value arguments. For example, incrementalClassificationNeuralNetwork(MaxNumClasses=5,LayerSizes=[50 30]) specifies a model with a maximum of five classes and two fully connected layers of size 50 and 30.

Properties of a default model contain placeholders for unknown model parameters. You must train a default model before you can track its performance or generate predictions from it.

When you call incrementalClassificationNeuralNetwork, you must specify at least one of the following name-value arguments or name-value argument combinations.

• MaxNumClasses — Maximum number of classes expected in the response data during incremental learning

• ClassNames — List of all class names expected in the response data during incremental learning

• LayerWeights, LayerBiases, and Activations — Weights, biases, and activation functions of the fully connected layers

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: incrementalClassificationNeuralNetwork(MaxNumClasses=5,NumPredictors=7) specifies a model with a maximum of five classes and seven predictors.

Classification Options

MaxNumClasses — Maximum number of classes

positive integer

Maximum number of classes expected in the response data during incremental learning, specified as a positive integer.

If you specify ClassNames, then MaxNumClasses must equal the number of class names.

Example: MaxNumClasses=5

Data Types: single | double

ClassNames — Unique class names

categorical array | string array | logical vector | numeric vector | cell array of character vectors

Names of the classes in the response variable, specified as a categorical array, string array, logical vector, numeric vector, or cell array of character vectors. ClassNames and the response data must have the same data type. This argument sets the ClassNames property.

ClassNames specifies the order of any input or output argument dimension that corresponds to the class order. For example, set ClassNames to specify the order of the dimensions of Cost and the column order of classification scores returned by predict.

If you do not specify ClassNames, the software infers the ClassNames property from the data during incremental learning.

Example: ClassNames=["cat","dog","bird"]

Data Types: single | double | logical | string | char | cell | categorical

Prior — Prior class probabilities

"empirical" (default) | "uniform" | numeric vector of nonnegative values

Prior class probabilities, specified as "empirical" (the default), "uniform", or a numeric vector of nonnegative values. If you do not specify ClassNames, then Prior must be "empirical". This argument sets the Prior property. incrementalClassificationNeuralNetwork stores the Prior value as a numeric vector.

Value	Description
"empirical"	Incremental learning functions infer prior class probabilities from the observed class relative frequencies in the response data during incremental training.
"uniform"	For each class, the prior probability is 1/K, where K is the number of classes.
numeric vector of nonnegative values	Custom, normalized prior probabilities. The order of the elements of Prior corresponds to the elements of the ClassNames property.

Example: Prior=[0.3 0.3 0.4]

Data Types: single | double | char | string

ScoreTransform — Score transformation function

"none" (default) | string scalar | character vector | function handle

Score transformation function describing how incremental learning functions transform raw response values, specified as a character vector, string scalar, or function handle. This argument sets the ScoreTransform property. incrementalClassificationNeuralNetwork stores the specified value as a character vector or function handle.

This table describes the available built-in functions for score transformation.

Value	Description
"doublelogit"	1/(1 + e–2x)
"invlogit"	log(x / (1 – x))
"ismax"	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to 0
"logit"	1/(1 + e–x)
"none" or "identity"	x (no transformation)
"sign"	–1 for x < 0 0 for x = 0 1 for x > 0
"symmetric"	2x – 1
"symmetricismax"	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to –1
"symmetriclogit"	2/(1 + e–x) – 1

For a MATLAB^® function or a function that you define, enter its function handle; for example, @function, where:

• function accepts an n-by-K matrix (the original scores) and returns a matrix of the same size (the transformed scores).

• n is the number of observations, and row j of the matrix contains the class scores of observation j.

• K is the number of classes, and column k is class ClassNames(k).

The default ScoreTransform value depends on how you create the model:

• If you convert a traditionally trained model to create Mdl, ScoreTransform is specified by the corresponding property of the traditionally trained model.

• The default "none" specifies returning posterior class probabilities.

Example: ScoreTransform="logit"

Data Types: char | string | function handle

Cost — Cost of misclassifying observation

square numeric matrix | structure array

Cost of misclassifying an observation, specified as a value in the following table, where c is the number of classes in the ClassNames property. This argument sets the Cost property.

Value	Description
c-by-c numeric matrix	Cost(i,j) is the cost of classifying an observation into class j when its true class is i, for classes ClassNames(i) and ClassNames(j). In other words, the rows correspond to the true class and the columns correspond to the predicted class. For example, Cost = [0 2;1 0] applies double the penalty for misclassifying ClassNames(1) than for misclassifying ClassNames(2).
Structure array	A structure array having two fields: ClassNames containing the class names, the same value as ClassNames ClassificationCosts containing the cost matrix, as previously described.

If you specify Cost, you must also specify the ClassNames argument.

The default is one of the following alternatives:

• An empty array [] when you specify MaxNumClasses

• A c-by-c matrix when you specify ClassNames, where Cost(i,j) = 1 for all i ≠ j, and Cost(i,j) = 0 for all i = j

Example: Cost=struct(ClassNames=["b","g"],ClassificationCosts=[0 2; 1 0])

Data Types: single | double | struct

Neural Network Options

Activations — Activation functions for fully connected layers

"relu" (default) | string array | cell array of character vectors

Activation functions for the fully connected layers of the neural network model, specified as one of the following values. The activation function for the final fully connected layer is always softmax. This argument sets the Activations property.

• String scalar or character vector — Use the specified activation function for each of the fully connected layers of the model, excluding the final fully connected layer. For more information, see Neural Network Structure.

• String array or cell array of character vectors — Use the ith element of Activations for the ith fully connected layer of the model. You cannot specify the activation function of the final fully connected layer.

Specify the activation functions using one or more of these values:

Value	Description
"relu"	Rectified linear unit (ReLU) function — Performs a threshold operation on each element of the input, where any value less than zero is set to zero, that is, $$f\left(x\right)=\{\begin{array}{cc}x,& x\ge 0\\ 0,& x<0\end{array}$$
"tanh"	Hyperbolic tangent (tanh) function — Applies the tanh function to each input element
"sigmoid"	Sigmoid function — Performs the following operation on each input element: $$f(x)=\frac{1}{1+e^{-x}}$$
"none"	Identity function — Returns each input element without performing any transformation, that is, f(x) = x

If you specify Activations, you must also specify either LayerSizes, or LayerWeights and LayerBiases.

Example: Activations="sigmoid"

Example: Activations=["relu","tanh"]

Data Types: char | string

LayerSizes — Output sizes of fully connected layers

10 (default) | positive integer vector

Output sizes of the fully connected layers in the neural network model, specified as a positive integer vector. This argument sets the LayerSizes property. The ith element of LayerSizes is the number of outputs in the ith fully connected layer of the network. You cannot specify the output size of the final connected layer, which has an output size equal to the number of classes. You cannot specify LayerSizes when you specify LayerWeights.

Example: LayerSizes=[50 30]

Data Types: single | double

LayerWeights — Weights for fully connected layers

cell array of numeric matrices

Weights for the fully connected layers, specified as a cell array of numeric matrices. This argument sets the LayerWeights property. The number of cell elements must equal the number of fully connected layers (numel(LayerSizes) + 1). The ith element contains the weight matrix for the ith fully connected layer. The first dimension of the last cell element determines the number of classes, and the second dimension of the first cell element determines the number of predictors. Layer weights are typically set during training or when converting from a traditionally trained model. You must specify LayerWeights, LayerBiases, and Activations together.

Data Types: cell

LayerWeightsInitializer — Initialization method for layer weights

"glorot" (default) | "he"

Initialization method for the layer weights, specified as one of these values:

• "glorot" — Initialize the weights with the Glorot initializer [1] (also known as the Xavier initializer). For each layer, the Glorot initializer independently samples from a uniform distribution with zero mean and variance 2/(I+O), where I is the input size and O is the output size for the layer.

• "he" — Initialize the weights with the He initializer [2]. For each layer, the He initializer samples from a normal distribution with zero mean and variance 2/I, where I is the input size for the layer.

The reset function uses the LayerWeightsInitializer function to initialize the layer weights.

Example: LayerWeightsInitializer="he"

Data Types: string | char

LayerBiases — Biases for fully connected layers

cell array of numeric column vectors

Biases for the fully connected layers, specified as a cell array of numeric column vectors. This argument sets the LayerBiases property. The number of cell elements must equal the number of layers (numel(LayerSizes) + 1). The ith element contains the weight matrix for the ith fully connected layer. The first dimension of the last cell element determines the number of classes, and the second dimension of the first cell element determines the number of predictors. Layer biases are typically set during training or when converting from a traditionally trained model. You must specify LayerWeights, LayerBiases, and Activations together.

Data Types: cell

LayerBiasesInitializer — Initialization method for layer biases

"zeros" (default) | "ones"

Initialization method for layer biases, specified as one of these values:

• "zeros" — Initialize the biases with a vector of zeros.

• "ones" — Initialize the biases with a vector of ones.

The reset function uses the LayerBiasesInitializer method to initialize the layer biases.

Example: LayerBiasesInitializer="ones"

Data Types: string | char

Training Parameters

TrainingOptions — Solver training options

TrainingOptionsMiniBatchLBFGS object | TrainingOptionsFREEREX object

Solver training options, specified as a TrainingOptionsMiniBatchLBFGS or TrainingOptionsFREEREX object returned by incrementalTrainingOptions. The training options specify the solver algorithm and its hyperparameters. This argument sets the TrainingOptions property. For more information about solver algorithms, see the Limited-Memory BFGS and FreeRex sections of the incrementalTrainingOptions reference page.

Example: TrainingOptions=incrementalTrainingOptions("freerex")

Standardize — Flag to standardize predictor data

false or 0 (default) | true or 1

Flag to standardize the predictor data, specified as a numeric or logical 0 (false) or 1 (true). If you set Standardize to true, then the software centers and scales each numeric predictor variable by the corresponding column mean and standard deviation.

If you specify Standardize=true and do not specify EstimationPeriod, the function sets the EstimationPeriod property value to 1000.

Example: Standardize=true

Data Types: logical

NumPredictors — Number of predictor variables

nonnegative integer

Number of predictor variables, specified as a nonnegative integer. This argument sets the NumPredictors property.

The default NumPredictors value depends on how you create the model:

• If you convert a traditionally trained model to create Mdl, then NumPredictors is specified by the corresponding property of the traditionally trained model.

• If you create Mdl by calling incrementalClassificationNeuralNetwork directly, you can specify NumPredictors by using name-value argument syntax. If you do not specify the value, then the default value is 0, and the incremental fitting functions infer NumPredictors from the predictor data during training.

Example: NumPredictors=6

Data Types: single | double

EstimationPeriod — Number of observations processed to estimate predictor means and standard deviations

nonnegative integer

Number of observations processed by the incremental fitting functions fit and updateMetricsAndFit to estimate the predictor means and standard deviations, specified as a nonnegative integer. This argument sets the EstimationPeriod property.

If you specify Standardize=true, the default value is 1000.

If you specify a positive value of EstimationPeriod when you create Mdl:

• The software sets EstimationPeriod=0 unless you specify Standardize=true.

• The incremental fitting functions use the first incoming EstimationPeriod observations to estimate the predictor means (Mu) and standard deviations (Sigma) prior to training the model.

• The software ignores observations that contain at least one missing value when processing observations during the estimation period.

• Estimation occurs only when Mdl.Mu is [] or an array of zeros, and Mdl.Sigma is [] or an array of ones.

• At the end of the estimation period, the software updates the Mu and Sigma properties of the model.

Example: EstimationPeriod=500

Data Types: single | double

Performance Metrics Options

Metrics — Model performance metrics to track during incremental learning

"mincost" (default) | "classiferror" | string vector | function handle | cell vector | structure array | "binodeviance" | "exponential" | "hinge" | "logit" | "quadratic" | "crossentropy"

Model performance metrics to track during incremental learning, in addition to minimal expected misclassification cost, specified as a built-in loss function name, string vector of names, function handle (for example, @metricName), structure array of function handles, or cell vector of names, function handles, or structure arrays. This argument sets the Metrics property.

When Mdl is warm (see IsWarm), the updateMetrics and updateMetricsAndFit functions track performance metrics in the Metrics property of Mdl.

The following table lists the built-in loss function names. You can specify more than one by using a string vector.

Name	Metrics Property Label	Description
"binodeviance"	BinomialDeviance	Binomial deviance
"classiferror"	ClassificationError	Misclassification error rate
"exponential"	ExponentialLoss	Exponential
"hinge"	HingeLoss	Hinge
"logit"	LogitLoss	Logistic
"mincost"	MinimalCost	Minimal expected misclassification cost (for classification scores that are posterior probabilities). incrementalClassificationNeuralNetwork always tracks this metric.
"quadratic"	QuadraticLoss	Quadratic
"crossentropy"	CrossEntropyLoss	Cross-entropy

For more details on the built-in loss functions, see loss.

Example: Metrics=["classiferror" "logit"]

To specify a custom function that returns a performance metric, use function handle notation. The function must have this form.

metric = customMetric(C,S,Cost)

• The output argument metric is an n-by-1 numeric vector, where each element is the loss of the corresponding observation in the data processed by the incremental learning functions during a learning cycle.

• You specify the function name (here, customMetric).

• C is an n-by-K logical matrix with rows indicating the class to which the corresponding observation belongs, where K is the number of classes. The column order corresponds to the class order in the ClassNames property. Create C by setting C(p,q) = 1, if observation p is in class q, for each observation in the specified data. Set the other element in row p to 0.

• S is an n-by-K numeric matrix of predicted classification scores. S is similar to the Posterior output of predict, where rows correspond to observations in the data and the column order corresponds to the class order in the ClassNames property. S(p,q) is the classification score of observation p being classified in class q.

• Cost is a K-by-K numeric matrix of misclassification costs. See the Cost name-value argument.

To specify multiple custom metrics and assign a custom name to each, use a structure array. To specify a combination of built-in and custom metrics, use a cell vector.

Example: Metrics=struct(Metric1=@customMetric1,Metric2=@customMetric2)

Example: Metrics={@customMetric1 @customMetric2 "logit" struct(Metric3=@customMetric3)}

updateMetrics and updateMetricsAndFit store specified metrics in a table in the Metrics property. The data type of Metrics determines the row names of the table.

"Metrics" Value Data Type	Description of Metrics Property Row Name	Example
String or character vector	Name of corresponding built-in metric	Row name for "classiferror" is "ClassificationError"
Structure array	Field name	Row name for struct(Metric1=@customMetric1) is "Metric1"
Function handle to function stored in a program file	Name of function	Row name for @customMetric is "customMetric"
Anonymous function	CustomMetric_j, where j is metric j in Metrics	Row name for @(C,S,Cost)customMetric(C,S,Cost)... is CustomMetric_1

For more details on performance metrics options, see Performance Metrics.

Data Types: char | string | struct | cell | function_handle

MetricsWarmupPeriod — Number of observations to fit during metrics warm-up period

1000 (default) | nonnegative integer

Number of observations to fit during the metrics warm-up period, specified as a nonnegative integer. This argument sets the MetricsWarmupPeriod property. The metrics warm-up period takes place after the solver tuning period and estimation period (if specified). The metrics warm-up period is completed when the incremental fitting functions have processed MetricsWarmupPeriod observations and at least one observation from each expected class. After the metrics warm-up period, the model object is warm and the incremental fitting functions compute and store performance metrics.

For more details, see Incremental Training Periods.

Example: MetricsWarmupPeriod=50

Data Types: single | double

MetricsWindowSize — Number of observations to use to compute window performance metrics

200 (default) | positive integer

Number of observations to use to compute window performance metrics, specified as a positive integer. This argument sets the MetricsWindowSize property.

For more details on performance metrics options, see Performance Metrics.

Example: MetricsWindowSize=250

Data Types: single | double

Properties

You can set most properties by using name-value pair argument syntax only when you call incrementalClassificationNeuralNetwork directly. You can set some properties when you call incrementalLearner to convert a traditionally trained model. You cannot set the properties IsWarm, Mu, Sigma, OutputLayerActivation, and NumTrainingObservations.

Classification Model Properties

ClassNames — All unique class labels

cell array of character vectors

This property is read-only after object creation.

All unique class labels expected in the response data during incremental learning, specified as a cell array of character vectors.

Data Types: cell

Cost — Cost of misclassifying observation

square numeric matrix | empty array []

This property is read-only after object creation.

Cost of misclassifying an observation, specified as an array.

If you specify Cost, its value sets Cost. If you specify a structure array, then Cost is the value of the ClassificationCosts field.

If you convert a traditionally trained model to create Mdl, then Cost is the Cost property of the traditionally trained model.

Data Types: double

NumPredictors — Number of predictor variables

nonnegative numeric scalar

This property is read-only after object creation.

Number of predictor variables, specified as a nonnegative numeric scalar.

The default NumPredictors value depends on how you create the model:

• If you convert a traditionally trained model to create Mdl, NumPredictors is specified by the corresponding property of the traditionally trained model.

• If you create Mdl by calling incrementalClassificationNeuralNetwork directly, you can specify NumPredictors by using name-value argument syntax. If you do not specify the value, then the default value is 0, and incremental fitting functions infer NumPredictors from the predictor data during training.

Data Types: double

Prior — Prior class probabilities

numeric vector

This property is read-only after object creation.

Prior class probabilities, specified as "empirical", "uniform", or a numeric vector. incrementalClassificationNeuralNetwork stores the Prior value as a numeric vector.

Value	Description
"empirical"	Incremental learning functions infer prior class probabilities from the observed class relative frequencies in the response data during incremental training.
"uniform"	For each class, the prior probability is 1/K, where K is the number of classes.
numeric vector	Custom, normalized prior probabilities. The order of the elements of Prior corresponds to the elements of the ClassNames property.

The default Prior value depends on how you create the model:

• If you convert a traditionally trained model to create Mdl, Prior is specified by the corresponding property of the traditionally trained model.

• Otherwise, the default value is "empirical".

Data Types: double

ScoreTransform — Score transformation function

character vector | function handle

This property is read-only after object creation.

Score transformation function describing how incremental learning functions transform raw response values, specified as a character vector, string scalar, or function handle. incrementalClassificationNeuralNetwork stores the specified value as a character vector or function handle.

This table describes the available built-in functions for score transformation.

Value	Description
"doublelogit"	1/(1 + e–2x)
"invlogit"	log(x / (1 – x))
"ismax"	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to 0
"logit"	1/(1 + e–x)
"none" or "identity"	x (no transformation)
"sign"	–1 for x < 0 0 for x = 0 1 for x > 0
"symmetric"	2x – 1
"symmetricismax"	Sets the score for the class with the largest score to 1, and sets the scores for all other classes to –1
"symmetriclogit"	2/(1 + e–x) – 1

For a MATLAB function or a function that you define, enter its function handle; for example, @function, where:

• function accepts an n-by-K matrix (the original scores) and returns a matrix of the same size (the transformed scores).

• n is the number of observations, and row j of the matrix contains the class scores of observation j.

• K is the number of classes, and column k is class ClassNames(k).

The default ScoreTransform value depends on how you create the model:

• If you convert a traditionally trained model to create Mdl, ScoreTransform is specified by the corresponding property of the traditionally trained model.

• The default "none" specifies returning posterior class probabilities.

Data Types: char | function_handle

Training Properties

Mu — Predictor means

Read-only: numeric vector | []

This property is read-only.

Predictor means, represented as a numeric vector.

• When you create Mdl and specify NumPredictors=0 or Standardize=false (the default), then Mu is an empty array [].

• When you create Mdl and set Standardize=true, and Mu is [] or an array of zeros, then the incremental fit function calculates the predictor variable means using all data points that do not have any missing values. At the end of the estimation period specified by EstimationPeriod, Mu is a NumPredictors-by-1 vector that contains the predictor means.

Data Types: double

Sigma — Predictor standard deviations

Read-only: numeric vector | []

This property is read-only.

Predictor standard deviations, represented as a numeric vector.

• When you create Mdl and specify NumPredictors=0 or Standardize=false (the default), then Sigma is an empty array [].

• When you create Mdl and set Standardize=true, and Sigma is [] or an array of zeros, then the incremental fit function calculates the predictor variable standard deviations using all data points that do not have any missing values. At the end of the estimation period specified by EstimationPeriod, Sigma is a NumPredictors-by-1 vector that contains the predictor standard deviations.

Data Types: double

EstimationPeriod — Number of observations processed to estimate predictor means and standard deviations

nonnegative integer

This property is read-only after object creation.

Number of observations processed by the incremental model to estimate the predictor means and standard deviations, represented as a nonnegative integer. If you specify Standardize=true when you create Mdl, the default value is 1000. Otherwise, the default value is 0.

If EstimationPeriod > 0:

• The incremental fitting functions use EstimationPeriod observations to estimate the predictor means (Mu) and standard deviations (Sigma) prior to training the model.

• The software ignores observations that contain at least one missing value when processing observations during the estimation period.

• The estimation period takes place before the solver tuning period and the metrics warm-up period (if specified).

For more information, see Incremental Training Periods.

Data Types: double

Solver — Objective function minimization technique

"minibatch-lbfgs" (default) | "freerex"

This property is read-only after object creation.

Objective function minimization technique, specified as one of the following values:

Value	Solver Name	More Information
"minibatch-lbfgs"	Mini-Batch Limited-memory Broyden–Fletcher–Goldfarb–Shanno (LBFGS)	Limited-Memory BFGS
"freerex"	FreeRex	FreeRex

If you convert a traditionally trained model to create Mdl, then Solver is "minibatch-lbfgs".

Data Types: string

TrainingOptions — Solver training options

TrainingOptionsMiniBatchLBFGS object | TrainingOptionsFREEREX object

This property is read-only after object creation.

Solver training options, specified as a TrainingOptionsMiniBatchLBFGS or TrainingOptionsFREEREX object. If you convert a traditionally trained model to create Mdl, the TrainingOptions name-value argument of the incrementalLearner function sets this property.

NumTrainingObservations — Number of observations fit to incremental model

Read-only: 0 (default) | nonnegative numeric scalar

This property is read-only.

Number of observations fit to the incremental model Mdl, represented as a nonnegative numeric scalar. NumTrainingObservations increases when you pass Mdl and training data to fit or updateMetricsAndFit.

Note

If you convert a traditionally trained model to create Mdl, incrementalClassificationNeuralNetwork does not add the number of observations fit to the traditionally trained model to NumTrainingObservations.

Data Types: double

Performance Metrics Properties

IsWarm — Flag indicating whether model tracks performance metrics

false or 0 | true or 1

Flag indicating whether the incremental model tracks performance metrics in the Metrics property, specified as logical 0 (false) or 1 (true).

When you create Mdl with the incrementalClassificationNeuralNetwork function, the model is warm(IsWarm is true) when the following are true:

• You specify ClassNames, Prior, and NumPredictors

• There is no estimation period, solver tuning period, or metrics warm-up period

When you create Mdl with the incrementalLearner function, the model is warm when MetricsWarmupPeriod is 0 and either of the following is true:

• Solver is "freerex"

• Solver is "minibatch-lbfgs" and Mdl.TrainingOptions.TuningPeriod is 0.

Otherwise, the incremental model becomes warm after the estimation period, solver tuning period, and metrics warm-up period (if specified). For more information, see Incremental Training Periods.

Data Types: logical

Metrics — Model performance metrics

table

Model performance metrics updated during incremental learning by updateMetrics and updateMetricsAndFit, specified as a table with two columns.

The table contains a row for the MinimalCost metric, and a row for each metric specified by the Metrics name-value argument. The supported metrics are as follows:

Label	Metrics Argument Value
MinimalCost	"mincost"
ClassificationError	"classiferror"
HingeLoss	"hinge"
QuadraticLoss	"quadratic"
BinomialDeviance	"binodeviance"
ExponentialLoss	"exponential"
LogitLoss	"logit"
CrossEntropyLoss	"crossentropy"

The columns of Metrics are labeled Cumulative and Window.

• Cumulative: Element j is the model performance, as measured by metric j, from the time the model became warm (IsWarm is 1).

• Window: Element j is the model performance, as measured by metric j, evaluated over all observations within the window specified by the MetricsWindowSize property. The software updates Window after it processes MetricsWindowSize observations.

If you convert a traditionally trained model to create Mdl, the Metrics name-value argument of the incrementalLearner function sets this property.

Data Types: table

MetricsWarmupPeriod — Number of observations in metrics warm-up period

0 (default) | nonnegative integer

This property is read-only after object creation.

Number of observations in the metrics warm-up period, specified as a nonnegative integer.

If you convert a traditionally trained model to create Mdl, the MetricsWindowSize name-value argument of the incrementalLearner function sets this property.

For more details about the metrics warm-up period, see Incremental Training Periods.

Data Types: double

MetricsWindowSize — Number of observations to use to compute window performance metrics

positive integer

This property is read-only after object creation.

Number of observations to use to compute window performance metrics, specified as a positive integer.

The default MetricsWindowSize value depends on how you create the model:

• If you convert a traditionally trained model to create Mdl, the MetricsWindowSize name-value argument of the incrementalLearner function sets this property. The default value of the argument is 200.

• Otherwise, the default value is 200.

For more details on performance metrics options, see Performance Metrics.

Data Types: double

Neural Network Properties

LayerSizes — Sizes of fully connected layers

vector of positive integers

This property is read-only after object creation.

Output sizes of the fully connected layers in the neural network model, specified as a positive integer vector. The ith element of LayerSizes is the number of outputs in the ith fully connected layer of the network.

Data Types: double

LayerWeights — Learned layer weights

cell array

This property is read-only after object creation.

Weights for the fully connected layers, specified as a cell array of numeric matrices. The ith element contains the weight matrix for the ith fully connected layer.

Data Types: cell

LayerBiases — Learned layer biases

cell array

This property is read-only after object creation.

Biases for the fully connected layers, specified as a cell array of numeric column vectors. The ith element contains the weight matrix for the ith fully connected layer.

Data Types: cell

Activations — Activation functions for fully connected layers

"relu" | "tanh" | "sigmoid" | "none" | string array

This property is read-only after object creation.

Activation functions for the fully connected layers of the neural network model, specified as a string or a string array containing one or more of the following values. The activation function for the final fully connected layer is always softmax.

Value	Description
"relu"	Rectified linear unit (ReLU) function — Performs a threshold operation on each element of the input, where any value less than zero is set to zero, that is, $$f\left(x\right)=\{\begin{array}{cc}x,& x\ge 0\\ 0,& x<0\end{array}$$
"tanh"	Hyperbolic tangent (tanh) function — Applies the tanh function to each input element
"sigmoid"	Sigmoid function — Performs the following operation on each input element: $$f(x)=\frac{1}{1+e^{-x}}$$
"none"	Identity function — Returns each input element without performing any transformation, that is, f(x) = x

Data Types: string

OutputLayerActivation — Activation function for final fully connected layer

"softmax"

This property is read-only after object creation.

Activation function for the final fully connected layer, specified as "softmax". The softmax function takes each input x_i and returns the following, where K is the number of classes in the response variable:

$$f(x_i)=\frac{\exp (x_i)}{{\displaystyle \sum _{j=1}^K\exp (x_j)}}.$$

The results correspond to the predicted classification scores (or posterior probabilities).

Object Functions

fit	Train neural network model for incremental learning
updateMetrics	Update performance metrics in neural network incremental learning model given new data
updateMetricsAndFit	Update performance metrics in neural network incremental learning model given new data and train model
loss	Loss of neural network incremental learning model on batch of data
perObservationLoss	Per observation classification error of model for incremental learning
predict	Predict responses for new observations from neural network incremental learning model
reset	Reset incremental classification model
dlnetwork (Deep Learning Toolbox)	Deep learning neural network

Examples

Create Incremental Learner with Little Prior Information

When you create a neural network classification model for incremental learning, you can specify the maximum number of classes that you expect the model to process (MaxNumClasses name-value argument). As you fit the model to incoming batches of data by using an incremental fitting function, the model collects new classes in its ClassNames property. If the specified maximum number of classes is inaccurate, one of the following occurs:

• Before an incremental fitting function processes the expected maximum number of classes, the model is not warm. Consequently, the updateMetrics and updateMetricsAndFit functions do not measure performance metrics.

• If the number of classes exceeds the maximum expected, the incremental fitting function issues an error.

This example shows how to create a neural network classification model for incremental learning when the only information you specify is the expected maximum number of classes in the data. Also, the example illustrates the consequences when incremental fitting functions process all expected classes early and late in the sample.

For this example, consider training a device to predict whether a subject is sitting, standing, walking, running, or dancing based on biometric data measured on the subject. Therefore, the device has a maximum of five classes from which to choose.

Process Expected Maximum Number of Classes Early in Sample

Create an incremental neural network model for multiclass learning. Specify a maximum of five classes in the data, and standardize the predictor values.

MdlEarly = incrementalClassificationNeuralNetwork(MaxNumClasses=5)

MdlEarly = 
  incrementalClassificationNeuralNetwork

                   IsWarm: 0
                  Metrics: [1×2 table]
               ClassNames: [1×0 double]
           ScoreTransform: 'none'
               LayerSizes: 10
              Activations: "relu"
    OutputLayerActivation: "softmax"
                   Solver: "minibatch-lbfgs"


  Properties, Methods

MdlEarly is an incrementalClassificationNeuralNetwork model object. MdlEarly must be fit to data before you can use it to perform any other operations.

Display the default training period values associated with the model object.

MdlEarly.TrainingOptions.TuningPeriod

ans = 
1000

MdlEarly.MetricsWarmupPeriod

ans = 
1000

When you use fit and updateMetricsAndFit to fit the model, these functions:

• Use the first incoming 1000 observations to tune the initial learning rate for the solver

• Process the next 1000 observations during the warm-up period

Once the model has been fit to all expected classes and at least 2000 observations, the model is warm, and the fit and updateMetricsAndFit functions compute and store performance metrics.

Load the human activity data set. Randomly shuffle the data.

load humanactivity
n = numel(actid);
rng(1); % For reproducibility
idx = randsample(n,n);
X = feat(idx,:);
Y = actid(idx);

For details on the data set, enter Description at the command line.

Fit the incremental model to the training data by using the updateMetricsAndFit function. Simulate a data stream by processing chunks of 50 observations at a time. At each iteration:

• Process 50 observations.

• Overwrite the previous incremental model with a new one fitted to the incoming observations.

• Store the cumulative metrics and the window metrics to see how they evolve during incremental learning.

% Preallocation
numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);
mc = array2table(zeros(nchunk,2),VariableNames=["Cumulative" "Window"]);
IsWarm = zeros(nchunk+1,1);    
% Incremental learning
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend = min(n,numObsPerChunk*j);
    idx = ibegin:iend;    
    MdlEarly = updateMetricsAndFit(MdlEarly,X(idx,:),Y(idx));
    mc{j,:} = MdlEarly.Metrics{"MinimalCost",:};
    IsWarm(j + 1) = MdlEarly.IsWarm;
end

MdlEarly is an incrementalClassificationNeuralNetwork model object trained on all the data in the stream. During incremental learning and after the model is warm, updateMetricsAndFit checks the performance of the model on the incoming observations, and then fits the model to those observations.

To see how the IsWarm property and performance metrics evolve during training, plot them on separate tiles.

t = tiledlayout(2,1);
nexttile
plot(IsWarm)
ylabel("IsWarm")
xlim([0 nchunk])
ylim([0 1.1])
xline((MdlEarly.TrainingOptions.TuningPeriod + ...
    MdlEarly.MetricsWarmupPeriod)/numObsPerChunk,"r-.")
nexttile
h = plot(mc.Variables);
xlim([0 nchunk])
ylabel("Minimal Cost")
xline((MdlEarly.TrainingOptions.TuningPeriod + ...
    MdlEarly.MetricsWarmupPeriod)/numObsPerChunk,"r-.")
legend(h,mc.Properties.VariableNames)
xlabel(t,"Iteration")

The plots indicate that updateMetricsAndFit performs the following actions:

• Compute the performance metrics after the tuning and metrics warm-up periods (red vertical line) only.

• Compute the cumulative metrics during each iteration.

• Compute the window metrics after processing 200 observations (4 iterations).

Process Expected Maximum Number of Classes Late in Sample

Create a different neural network model for incremental learning for the objective.

MdlLate = incrementalClassificationNeuralNetwork(MaxNumClasses=5, ...
    Standardize=true)

MdlLate = 
  incrementalClassificationNeuralNetwork

                   IsWarm: 0
                  Metrics: [1×2 table]
               ClassNames: [1×0 double]
           ScoreTransform: 'none'
               LayerSizes: 10
              Activations: "relu"
    OutputLayerActivation: "softmax"
                   Solver: "minibatch-lbfgs"


  Properties, Methods

Move all observations labeled with class 5 to the end of the sample.

idx5 = Y == 5;
Xnew = [X(~idx5,:); X(idx5,:)];
Ynew = [Y(~idx5) ;Y(idx5)];

Fit the incremental model and plot the results.

mcnew = array2table(zeros(nchunk,2),VariableNames=["Cumulative" "Window"]);

for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend   = min(n,numObsPerChunk*j);
    idx = ibegin:iend;    
    MdlLate = updateMetricsAndFit(MdlLate,Xnew(idx,:),Ynew(idx));
    mcnew{j,:} = MdlLate.Metrics{"MinimalCost",:};
 end

figure
h = plot(mcnew.Variables);
xlim([0 nchunk]);
ylabel("Minimal Cost")
xline((MdlLate.TrainingOptions.TuningPeriod + ...
    MdlLate.MetricsWarmupPeriod)/numObsPerChunk,"r-.")
xline(sum(~idx5)/numObsPerChunk,"g-.")
legend(h,mcnew.Properties.VariableNames,Location="best")
xlabel("Iteration")

The updateMetricsAndFit function trains the model throughout incremental learning, but the function starts tracking performance metrics only after the model is fit to all expected number of classes (the green vertical line).

Specify All Class Names

Create an incremental neural network model when you know all the class names in the data.

Consider training a device to predict whether a subject is sitting, standing, walking, running, or dancing based on biometric data measured on the subject. The class names map 1 through 5 to an activity.

Create an incremental neural network model for multiclass learning. Specify the class names.

classnames = 1:5;
Mdl = incrementalClassificationNeuralNetwork(ClassNames=classnames)

Mdl = 
  incrementalClassificationNeuralNetwork

                   IsWarm: 0
                  Metrics: [1×2 table]
               ClassNames: [1 2 3 4 5]
           ScoreTransform: 'none'
               LayerSizes: 10
              Activations: "relu"
    OutputLayerActivation: "softmax"
                   Solver: "minibatch-lbfgs"


  Properties, Methods

Mdl is an incrementalClassificationNeuralNetwork model object.

Mdl must be fit to data before you can use it to perform any other operations.

Load the human activity data set. Randomly shuffle the data.

load humanactivity
n = numel(actid);
rng(1) % For reproducibility
idx = randsample(n,n);
X = feat(idx,:);
Y = actid(idx);

For details on the data set, enter Description at the command line.

Fit the incremental model to the training data by using the updateMetricsAndFit function. Simulate a data stream by processing chunks of 50 observations at a time. At each iteration:

• Process 50 observations.

• Overwrite the previous incremental model with a new one fitted to the incoming observations.

% Preallocation
numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);

% Incremental learning
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend   = min(n,numObsPerChunk*j);
    idx = ibegin:iend;    
    Mdl = updateMetricsAndFit(Mdl,X(idx,:),Y(idx));
end

Specify Neural Network Parameters

Load the human activity data set. Randomly shuffle the data.

load humanactivity
n = numel(actid);
idx = randsample(n,n);
X = feat(idx,:);
Y = actid(idx);

Fit an initial classification neural network model object to the first 1000 observations using the fitcnet function..

Mdl = fitcnet(X(1:1000,:),Y(1:1000));

Create an incrementalClassificationNeuralnetwork model object by specifying the network layer activations, weights, and biases from the initial model.

incrMdl = incrementalClassificationNeuralNetwork(Activations=Mdl.Activations, ...
    LayerWeights=Mdl.LayerWeights,LayerBiases=Mdl.LayerBiases)

incrMdl = 
  incrementalClassificationNeuralNetwork

                   IsWarm: 0
                  Metrics: [1×2 table]
               ClassNames: [1×0 double]
           ScoreTransform: 'none'
               LayerSizes: 10
              Activations: "relu"
    OutputLayerActivation: "softmax"
                   Solver: "minibatch-lbfgs"


  Properties, Methods

The model object incrMdl contains the network parameters from the initial model and is ready for incremental learning using the fit and updateMetricsAndFit functions.

Configure Incremental Learning Options

Load the human activity data set. Randomly shuffle the data.

load humanactivity
n = numel(actid);
rng(0,"twister"); % For reproducibility
idx = randsample(n,n);
X = feat(idx,:);
Y = actid(idx);

The class names map 1 through 5 to an activity—sitting, standing, walking, running, or dancing, respectively—based on biometric data measured on the subject. For details on the data set, enter Description at the command line.

Create an incremental neural network model for multiclass learning. Configure the model as follows:

• Specify a metrics warm-up period of 5000 observations.

• Specify a metrics window size of 500 observations.

• Standardize the predictor data and specify an estimation period of 1000 observations.

• Use the mini-batch LBFGS solver and a solver tuning period of 500 observations.

• Double the penalty to the classifier when it mistakenly classifies class 2.

• Track the classification error and minimal cost to measure the performance of the model. You do not have to specify mincost for Metrics because incrementalClassificationNeuralNetwork always tracks this metric.

C = ones(5) - eye(5);
C(2,[1 3 4 5]) = 2;
Mdl = incrementalClassificationNeuralNetwork(ClassNames=1:5, ...
    MetricsWarmupPeriod=5000,MetricsWindowSize=500, ...
    Standardize=true,EstimationPeriod=1000, ...
    TrainingOptions=incrementalTrainingOptions("minibatch-lbfgs", ...
    TuningPeriod=500),Cost=C,Metrics="classiferror")

Mdl = 
  incrementalClassificationNeuralNetwork

                   IsWarm: 0
                  Metrics: [2×2 table]
               ClassNames: [1 2 3 4 5]
           ScoreTransform: 'none'
               LayerSizes: 10
              Activations: "relu"
    OutputLayerActivation: "softmax"
                   Solver: "minibatch-lbfgs"


  Properties, Methods

Mdl is an incrementalClassificationNeuralNetwork model object configured for incremental learning.

Fit the incremental model to the rest of the data by using the updateMetricsAndFit function. At each iteration:

• Simulate a data stream by processing a chunk of 50 observations.

• Overwrite the previous incremental model with a new one fitted to the incoming observations.

• Store the standard deviation of the first predictor variable ${\sigma }_1$, the cumulative metrics, and the window metrics to see how they evolve during incremental learning.

% Preallocation
numObsPerChunk = 50;
nchunk = floor(n/numObsPerChunk);
ce = array2table(zeros(nchunk,2),VariableNames=["Cumulative" "Window"]);
mc = array2table(zeros(nchunk,2),VariableNames=["Cumulative" "Window"]);
sigma1 = zeros(nchunk+1,1);    

% Incremental fitting
for j = 1:nchunk
    ibegin = min(n,numObsPerChunk*(j-1) + 1);
    iend   = min(n,numObsPerChunk*j);
    idx = ibegin:iend;    
    Mdl = updateMetricsAndFit(Mdl,X(idx,:),Y(idx));
    ce{j,:} = Mdl.Metrics{"ClassificationError",:};
    mc{j,:} = Mdl.Metrics{"MinimalCost",:};
    sigma1(j) = Mdl.Sigma(1);
end

Mdl is an incrementalClassificationNeuralNetwork model object trained on all the data in the stream. During incremental learning and after the model is warmed up, updateMetricsAndFit checks the performance of the model on the incoming observations, and then fits the model to those observations.

To see how the performance metrics and ${\sigma }_1$ evolve during training, plot them on separate tiles.

tiledlayout(2,2)
nexttile
plot(sigma1)
ylabel("\sigma_{1}")
xlim([0 nchunk]);
xline(Mdl.EstimationPeriod/numObsPerChunk,"b--")
xlabel("Iteration")
nexttile
h = plot(ce.Variables);
xlim([0 nchunk])
ylabel("Classification Error")
xline((Mdl.EstimationPeriod + Mdl.TrainingOptions.TuningPeriod + ...
 Mdl.MetricsWarmupPeriod)/numObsPerChunk,"r-.")
legend(h,ce.Properties.VariableNames)
xlabel("Iteration")
nexttile
h = plot(mc.Variables);
xlim([0 nchunk]);
ylabel("Minimal Cost")
xline((Mdl.EstimationPeriod + Mdl.TrainingOptions.TuningPeriod + ...
    Mdl.MetricsWarmupPeriod)/numObsPerChunk,"r-.")
legend(h,mc.Properties.VariableNames)
xlabel("Iteration")

The plots indicate that updateMetricsAndFit performs the following actions:

• Fit ${\sigma }_1$ after the estimation period (blue vertical line).

• Compute the performance metrics after the estimation period, tuning period, and metrics warm-up period (red vertical line) only.

• Compute the cumulative metrics during each iteration.

• Compute the window metrics after processing 500 observations (10 iterations).

More About

Incremental Learning

Incremental learning, or online learning, is a branch of machine learning concerned with processing incoming data from a data stream, possibly given little to no knowledge of the distribution of the predictor variables, aspects of the prediction or objective function (including tuning parameter values), or whether the observations are labeled. Incremental learning differs from traditional machine learning, where enough labeled data is available to fit to a model, perform cross-validation to tune hyperparameters, and infer the predictor distribution.

Given incoming observations, an incremental learning model processes data in any of the following ways, but usually in this order:

• Predict labels.

• Measure the predictive performance.

• Check for structural breaks or drift in the model.

• Fit the model to the incoming observations.

For more details, see Incremental Learning Overview.

Neural Network Structure

The default neural network classifier has the following layer structure.

Structure	Description
	Input — This layer corresponds to the predictor data in Tbl or X.
	First fully connected layer — This layer has 10 outputs by default. You can widen the layer or add more fully connected layers to the network by specifying the LayerSizes name-value argument. You can find the weights and biases for this layer in the Mdl.LayerWeights{1} and Mdl.LayerBiases{1} properties of Mdl, respectively.
	ReLU activation function — fitcnet applies this activation function to the first fully connected layer. You can change the activation function by specifying the Activations name-value argument.
	Final fully connected layer — This layer has K outputs, where K is the number of classes in the response variable. You can find the weights and biases for this layer in the Mdl.LayerWeights{end} and Mdl.LayerBiases{end} properties of Mdl, respectively.
	Softmax function (for both binary and multiclass classification) — fitcnet applies this activation function to the final fully connected layer. The function takes each input xi and returns the following, where K is the number of classes in the response variable: $$f(x_i)=\frac{\exp (x_i)}{{\displaystyle \sum _{j=1}^K\exp (x_j)}}.$$ The results correspond to the predicted classification scores (or posterior probabilities).

Standardize Data

If incremental learning functions are configured to standardize predictor variables, they do so using the means and standard deviations stored in the Mu and Sigma properties of the incremental learning model Mdl.

• When you set Standardize=true and a positive estimation period (see EstimationPeriod), and Mdl.Mu and Mdl.Sigma are empty, incremental fitting functions estimate means and standard deviations using the estimation period observations.

• When incremental fitting functions estimate predictor means and standard deviations, the functions compute weighted means and weighted standard deviations using the estimation period observations. Specifically, the functions standardize predictor j (x_j) using

  $$x_j^{\ast }=\frac{x_j-{\mu }_j^{\ast }}{{\sigma }_j^{\ast }}.$$

  • x_j is predictor j, and x_jk is observation k of predictor j in the estimation period.

  • $${\mu }_j^{\ast }=\frac{1}{{\displaystyle \sum _k{w}_k^{\ast }}}{\displaystyle \sum _k{w}_k^{\ast }{x}_{jk}}.$$

  • $${\left({\sigma }_j^{\ast }\right)}^2=\frac{1}{{\displaystyle \sum _k{w}_k^{\ast }}}{\displaystyle \sum _k{w}_k^{\ast }{\left(x_{jk}-{\mu }_j^{\ast }\right)}^2}.$$

  • $$w_j^{\ast }=\frac{w_j}{{\displaystyle \sum _{\forall j\in \text{Class }k}{w}_j}}{p}_k,$$

    • p_k is the prior probability of class k (Prior property of the incremental model).

    • w_j is observation weight j.

Performance Metrics

• The updateMetrics and updateMetricsAndFit functions track model performance metrics from new data only when the incremental model is warm (IsWarm property is true).

  The Metrics property of the incremental model stores two forms of each performance metric as variables (columns) of a table, Cumulative and Window, with individual metrics in rows. When the incremental model is warm, updateMetrics and updateMetricsAndFit update the metrics at the following frequencies:

  • Cumulative — The functions compute cumulative metrics since the start of model performance tracking. The functions update metrics every time you call the functions and base the calculation on the entire supplied data set.

  • Window — The functions compute metrics based on all observations within a window determined by the MetricsWindowSize name-value argument. MetricsWindowSize also determines the frequency at which the software updates Window metrics. For example, if MetricsWindowSize is 20, the functions compute metrics based on the last 20 observations in the supplied data (X((end – 20 + 1):end,:) and Y((end – 20 + 1):end)).

    Incremental functions that track performance metrics within a window use the following process:

    1. Store a buffer of length MetricsWindowSize for each specified metric, and store a buffer of observation weights.

    2. Populate elements of the metrics buffer with the model performance based on batches of incoming observations, and store corresponding observation weights in the weights buffer.

    3. When the buffer is full, overwrite Mdl.Metrics.Window with the weighted average performance in the metrics window. If the buffer overfills when the function processes a batch of observations, the latest incoming MetricsWindowSize observations enter the buffer, and the earliest observations are removed from the buffer. For example, suppose MetricsWindowSize is 20, the metrics buffer has 10 values from a previously processed batch, and 15 values are incoming. To compose the length 20 window, the functions use the measurements from the 15 incoming observations and the latest 5 measurements from the previous batch.

The incremental fitting functions omit an observation with a NaN score when computing the Cumulative and Window performance metric values.

Incremental Training Periods

When you train an incremental neural network model with the incremental fitting functions fit and updateMetricsAndFit, then depending on the model's properties, up to three incremental training periods can occur in the following order: the estimation period, the solver tuning period, and the metrics warm-up period. Following these periods, the incremental model is warm and the incremental fitting functions track model performance metrics from new data.

Estimation Period

During the estimation period, fit does not fit the model, and updateMetricsAndFit does not fit the model or update the performance metrics. The incremental fitting functions use the first incoming EstimationPeriod observations to estimate the predictor means and standard deviation hyperparameters required to standardize the data during incremental training. The fitting functions store the hyperparameter estimates in the Mu and Sigma properties of Mdl.

The hyperparameters are estimated when both of these conditions apply:

• Incremental fitting functions are configured to standardize predictor data (see Standardize Data).

• Mu and Sigma are empty arrays [].

When you create the model object using the incrementalLearner function, EstimationPeriod is always 0.

Solver Tuning Period

During the solver tuning period, the incremental fitting functions use Mdl.TrainingOptions.TuningPeriod observations to tune the parameters of the mini-batch LBFGS solver (the default solver). There is no solver turning period for the FreeREX solver. You can select the solver algorithm and the length of the solver tuning period using the TrainingOptions name-value argument when you create the model object. For more information, see the Limited-Memory BFGS and FreeRex sections of the incrementalTrainingOptions reference page.

Metrics Warm-Up Period

During the metrics warm-up period, the incremental fitting functions fit the incremental model.

• An incrementalClassificationNeuralNetwork model object is warm and tracks the performance metrics in its Metrics property after the incremental fitting functions process MetricsWarmupPeriod observations and fit at least one observation from each expected class (see the MaxNumClasses and ClassNames arguments of incrementalClassificationNeuralNetwork).

• An incrementalRegressionNeuralNetwork model object is warm after the incremental fitting functions process MetricsWarmupPeriod observations.

References

[1] Glorot, Xavier, and Yoshua Bengio. “Understanding the difficulty of training deep feedforward neural networks.” In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249–256. 2010.

[2] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.” In Proceedings of the IEEE international conference on computer vision, pp. 1026–1034. 2015.

Version History

Introduced in R2026b