rlOptimizer

Creates an optimizer object for actors and critics

Since R2022a

Syntax

optObj = rlOptimizer

optObj = rlOptimizer(optOptions)

Description

Create an optimizer object that updates the learnable parameters of an actor or critic in a custom training loop

optObj = rlOptimizer returns a default optimizer object. You can modify the object properties using dot notation.

optObj = rlOptimizer(optOptions) returns an optimizer object with properties specified by the optimizer options object algOptions.

Examples

Create Optimizer Object

Use rlOptimizer to create a default optimizer algorithm object to use for the training of an actor or critic in a custom training loop.

myAlg = rlOptimizer

myAlg = 
  rlADAMOptimizer with properties:

           GradientDecayFactor: 0.9000
    SquaredGradientDecayFactor: 0.9990
                       Epsilon: 1.0000e-08
                     LearnRate: 0.0100
        L2RegularizationFactor: 1.0000e-04
             GradientThreshold: Inf
       GradientThresholdMethod: "l2norm"

By default, the function returns an rlADAMOptimizer object with default options. You can use dot notation to change some parameters.

myAlg.LearnRate = 0.1;

You can now create a structure and set its CriticOptimizer or ActorOptimizer field to myAlg. When you call runEpisode, pass the structure as an input parameter. The runEpisode function can then use the update method of myAlg to update the learnable parameters of your actor or critic.

Create Optimizer Object Specifying an Option Set

Use rlOptimizerOptions to create an optimizer option object. Specify the algorithm as "rmsprop" and set the learning rate to 0.2.

myOptions=rlOptimizerOptions( ...
    Algorithm="rmsprop", ...
    LearnRate=0.2);

Use rlOptimizer to create an optimizer algorithm object to use for the training of an actor or critic in a custom training loop. Specify the optimizer option set myOptions as input parameter.

myAlg=rlOptimizer(myOptions)

myAlg = 
  rlRMSPropOptimizer with properties:

    SquaredGradientDecayFactor: 0.9990
                       Epsilon: 1.0000e-08
                     LearnRate: 0.2000
        L2RegularizationFactor: 1.0000e-04
             GradientThreshold: Inf
       GradientThresholdMethod: "l2norm"

The function returns an rlRMSPropOptimizer object with default options. You can use dot notation to change some parameters.

myAlg.GradientThreshold = 2;

You can now create a structure and set its CriticOptimizer or ActorOptimizer field to myAlg. When you call runEpisode, pass the structure as an input parameter. The runEpisode function can then use the update method of myAlg to update the learnable parameters of your actor or critic.

Input Arguments

optOptions — Optimizer options object

default Adam option set (default) | rlOptimizerOptions object

Optimizer options object for an optimization algorithm, specified as an rlOptimizerOptions object.

Example: rlOptimizerOptions(Algorithm="sgdm",LearnRate=0.2)

Output Arguments

optObj — Optimizer object

rlADAMOptimizer object | rlSGDMOptimizer object | rlRMSPropOptimizer object

Optimizer object, returned as an rlADAMOptimizer, rlSGDMOptimizer, or rlRMSPropOptimizer object. The object implements an optimization algorithm. The runEpisode function uses the update method of the returned object to update the learnable parameter of an actor or critic.

Version History

Introduced in R2022a