rlNumericSpec

Create continuous action or observation data specifications for reinforcement learning environments

Description

An rlNumericSpec object specifies continuous action or observation data specifications for reinforcement learning environments.

Creation

Syntax

spec = rlNumericSpec(dimension)

spec = rlNumericSpec(dimension,Name,Value)

Description

spec = rlNumericSpec(dimension) creates a data specification for continuous actions or observations and sets the Dimension property.

spec = rlNumericSpec(dimension,Name,Value) sets Properties using name-value pair arguments.

Properties

LowerLimit — Lower limit of the data space

-Inf (default) | scalar | matrix

Lower limit of the data space, specified as a scalar or matrix of the same size as the data space. When LowerLimit is specified as a scalar, rlNumericSpec applies it to all entries in the data space.

UpperLimit — Upper limit of the data space

Inf (default) | scalar | matrix

Upper limit of the data space, specified as a scalar or matrix of the same size as the data space. When UpperLimit is specified as a scalar, rlNumericSpec applies it to all entries in the data space.

Name — Name of the rlNumericSpec object

string

Name of the rlNumericSpec object, specified as a string.

Description — Description of the rlNumericSpec object

string

Description of the rlNumericSpec object, specified as a string.

Dimension — Dimension of the data space

numeric vector

This property is read-only.

Dimension of the data space, specified as a numeric vector.

DataType — Information about the type of data

"double" (default) | string

This property is read-only.

Information about the type of data, specified as a string, such as "double" or "single".

Object Functions

rlSimulinkEnv	Create reinforcement learning environment using dynamic model implemented in Simulink
rlFunctionEnv	Specify custom reinforcement learning environment dynamics using functions
rlValueFunction	Value function approximator object for reinforcement learning agents
rlQValueFunction	Q-Value function approximator object for reinforcement learning agents
rlVectorQValueFunction	Vector Q-value function approximator for reinforcement learning agents
rlContinuousDeterministicActor	Deterministic actor with a continuous action space for reinforcement learning agents
rlDiscreteCategoricalActor	Stochastic categorical actor with a discrete action space for reinforcement learning agents
rlContinuousGaussianActor	Stochastic Gaussian actor with a continuous action space for reinforcement learning agents

Examples

Create Reinforcement Learning Environment for Simulink Model

For this example, consider the rlSimplePendulumModel Simulink model. The model is a simple frictionless pendulum that initially hangs in a downward position.

Open the model.

mdl = 'rlSimplePendulumModel';
open_system(mdl)

Create rlNumericSpec and rlFiniteSetSpec objects for the observation and action information, respectively.

The observation is a vector containing three signals: the sine, cosine, and time derivative of the angle.

obsInfo = rlNumericSpec([3 1]) 

obsInfo = 
  rlNumericSpec with properties:

     LowerLimit: -Inf
     UpperLimit: Inf
           Name: [0×0 string]
    Description: [0×0 string]
      Dimension: [3 1]
       DataType: "double"

The action is a scalar expressing the torque and can be one of three possible values, -2 Nm, 0 Nm and 2 Nm.

actInfo = rlFiniteSetSpec([-2 0 2])

actInfo = 
  rlFiniteSetSpec with properties:

       Elements: [3×1 double]
           Name: [0×0 string]
    Description: [0×0 string]
      Dimension: [1 1]
       DataType: "double"

You can use dot notation to assign property values for the rlNumericSpec and rlFiniteSetSpec objects.

obsInfo.Name = 'observations';
actInfo.Name = 'torque';

Assign the agent block path information, and create the reinforcement learning environment for the Simulink model using the information extracted in the previous steps.

agentBlk = [mdl '/RL Agent'];
env = rlSimulinkEnv(mdl,agentBlk,obsInfo,actInfo)

env = 
SimulinkEnvWithAgent with properties:

           Model : rlSimplePendulumModel
      AgentBlock : rlSimplePendulumModel/RL Agent
        ResetFcn : []
  UseFastRestart : on

You can also include a reset function using dot notation. For this example, randomly initialize theta0 in the model workspace.

env.ResetFcn = @(in) setVariable(in,'theta0',randn,'Workspace',mdl)

env = 
SimulinkEnvWithAgent with properties:

           Model : rlSimplePendulumModel
      AgentBlock : rlSimplePendulumModel/RL Agent
        ResetFcn : @(in)setVariable(in,'theta0',randn,'Workspace',mdl)
  UseFastRestart : on

Version History

Introduced in R2019a