setLearnableParameters

Set learnable parameter values of actor or critic function object

Syntax

newFcn = setLearnableParameters(oldFcn,val)

Description

newFcn = setLearnableParameters(oldFcn,val) returns a new actor or critic function object, newFcn, with the same structure as the original function object, oldFcn, and the learnable parameter values specified in val.

Examples

Modify Critic Parameter Values

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Train DDPG Agent to Control Double Integrator System.

load('DoubleIntegDDPG.mat','agent') 

Obtain the critic function approximator from the agent.

critic = getCritic(agent);

Obtain the learnable parameters from the critic.

params = getLearnableParameters(critic)

params=2×1 cell array
    {[-5.0077 -1.5619 -0.3475 -0.0961 -0.0455 -0.0026]}
    {[                                              0]}

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

modifiedParams = cellfun(@(x) x*2,params,'UniformOutput',false);

Set the parameter values of the critic to the new modified values.

critic = setLearnableParameters(critic,modifiedParams);

Set the critic in the agent to the new modified critic.

setCritic(agent,critic);

Display the new parameter values.

getLearnableParameters(getCritic(agent))

ans=2×1 cell array
    {[-10.0154 -3.1238 -0.6950 -0.1922 -0.0911 -0.0052]}
    {[                                               0]}

Modify Actor Parameter Values

Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent from Train DDPG Agent to Control Double Integrator System.

load('DoubleIntegDDPG.mat','agent') 

Obtain the actor function approximator from the agent.

actor = getActor(agent);

Obtain the learnable parameters from the actor.

params = getLearnableParameters(actor)

params=2×1 cell array
    {[-15.4689 -7.1635]}
    {[               0]}

Modify the parameter values. For this example, simply multiply all of the parameters by 2.

modifiedParams = cellfun(@(x) x*2,params,'UniformOutput',false);

Set the parameter values of the actor to the new modified values.

actor = setLearnableParameters(actor,modifiedParams);

Set the actor in the agent to the new modified actor.

setActor(agent,actor);

Display the new parameter values.

getLearnableParameters(getActor(agent))

ans=2×1 cell array
    {[-30.9378 -14.3269]}
    {[                0]}

Input Arguments

oldFcn — Original actor or critic function object

rlValueFunction object | rlQValueFunction object | rlVectorQValueFunction object | rlContinuousDeterministicActor object | rlDiscreteCategoricalActor object | rlContinuousGaussianActor object

Original actor or critic function object, specified as one of the following:

• rlValueFunction object — Value function critic

• rlQValueFunction object — Q-value function critic

• rlVectorQValueFunction object — Multi-output Q-value function critic with a discrete action space

• rlContinuousDeterministicActor object — Deterministic policy actor with a continuous action space

• rlDiscreteCategoricalActor — Stochastic policy actor with a discrete action space

• rlContinuousGaussianActor object — Stochastic policy actor with a continuous action space

To create an actor or critic function object, use one of the following methods.

• Create a function object directly.

• Obtain the existing critic from an agent using getCritic.

• Obtain the existing actor from an agent using getActor.

val — Learnable parameter values

cell array

Learnable parameter values for the representation object, specified as a cell array. The parameters in val must be compatible with the structure and parameterization of oldRep.

To obtain a cell array of learnable parameter values from an existing function approximator, which you can then modify, use the getLearnableParameters function.

Output Arguments

newFcn — New actor or critic function object

rlValueFunction object | rlQValueFunction object | rlVectorQValueFunction object | rlContinuousDeterministicActor object | rlDiscreteCategoricalActor object | rlContinuousGaussianActor object

New actor or critic function object, returned as a function object of the same type as oldFcn. Apart from the learnable parameter values, newFcn is the same as oldFcn.

Version History

Introduced in R2019a

R2022a: setLearnableParameters now uses approximator objects instead of representation objects

Behavior changed in R2022a

Using representation objects to create actors and critics for reinforcement learning agents is no longer recommended. Therefore, setLearnableParameters now uses function approximator objects instead.

R2020a: setLearnableParameterValues is now setLearnableParameters

Behavior changed in R2020a

setLearnableParameterValues is now setLearnableParameters. To update your code, change the function name from setLearnableParameterValues to setLearnableParameters. The syntaxes are equivalent.