I get error when I want to define an environment for DQN, that is related to RL observation.

Hi everyone,

I want to generate a DQN actor, but when I want to create an environment, it gives an error, I will be appreciated if anyone can help me with this problem. I have 11 inputs and 3 actions for outputs.

mdl = "challenge";

%% Observation and actions

obsInfo = rlNumericSpec([11 1],'LowerLimit',-inf*ones(11,1),'UpperLimit',inf*ones(11,1));

actInfo = rlFiniteSetSpec({[0 ; 1], [0; 1],[0 ;1]});

obsInfo.Name = "Observation";

actInfo.Name = "Actions";

%% Environment

agentBlk = [mdl '/rlAgent'];

env = rlSimulinkEnv(mdl,agentBlk,obsInfo,actInfo);

it gives me the following error:

Error using rlSimulinkEnv>localValidateIOSpecs (line 172)

Invalid size or type for observation specification.

Error in rlSimulinkEnv (line 90)

localValidateIOSpecs(numAgents,observationInfo,actionInfo);

Answer 1

[Ayush Aniket]

Hi Armin,

As per my understanding you are getting the error because of the incorrect definition of "Agent Block Paths". For a single RL Agent block, the input to "agentBlocks" argument needs to be a string or character vector containing the block path. However, by your definition it is a string array specifying multiple agents and hence the discrepancy in the observation information. You can correct it as:

%% Environment
agentBlk = mdl + '/rlAgent'; 

You can read more about the input arguments of "rlSimulinkEnv" function by referring to the following link:

https://www.mathworks.com/help/reinforcement-learning/ref/rlsimulinkenv.html#mw_7e7f2e6a-759a-40f8-8323-3e255434e3ed

Hope it helps.