Reinforcement Learning Error with two scalar inputs

1 Ansicht (letzte 30 Tage)
ali farid
ali farid am 31 Jan. 2024
I have a strange error from a critic network that has 3 inputs, image, and two scalars. But I see the following error:
Error using rl.internal.validate.mapFunctionObservationInput
Unable to automatically specify deep neural network observation input layer names because some specifications have similar dimension. Specify "ObservationInputNames" name-value pair when
creating function object.
Error in rlContinuousGaussianActor (line 95)
modelInputMap = rl.internal.validate.mapFunctionObservationInput(model,observationInfo,nameValueArgs.ObservationInputNames);
Do you have any idea? It is really long time I am working on this error. My network is as follows:
type=3;
w1=2;
w2=1;
% obsMat = [1 1];
obsMat = [4 3; 5 3; 6 3; 7 3; 8 3; 9 3; 5 11; 6 11; 7 11; 8 11; 6 12; 7 12; 10 12; ];
sA0 = [2 5];
sB0 = [11 5];
sC0 = [3 2];
sD0 = [6 5];
sE0 = [12 5];
sF0 = [6 12];
sG0 = [11 5];
sH0 = [3 11];
sI0 = [6 3];
sJ0 = [1 11];
s0 = [sA0; sB0; sC0];
% s0 = [sA0; sB0; sC0; sD0];
% s0 = [sA0; sB0; sC0];
Ts = 0.1;
Tf = 100;
maxsteps = ceil(Tf/Ts);
mdl = "rlAreaCoverage32024";
open_system(mdl)
% Define observation specifications.
scalarObs1Info = rlNumericSpec([1 1]);
scalarObs1Info.Name ="scalarObservation1";
scalarObs2Info = rlNumericSpec([1 1]);
scalarObs2Info.Name ="scalarObservation2";
obsSize = [12 12 4];
oinfo = rlNumericSpec(obsSize);
oinfo.Name ="image";
% oinfo.Name = "observations";
allObsInfo = [ oinfo, scalarObs1Info, scalarObs2Info];%, scalarObs3Info];
actionSpace = {1,2,3,4};
ActionInfo = rlNumericSpec([1, 2], 'Lowerlimit', -1, 'Upperlimit', 1); ainfo = ActionInfo;
ainfo.Name = "actions";
actInfo.UpperLimit=1;
actInfo.Lowerlimit=-1;
blks = mdl + ["/Agent A (Red)","/Agent B (Green)","/Agent C (Blue)"];
env = rlSimulinkEnv(mdl,blks,{allObsInfo,allObsInfo,allObsInfo},{ainfo,ainfo,ainfo});
env.ResetFcn = @(in) resetMap(in, obsMat);
rng(0)
for idx = 1:type
lgraph = layerGraph();
tempLayers = [
featureInputLayer(w1,"Name","scalarInput1")
reluLayer("Name","relu_3")
fullyConnectedLayer(1,"Name","fc_4")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
featureInputLayer(w3,"Name","scalarInput3")
reluLayer("Name","relu_1")
fullyConnectedLayer(1,"Name","fc_2")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
imageInputLayer(obsSize,Normalization="none")
convolution2dLayer(8,16, ...
Stride=1,Padding=1,WeightsInitializer="he")
reluLayer
convolution2dLayer(4,8, ...
Stride=1,Padding="same",WeightsInitializer="he")
reluLayer
fullyConnectedLayer(256,WeightsInitializer="he")
reluLayer
fullyConnectedLayer(128,WeightsInitializer="he")
% Hidden units (default = 128) are the number of units in the hidden layer of the neural network. Its size depends on the complexity of the problem, and should be set larger when there is a complex relationship between agent actions and observed variables.
reluLayer
fullyConnectedLayer(1,"Name","fc_1")];
lgraph = addLayers(lgraph,tempLayers);
tempLayers = [
concatenationLayer(1,3,"Name","concat")
softmaxLayer("Name","softmax")];
lgraph = addLayers(lgraph,tempLayers);
% clean up helper variable
clear tempLayers;
lgraph = connectLayers(lgraph,"fc_2","concat/in3");
% lgraph = connectLayers(lgraph,"fc_3","concat/in2");
lgraph = connectLayers(lgraph,"fc_1","concat/in4");
lgraph = connectLayers(lgraph,"fc_4","concat/in1");
plot(lgraph);
actorNetwork=lgraph;
% dlnetwork(actorNetwork);
actorOptions = rlOptimizerOptions('LearnRate',0.1,'GradientThreshold',inf);
actor(idx) = rlContinuousGaussianActor(actorNetwork,allObsInfo,ainfo);
  1 Kommentar
Emmanouil Tzorakoleftherakis
The code you sent cannot be executed as a standalone. Also, I am getting different errors that what you sent. Please modify the code to reproduce the error you mentioned above.

Melden Sie sich an, um zu kommentieren.

Antworten (0)

Kategorien

Mehr zu Deep Learning Toolbox finden Sie in Help Center und File Exchange

Produkte


Version

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by