Unable to run 'rlwatertank' example in R2020a
Ältere Kommentare anzeigen
Hello everyone
While I successfully ran this example in R2019b, I could not successfully train the agent for this example in R2020a.
I also tried other available examples in the documentation; however, the learning plots in Reinforcement Learning Episode Manager differed from the plots exhibited in documentation.
I should mention I followed the exact steps in documentation and did not change value of any parameter.
Is this some sort of bug in "Reinforcement Learning Toolbox" at R2020a release?
2 Kommentare
Radovan Vuletic
am 8 Feb. 2021
I have a same problem with R2020b
>> open_system('rlwatertank')
No system or file called 'rlwatertank' found. 'rlwatertank' is used in
Create Simulink Environment and Train Agent
Create Simulink Environment Using Agent in Workspace
Validate Simulink Environment
Water Tank Reinforcement Learning Environment Model.
Any official statement on this?
Basically I am interested on PMSM control with RL, but that example dosn't work neither.
Mariam Kashkash
am 21 Mär. 2021
I have also the same problem the rlwatertank does not open. Do you find any solution for this problem?
Akzeptierte Antwort
Weitere Antworten (1)
Stefalo Acha
am 18 Feb. 2023
0 Stimmen
Hello everyone
I was trying to run the below code in R2022b but keep an error about "No system or file called 'rlAreaCoverage' on the command line #15 --> mdl = "rlAreaCoverage";
Please, I will apreciate any help. Thanks
%define the drones location of obstacles within the grid using matrix of indeces
clc;
clear all;
close all;
obsMat = [4 3; 5 3; 6 3; 7 3; 8 3; 9 3; 5 11; 6 11; 7 11; 8 11; 5 12; 6 12; 7 12; 8 12];
%initialize the drones position
sA0 = [2 2];
sB0 = [11 4];
sC0 = [3 12];
s0 = [sA0; sB0; sC0];
%specify sample time
Ts = 0.1;
Tf = 100;
maxsteps = ceil(Tf/Ts);
mdl = "rlAreaCoverage";
open_system(mdl)
% Define observation specification
obssize = [12 12 4]
oinfo = rlNumericSpec(actionSpace)
ainfo.Nmae = 'action';
blks = mdl + ["/Agent A (Red)","/Agent B (Green)","/Agent C (Blue)"];
env = rlSimulinkEnv(mdl,blks,{oinfo,oinfo,oinfo},{ainfo,ainfo,ainfo});
env.ResetFcn = @(in) resetMap(in, obsMat);
rng(0)
for idx = 1:3
% Create actor deep neural network.
actorNetWork = [
imageInputLayer(obsSize,'Normalization','none','Name','observations')
convolution2dLayer(8,16,'Name','conv1','Stride',1,'Padding',1,'WeightsInitializer','he')
reluLayer('Name','relu1')
convolution2dLayer(4,8,'Name','conv2','Stride',1,'Padding','same','WeightsInitializer','he')
reluLayer('Name','relu2')
fullyConnectedLayer(256,'Name','fc1','WeightsInitializer','he')
reluLayer('Name','relu3')
fullyConnectedLayer(128,'Name','fc2','WeightsInitializer','he')
reluLayer('Name','relu4')
fullyConnectedLayer(64,'Name','fc3','WeightsInitializer','he')
reluLayer('Name','relu5')
fullyConnectedLayer(numAct,'Name','output')
softmaxLayer('Name','action')];
actorNetWork = dlnetwork(actorNetWork);
% Create critic deep neural network.
criticNetwork = [
imageInputLayer(obsSize,'Normalization','none','Name','observations')
convolution2dLayer(8,16,'Name','conv1','Stride',1,'Padding',1,'WeightsInitializer','he')
reluLayer('Name','relu1')
convolution2dLayer(4,8,'Name','conv2','Stride',1,'Padding','same','WeightsInitializer','he')
reluLayer('Name','relu2')
fullyConnectedLayer(256,'Name','fc1','WeightsInitializer','he')
reluLayer('Name','relu3')
fullyConnectedLayer(128,'Name','fc2','WeightsInitializer','he')
reluLayer('Name','relu4')
fullyConnectedLayer(64,'Name','fc3','WeightsInitializer','he')
reluLayer('Name','relu5')
fullyConnectedLayer(1,'Name','output')];
criticNetwork = dlnetwork(criticNetwork);
% create actor and critic
actor(idx) = rlDiscreteCategoricalActor(actorNetWork,oinfo,ainfo); %#ok<*SAGROW>
critic(idx) = rlValueFunction(criticNetwork,oinfo);
end
actorOpts = rlOptimizerOptions('LearnRate',1e-4,'GradientThreshold',1);
criticOpts = rlOptimizerOptions('LearnRate',1e-4,'GradientThreshold',1);
opt = rlPPOAgentOptions(...
'ActorOptimizerOptions',actorOpts,...
'CriticOptimizerOptions',criticOpts,...
'ExperienceHorizon',128,...
'ClipFactor',0.2,...
'EntropyLossWeight',0.01,...
'MiniBatchSize',64,...
'NumEpoch',3,...
'AdvantageEstimateMethod','gae',...
'GAEFactor',0.95,...
'SampleTime',Ts,...
'DiscountFactor',0.995);
agentA = rlPPOAgent(actor(1),critic(1),opt);
agentB = rlPPOAgent(actor(2),critic(2),opt);
agentC = rlPPOAgent(actor(3),critic(3),opt);
trainOpts = rlMultiAgentTrainingOptions(...
"AgentGroups","auto",...
"LearningStrategy","decentralized",...
'MaxEpisodes',1000,...
'MaxStepsPerEpisode',maxsteps,...
'Plots','training-progress',...
'ScoreAveragingWindowLength',100,...
'StopTrainingCriteria','AverageReward',...
'StopTrainingValue',80);
doTraining = false;
if doTraining
result = train([agentA,agentB,agentC],env,trainOpts);
else
load('rlAreaCoverageAgents.mat');
end
Kategorien
Mehr zu Reinforcement Learning finden Sie in Hilfe-Center und File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
