Hyperparameter optimization and saving the best agents for Reinforcement Learning

I am trying to train my RL agent (ddpg) but it's performing quite poorly. I think it may be a problem with the hyperparameter values since I have not tuning. Now I have two questions--

1. If there is anything in MATLAB that may help solve this problem of hyperparameter optimization other than manual trial-and-error?
2. How do I save the best performing agent given I don't know the critical values (i.e. don't know the range of the reward)? Basically, I want to save the agent that provides maximum reward or, say, top-5 highest rewarding agents?

Thanks.

Answer 1

[Emmanouil Tzorakoleftherakis]

Hello,

1. You can use something like this. We do not have any examples with Reinforcement Learning Toolbox that show how to use this yet unfortunately.
2. If it's challenging to estimate what a good episode reward is, you can run a singe training session for a good number of episodes (e.g. 5k episodes) to get some idea how the agent is doing and then use that knowledge from the training plot to set the 'SaveAgent' option as needed. Most of the time you will need to run multiple training sessions either way to tweak parameters, rewards, etc, so just use the first one to get some intuition.

[laha_M]

Thanks, Emmanouil.

[Francisco Serra]

Hey @laha_M, did you manage to to this with RL Toolbox?