Missing savedAgentResultStruct | How do I get the elapsed time from saved agent?

Hi,

Currently I am working on Reinforcement Learning.

I would like to display result of training curve of various training condition.

I would like to use the comparePerformance.m from the provided walking biped which save the agent in savedAgentResultstruct.

However my saved agent in saved_agent and savedAgentresult.

My question is:

1. How do I convert or save properly for having saveAgentResultStruct?
2. Alternatively, the needed info for comparePerformance.m is elapsed time. How do I retrieved this info from my saved agent?

Thank you in advance.

Answer 1

[Ari Biswas]

We have recently improved the design of saving agents with relevant training information. In the new design (available from R2022a), the saved agent MAT file will contain the agent (under the variable "saved_agent") and a training result object (under the variable "savedAgentResult"). The training result object replaces the previous "savedAgentResultStruct". We will fix the example to reflect the change.

The example should still work fine when comparing the existing "DDPGAgent" and "TD3Agent" folders. For new trainings, you can replace lines 76-82 in comparePerformance.m with the following code:

for i = 1:size(files,1)
    S = load(files(i).name);
    if isfield(S,'savedAgentResultStruct')
        res = S.savedAgentResultStruct.TrainingStats;
        info = S.savedAgentResultStruct.Information;
    else
        res = S.savedAgentResult;
        info = res.Information;
    end
    agent.averageReward(:,i) = res.AverageReward;
    agent.EpisodeQ0(:,i) = res.EpisodeQ0;
    agent.EpisodeIndex = res.EpisodeIndex;
    agent.ElapsedTime = info.ElapsedTime;
end

You can retrieve the elapsed time from the training result object using

t = savedAgentResult.Information.ElapsedTime

Do remember that for each new training you will need to extract the MAT-file of interest and place in a separate folder other than the auto-generated "savedAgents" folder before doing the comparison.