Salvataggio Agente trainato per code generation

Salve, ho allenato un agente DDPG tramite il "ReinforcementLearningDesigner", successivamente ho ottenuto un blocco .mat con tutti i dati della sessione, vorrei sapere dove è salvato l'agente trainato, chiedo questo perche vorrei fare code generation, quindi vorrei generare la policy da questo agente trainato per poi inserirla dentro il "policy block".

Inoltre mi chiedo, com'è possibile che il blocco policy funzioni se non ha in input anche i dati sulla "reward"?

[Walter Roberson]

Approximate translation:

Hi, I trained a DDPG agent via the "ReinforcementLearningDesigner", subsequently I obtained a .mat block with all the session data, I would like to know where the trained agent is saved, I ask this because I would like to do code generation, so I would like to generate the policy driven by this agent and then inserting it into the "policy block".I also wonder, how is it possible for the policy block to work if it doesn't also have the "reward" data as input?

Answer 1

[Aiswarya]

Hi,

Si prega di notare che risponderò alla domanda in inglese.

( Please note that I will be answering the question in English. )

In order to save your trained agent in the Reinforcement Learning Designer, you can export your agent to MATLAB workspace. To do this, navigate to Reinforcement Learning tab, and under Export select the trained agent. You may refer to this following documentation on how to export agent and save the session: https://www.mathworks.com/help/reinforcement-learning/ug/design-dqn-using-rl-designer.html#mw_abc1bb48-f0fc-400d-98a5-e222c80d131d

You can save your agent in a MAT file using the below command :

save("Agent.mat","agent")

Then you can directly create a Simulink "Policy" block using command line as follows:

load("Agent.mat","agent")
generatePolicyBlock(agent);

You may refer to the following link for more information on the "generatePolicyBlock" function: https://www.mathworks.com/help/reinforcement-learning/ref/generatepolicyblock.html

During training the agent uses the reward signal and updates its policy based on it. Once the policy is trained, you can deploy it to make decisions and hence the policy block only needs the observation as input to output an action. The "Policy" block is using the policy as a parameter, which has already been learned through the rewards received during training. Hence, it does not require the reward as input.