Normalization of observation in RL in DDPG

I am training a smart train optimization model using reinforcement learning, where distance and velocity are part of the observation space. The distance ranges from 0 to 10,000 meters, while the velocity has significantly smaller values. I'm considering whether to convert the distance to kilometers for scaling or use a different normalization technique, as I’ve done in PyTorch. However, I haven't come across built-in normalization options in MATLAB for this task.

Answer 1

[Drishti]

Hi Laxmi,

The Reinforcement Learning toolbox provides the ‘rlNormalizer’ object which configure normalization for input of function approximator object. It ensures that the inputs are on a similar scale when provided to the network.

The ‘rlNormalizer’ has been introduced in MATLAB R2024a and later versions.

You can refer to the MATLAB Documentation of ‘rlNormalizer’ for better understanding:

https://www.mathworks.com/help/releases/R2024a/reinforcement-learning/ref/rlnormalizer.html

You can also leverage the ‘unitsratio’ function for the conversion of input from one unit to another.

Refer to the example below:

mPerFoot = unitsratio("meter","feet");

MATLAB provides explicit functions ‘normalize’ and ‘zscore’ to scale the data. The ‘normalize’ function normalizes data within specified range whereas the ‘zscore’ depicts the standardized z-score.

Refer to the implementation below:

distance_normalized = normalize(distance, "range", [0, 1]);
distance_standardized = zscore(distance);

You can also leverage the MATLAB Documentation of ‘normalize’ and ‘zscore’ functions:

• ‘normalize’: https://www.mathworks.com/help/releases/R2024a/matlab/ref/double.normalize.html
• ‘zscore’: https://www.mathworks.com/help/releases/R2024a/stats/zscore.html

I hope this helps in getting started.