lit_sb3.vec_env.vec_normalize

source module lit_sb3.vec_env.vec_normalize

Classes

• VecNormalize — A moving average, normalizing wrapper for vectorized environment.

source class VecNormalize(venv: VecEnv, training: bool = True, norm_obs: bool = True, norm_reward: bool = True, clip_obs: float = 10.0, clip_reward: float = 10.0, gamma: float = 0.99, epsilon: float = 1e-08, norm_obs_keys: list[str] | None = None)

Bases : VecEnvWrapper

A moving average, normalizing wrapper for vectorized environment.

has support for saving/loading moving average,.

Parameters

• venv : VecEnv — the vectorized environment to wrap

• training : bool — Whether to update or not the moving average

• norm_obs : bool — Whether to normalize observation or not (default: True)

• norm_reward : bool — Whether to normalize rewards or not (default: True)

• clip_obs : float — Max absolute value for observation

• clip_reward : float — Max value absolute for discounted reward

• gamma : float — discount factor

• epsilon : float — To avoid division by zero

• norm_obs_keys : list[str] | None — Which keys from observation dict to normalize. If not specified, all keys will be normalized.

Methods

• set_venv — Sets the vector environment to wrap to venv.

• step_wait — Apply sequence of actions to sequence of environments.

• normalize_obs — Normalize observations using this VecNormalize's observations statistics.

• normalize_reward — Normalize rewards using this VecNormalize's rewards statistics.

• unnormalize_obs

• unnormalize_reward

• get_original_obs — Returns an unnormalized version of the observations from the most recent.

• get_original_reward — Returns an unnormalized version of the rewards from the most recent step.

• reset — Reset all environments.

• load — Loads a saved VecNormalize object.

• save — Save current VecNormalize object with.

source method VecNormalize.set_venv(venv: VecEnv) → None

Sets the vector environment to wrap to venv.

Also sets attributes derived from this such as num_env.

Parameters

• venv : VecEnv

• venv : VecEnv — The venv parameter.

Raises

• ValueError

source method VecNormalize.step_wait() → VecEnvStepReturn

Apply sequence of actions to sequence of environments.

actions -> (observations, rewards, dones).

where dones is a boolean vector indicating whether each element is new.

source method VecNormalize.normalize_obs(obs: np.ndarray | dict[str, np.ndarray]) → np.ndarray | dict[str, np.ndarray]

Normalize observations using this VecNormalize's observations statistics.

Calling this method does not update statistics.

source method VecNormalize.normalize_reward(reward: np.ndarray) → np.ndarray

Normalize rewards using this VecNormalize's rewards statistics.

Calling this method does not update statistics.

source method VecNormalize.unnormalize_obs(obs: np.ndarray | dict[str, np.ndarray]) → np.ndarray | dict[str, np.ndarray]

source method VecNormalize.unnormalize_reward(reward: np.ndarray) → np.ndarray

source method VecNormalize.get_original_obs() → np.ndarray | dict[str, np.ndarray]

Returns an unnormalized version of the observations from the most recent.

step or reset.

source method VecNormalize.get_original_reward() → np.ndarray

Returns an unnormalized version of the rewards from the most recent step.

source method VecNormalize.reset() → np.ndarray | dict[str, np.ndarray]

Reset all environments.

Returns

• np.ndarray | dict[str, np.ndarray] — first observation of the episode

source staticmethod VecNormalize.load(load_path: str | Path, venv: VecEnv) → VecNormalize

Loads a saved VecNormalize object.

Parameters

• load_path : str | Path — the path to load from.

• venv : VecEnv — the VecEnv to wrap.

source method VecNormalize.save(save_path: str | Path) → None

Save current VecNormalize object with.

all running statistics and settings (e.g. clip_obs).

Parameters

• save_path : str | Path — The path to save to