lit_sb3.wrappers

Classes

ActionMasker — Env wrapper providing the method required to support masking.
ActionNoiseWrapper — Add Gaussian noise to the action (without telling the agent).
ActionSmoothingWrapper — Smooth the action using exponential moving average.
DelayedRewardWrapper — Delay the reward by delay steps (accumulated until released).
FrameSkip — Return only every skip-th frame (frameskipping).
HistoryWrapper — Stack past observations and actions to give a history to the agent.
HistoryWrapperObsDict — History Wrapper for dict observation spaces.
MaskVelocityWrapper — Mask velocity terms in observations to make the MDP partially observable.
TimeFeatureWrapper — Add remaining, normalized time to observation space for fixed length episodes.
TruncatedOnSuccessWrapper — Reset on success and offset the reward.
YAMLCompatResizeObservation — ResizeObservation that accepts a list instead of a tuple for YAML compatibility.

source class ActionMasker(env: gym.Env, action_mask_fn: str | Callable[[gym.Env], np.ndarray])

Bases : gym.Wrapper

Env wrapper providing the method required to support masking.

Exposes a method called action_masks(), which returns masks for the wrapped env. This wrapper is not needed if the env exposes the expected method itself.

Parameters

env : gym.Env — the Gym environment to wrap
action_mask_fn : str | Callable[[gym.Env], np.ndarray] — A function that takes a Gym environment and returns an action mask, or the name of such a method provided by the environment.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

action_masks

source method ActionMasker.action_masks() → np.ndarray

source class ActionNoiseWrapper(env: gym.Env, noise_std: float = 0.1)

Bases : gym.Wrapper[ObsType, np.ndarray, ObsType, np.ndarray]

Add Gaussian noise to the action (without telling the agent).

Useful to test the robustness of the control.

:param env: Gym environment. :param noise_std: Standard deviation of the Gaussian noise.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

step

source method ActionNoiseWrapper.step(action: np.ndarray) → tuple[ObsType, SupportsFloat, bool, bool, dict[str, Any]]

source class ActionSmoothingWrapper(env: gym.Env, smoothing_coef: float = 0.0)

Bases : gym.Wrapper

Smooth the action using exponential moving average.

:param env: Gym environment. :param smoothing_coef: Smoothing coefficient (0 = no smoothing, 1 = very smooth).

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

reset
step

source method ActionSmoothingWrapper.reset(seed: int | None = None, options: dict | None = None) → GymResetReturn

source method ActionSmoothingWrapper.step(action: np.ndarray) → GymStepReturn

source class DelayedRewardWrapper(env: gym.Env, delay: int = 10)

Bases : gym.Wrapper

Delay the reward by delay steps (accumulated until released).

Makes the task harder but more realistic.

:param env: Gym environment. :param delay: Number of steps to delay the reward.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

reset
step

source method DelayedRewardWrapper.reset(seed: int | None = None, options: dict | None = None) → GymResetReturn

source method DelayedRewardWrapper.step(action: Any) → GymStepReturn

source class FrameSkip(env: gym.Env, skip: int = 4)

Bases : gym.Wrapper

Return only every skip-th frame (frameskipping).

:param env: Gym environment. :param skip: Number of frames to skip (repeat action for).

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

step — Step the environment, repeating action and summing reward.

source method FrameSkip.step(action: Any) → GymStepReturn

Step the environment, repeating action and summing reward.

:param action: The action to repeat. :return: observation, cumulative reward, terminated, truncated, info.

source class HistoryWrapper(env: gym.Env, horizon: int = 2)

Bases : gym.Wrapper[np.ndarray, np.ndarray, np.ndarray, np.ndarray]

Stack past observations and actions to give a history to the agent.

:param env: Gym environment. :param horizon: Number of past steps to include in the observation.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

reset
step

source method HistoryWrapper.reset(seed: int | None = None, options: dict | None = None) → tuple[np.ndarray, dict]

source method HistoryWrapper.step(action: np.ndarray) → tuple[np.ndarray, SupportsFloat, bool, bool, dict]

source class HistoryWrapperObsDict(env: gym.Env, horizon: int = 2)

Bases : gym.Wrapper

History Wrapper for dict observation spaces.

:param env: Gym environment with a dict observation space. :param horizon: Number of past steps to include.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

reset
step

source method HistoryWrapperObsDict.reset(seed: int | None = None, options: dict | None = None) → tuple[dict[str, np.ndarray], dict]

source method HistoryWrapperObsDict.step(action: np.ndarray) → tuple[dict[str, np.ndarray], SupportsFloat, bool, bool, dict]

source class MaskVelocityWrapper(env: gym.Env)

Bases : gym.ObservationWrapper

Mask velocity terms in observations to make the MDP partially observable.

Adapted from https://github.com/LiuWenlin595/FinalProject.

:param env: Gym environment.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

observation

source method MaskVelocityWrapper.observation(observation: np.ndarray) → np.ndarray

source class TimeFeatureWrapper(env: gym.Env, max_steps: int = 1000, test_mode: bool = False)

Bases : gym.Wrapper[TimeFeatureObs, ActType, TimeFeatureObs, ActType]

Add remaining, normalized time to observation space for fixed length episodes.

See https://arxiv.org/abs/1712.00378 and https://github.com/aravindr93/mjrl/issues/13.

.. note::

Only ``gym.spaces.Box`` and ``gym.spaces.Dict`` (``gym.GoalEnv``) 1D observation spaces
are supported for now.

Parameters

env : gym.Env — Gym env to wrap.
max_steps : int — Max number of steps of an episode if it is not wrapped in a TimeLimit object.
test_mode : bool — In test mode, the time feature is constant, equal to zero. This allow to check that the agent did not overfit this feature, learning a deterministic pre-defined sequence of actions.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

reset
step

source method TimeFeatureWrapper.reset(**kwargs) → tuple[TimeFeatureObs, dict[str, Any]]

source method TimeFeatureWrapper.step(action: ActType) → tuple[TimeFeatureObs, SupportsFloat, bool, bool, dict[str, Any]]

source class TruncatedOnSuccessWrapper(env: gym.Env, reward_offset: float = 0.0, n_successes: int = 1)

Bases : gym.Wrapper

Reset on success and offset the reward.

Useful for GoalEnv environments where success should end the episode.

:param env: Gym environment. :param reward_offset: Reward added on top of the env reward at each step. :param n_successes: Number of consecutive successes required to truncate.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

source method TruncatedOnSuccessWrapper.reset(seed: int | None = None, options: dict | None = None) → GymResetReturn

source method TruncatedOnSuccessWrapper.step(action: Any) → GymStepReturn

source method TruncatedOnSuccessWrapper.compute_reward(achieved_goal: Any, desired_goal: Any, info: dict[str, Any]) → SupportsFloat

source class YAMLCompatResizeObservation(env: gym.Env, shape: list[int])

Bases : ResizeObservation

ResizeObservation that accepts a list instead of a tuple for YAML compatibility.

Attributes

metadata : dict[str, Any] — attr:Env :attr:metadata.
render_mode : str | None — attr:Env :attr:render_mode.
spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.
action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.
unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.
np_random_seed : int | None — attr:np_random_seed.
np_random : np.random.Generator — attr:Env :attr:np_random attribute.