Skip to content

lit_sb3.wrappers.atari.episodic_life

source module lit_sb3.wrappers.atari.episodic_life

Classes

  • EpisodicLifeEnv — Make end-of-life == end-of-episode, but only reset on true game over.

source class EpisodicLifeEnv(env: gym.Env)

Bases : gym.Wrapper[np.ndarray, int, np.ndarray, int]

Make end-of-life == end-of-episode, but only reset on true game over.

Done by DeepMind for the DQN and co. since it helps value estimation.

.. note:: This wrapper changes the behavior of env.reset(). When the environment terminates due to a loss of life (but not game over), calling reset() will perform a no-op step instead of truly resetting the environment. This can be confusing when evaluating or testing agents. To avoid this behavior and ensure reset() always resets to the env, set terminal_on_life_loss=False when using make_atari_env().

Parameters

  • env : gym.Env — Environment to wrap

Attributes

  • metadata : dict[str, Any] — attr:Env :attr:metadata.

  • render_mode : str | None — attr:Env :attr:render_mode.

  • spec : EnvSpec | None — attr:Env :attr:spec attribute with the WrapperSpec if the wrapper inherits from EzPickle.

  • action_space : spaces.Space[ActType] | spaces.Space[WrapperActType] — attr:Env :attr:action_space unless overwritten then the wrapper :attr:action_space is used.

  • observation_space : spaces.Space[ObsType] | spaces.Space[WrapperObsType] — attr:Env :attr:observation_space unless overwritten then the wrapper :attr:observation_space is used.

  • unwrapped : Env[ObsType, ActType] — Returns the base environment of the wrapper.

  • np_random_seed : int | None — attr:np_random_seed.

  • np_random : np.random.Generator — attr:Env :attr:np_random attribute.

Methods

  • step

  • reset — Calls the Gym environment reset, only when lives are exhausted.

source method EpisodicLifeEnv.step(action: int) → AtariStepReturn

source method EpisodicLifeEnv.reset(**kwargs) → AtariResetReturn

Calls the Gym environment reset, only when lives are exhausted.

This way all states are still reachable even though lives are episodic, and the learner need not know about any of this behind-the-scenes.

Parameters

  • kwargs — Extra keywords passed to env.reset() call

Returns

  • AtariResetReturn — the first observation of the environment