Skip to content

lit_sb3.buffers.maskable.dict_rollout

source module lit_sb3.buffers.maskable.dict_rollout

Classes

source class MaskableDictRolloutBufferSamples()

Bases : NamedTuple

source class MaskableDictRolloutBuffer(buffer_size: int, observation_space: spaces.Dict, action_space: spaces.Space, device: torch.device | str = 'auto', gae_lambda: float = 1, gamma: float = 0.99, n_envs: int = 1)

Bases : DictRolloutBuffer

Dict Rollout buffer used in on-policy algorithms like A2C/PPO.

Extends the RolloutBuffer to use dictionary observations.

It corresponds to buffer_size transitions collected using the current policy. This experience will be discarded after the policy update. In order to use PPO objective, we also store the current value of each state and the log probability of each taken action.

The term rollout here refers to the model-free notion and should not be used with the concept of rollout used in model-based RL or planning. Hence, it is only involved in policy and value function training but not action selection.

Parameters

  • buffer_size : int — Max number of element in the buffer

  • observation_space : spaces.Dict — Observation space

  • action_space : spaces.Space — Action space

  • device : torch.device | str — PyTorch device

  • gae_lambda : float — Factor for trade-off of bias vs variance for Generalized Advantage Estimator Equivalent to classic advantage when set to 1.

  • gamma : float — Discount factor

  • n_envs : int — Number of parallel environments

Methods

source method MaskableDictRolloutBuffer.reset() → None

Raises

  • TypeError

source method MaskableDictRolloutBuffer.add(*args, action_masks: np.ndarray | None = None, **kwargs) → None

Add transition to buffer.

Parameters

  • *args — Positional arguments forwarded to parent add.

  • action_masks : np.ndarray | None — Masks applied to constrain the choice of possible actions.

  • **kwargs — Keyword arguments forwarded to parent add.

source method MaskableDictRolloutBuffer.get(batch_size: int | None = None) → Generator[MaskableDictRolloutBufferSamples, None, None]