Skip to content

lit_sb3.policies.recurrent.recurrent_actor_critic_cnn

source module lit_sb3.policies.recurrent.recurrent_actor_critic_cnn

Classes

source class RecurrentActorCriticCnnPolicy(observation_space: spaces.Space, action_space: spaces.Space, lr_schedule: Schedule, net_arch: list[int] | dict[str, list[int]] | None = None, activation_fn: type[nn.Module] = nn.Tanh, ortho_init: bool = True, use_sde: bool = False, log_std_init: float = 0.0, full_std: bool = True, use_expln: bool = False, squash_output: bool = False, features_extractor_class: type[BaseFeaturesExtractor] = NatureCNN, features_extractor_kwargs: dict[str, Any] | None = None, share_features_extractor: bool = True, normalize_images: bool = True, optimizer_class: type[torch.optim.Optimizer] = torch.optim.Adam, optimizer_kwargs: dict[str, Any] | None = None, lstm_hidden_size: int = 256, n_lstm_layers: int = 1, shared_lstm: bool = False, enable_critic_lstm: bool = True, lstm_kwargs: dict[str, Any] | None = None)

Bases : RecurrentActorCriticPolicy

CNN recurrent policy class for actor-critic algorithms (has both policy and value prediction).

Used by A2C, PPO and the likes.

Parameters

  • observation_space : spaces.Space — Observation space

  • action_space : spaces.Space — Action space

  • lr_schedule : Schedule — Learning rate schedule (could be constant)

  • net_arch : list[int] | dict[str, list[int]] | None — The specification of the policy and value networks.

  • activation_fn : type[nn.Module] — Activation function

  • ortho_init : bool — Whether to use or not orthogonal initialization

  • use_sde : bool — Whether to use State Dependent Exploration or not

  • log_std_init : float — Initial value for the log standard deviation

  • full_std : bool — Whether to use (n_features x n_actions) parameters for the std instead of only (n_features,) when using gSDE

  • use_expln : bool — Use expln() function instead of exp() to ensure a positive standard deviation (cf paper). It allows to keep variance above zero and prevent it from growing too fast. In practice, exp() is usually enough.

  • squash_output : bool — Whether to squash the output using a tanh function, this allows to ensure boundaries when using gSDE.

  • features_extractor_class : type[BaseFeaturesExtractor] — Features extractor to use.

  • features_extractor_kwargs : dict[str, Any] | None — Keyword arguments to pass to the features extractor.

  • share_features_extractor : bool — If True, the features extractor is shared between the policy and value networks.

  • normalize_images : bool — Whether to normalize images or not, dividing by 255.0 (True by default)

  • optimizer_class : type[torch.optim.Optimizer] — The optimizer to use, torch.optim.Adam by default

  • optimizer_kwargs : dict[str, Any] | None — Additional keyword arguments, excluding the learning rate, to pass to the optimizer

  • lstm_hidden_size : int — Number of hidden units for each LSTM layer.

  • n_lstm_layers : int — Number of LSTM layers.

  • shared_lstm : bool — Whether the LSTM is shared between the actor and the critic. By default, only the actor has a recurrent network.

  • enable_critic_lstm : bool — Use a seperate LSTM for the critic.

  • lstm_kwargs : dict[str, Any] | None — Additional keyword arguments to pass the the LSTM constructor.

Attributes

  • device : torch.device — Infer which device this policy lives on by inspecting its parameters.

  • squash_output : bool — (bool) Getter for squash_output.