Skip to content

lit_sb3.policies.multi_input_actor_critic

source module lit_sb3.policies.multi_input_actor_critic

Policies: abstract base class and concrete implementations.

Classes

  • MultiInputActorCriticPolicy — MultiInputActorClass policy class for actor-critic algorithms (has both policy and value prediction).

source class MultiInputActorCriticPolicy(observation_space: spaces.Dict, action_space: spaces.Space, lr_schedule: Schedule, net_arch: list[int] | dict[str, list[int]] | None = None, activation_fn: type[nn.Module] = nn.Tanh, ortho_init: bool = True, use_sde: bool = False, log_std_init: float = 0.0, full_std: bool = True, use_expln: bool = False, squash_output: bool = False, features_extractor_class: type[BaseFeaturesExtractor] = CombinedExtractor, features_extractor_kwargs: dict[str, Any] | None = None, share_features_extractor: bool = True, normalize_images: bool = True, optimizer_class: type[torch.optim.Optimizer] = torch.optim.Adam, optimizer_kwargs: dict[str, Any] | None = None)

Bases : ActorCriticPolicy

MultiInputActorClass policy class for actor-critic algorithms (has both policy and value prediction).

Used by A2C, PPO and the likes.

Parameters

  • observation_space : spaces.Dict — Observation space (Tuple)

  • action_space : spaces.Space — Action space

  • lr_schedule : Schedule — Learning rate schedule (could be constant)

  • net_arch : list[int] | dict[str, list[int]] | None — The specification of the policy and value networks.

  • activation_fn : type[nn.Module] — Activation function

  • ortho_init : bool — Whether to use or not orthogonal initialization

  • use_sde : bool — Whether to use State Dependent Exploration or not

  • log_std_init : float — Initial value for the log standard deviation

  • full_std : bool — Whether to use (n_features x n_actions) parameters for the std instead of only (n_features,) when using gSDE

  • use_expln : bool — Use expln() function instead of exp() to ensure a positive standard deviation (cf paper). It allows to keep variance above zero and prevent it from growing too fast. In practice, exp() is usually enough.

  • squash_output : bool — Whether to squash the output using a tanh function, this allows to ensure boundaries when using gSDE.

  • features_extractor_class : type[BaseFeaturesExtractor] — Uses the CombinedExtractor

  • features_extractor_kwargs : dict[str, Any] | None — Keyword arguments to pass to the features extractor.

  • share_features_extractor : bool — If True, the features extractor is shared between the policy and value networks.

  • normalize_images : bool — Whether to normalize images or not, dividing by 255.0 (True by default)

  • optimizer_class : type[torch.optim.Optimizer] — The optimizer to use, torch.optim.Adam by default

  • optimizer_kwargs : dict[str, Any] | None — Additional keyword arguments, excluding the learning rate, to pass to the optimizer

Attributes

  • device : torch.device — Infer which device this policy lives on by inspecting its parameters.

  • squash_output : bool — (bool) Getter for squash_output.