lit_sb3.envs

source package lit_sb3.envs

Classes

• BitFlippingEnv — Simple bit flipping env, useful to test HER.

• FakeImageEnv — Fake image environment for testing purposes, it mimics Atari games.

• IdentityEnv — Identity environment for testing purposes.

• IdentityEnvBox — Identity environment for testing purposes.

• IdentityEnvMultiBinary — Identity environment for testing purposes.

• IdentityEnvMultiDiscrete — Identity environment for testing purposes.

• SimpleMultiObsEnv — Base class for GridWorld-based MultiObs Environments 4x4 grid world.

source class BitFlippingEnv(n_bits: int = 10, continuous: bool = False, max_steps: int | None = None, discrete_obs_space: bool = False, image_obs_space: bool = False, channel_first: bool = True, render_mode: str = 'human')

Bases : Env

Simple bit flipping env, useful to test HER.

The goal is to flip all the bits to get a vector of ones. In the continuous variant, if the ith action component has a value > 0, then the ith bit will be flipped. Uses a MultiBinary observation space by default.

Parameters

• n_bits : int — Number of bits to flip

• continuous : bool — Whether to use the continuous actions version or not, by default, it uses the discrete one

• max_steps : int | None — Max number of steps, by default, equal to n_bits

• discrete_obs_space : bool — Whether to use the discrete observation version or not, ie a one-hot encoding of all possible states

• image_obs_space : bool — Whether to use an image observation version or not, ie a greyscale image of the state

• channel_first : bool — Whether to use channel-first or last image.

Attributes

• unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

• np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

• np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

Methods

• seed

• convert_if_needed — Convert to discrete space if needed.

• convert_to_bit_vector — Convert to bit vector if needed.

• reset

• step — Step into the env.

• compute_reward

• render

• close

source method BitFlippingEnv.seed(seed: int) → None

source method BitFlippingEnv.convert_if_needed(state: np.ndarray) → int | np.ndarray

Convert to discrete space if needed.

Parameters

• state : np.ndarray

• state : np.ndarray — The state to restore.

source method BitFlippingEnv.convert_to_bit_vector(state: int | np.ndarray, batch_size: int) → np.ndarray

Convert to bit vector if needed.

Parameters

• state : int | np.ndarray — The state to be converted, which can be either an integer or a numpy array.

• batch_size : int — The batch size.

Returns

• np.ndarray — The state converted into a bit vector.

source method BitFlippingEnv.reset(*, seed: int | None = None, options: dict | None = None) → tuple[dict[str, int | np.ndarray], dict]

source method BitFlippingEnv.step(action: np.ndarray | int) → GymStepReturn

Step into the env.

Parameters

• action : np.ndarray | int

• action : np.ndarray | int — The action to execute in the environment.

source method BitFlippingEnv.compute_reward(achieved_goal: int | np.ndarray, desired_goal: int | np.ndarray, _info: dict[str, Any] | None) → np.float32

source method BitFlippingEnv.render() → np.ndarray | None

source method BitFlippingEnv.close() → None

source class FakeImageEnv(action_dim: int = 6, screen_height: int = 84, screen_width: int = 84, n_channels: int = 1, discrete: bool = True, channel_first: bool = False)

Bases : gym.Env

Fake image environment for testing purposes, it mimics Atari games.

Parameters

• action_dim : int — Number of discrete actions

• screen_height : int — Height of the image

• screen_width : int — Width of the image

• n_channels : int — Number of color channels

• discrete : bool — Create discrete action space instead of continuous

• channel_first : bool — Put channels on first axis instead of last

Attributes

• unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

• np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

• np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

Methods

• reset

• step

• render

source method FakeImageEnv.reset(*, seed: int | None = None, options: dict | None = None) → tuple[np.ndarray, dict]

source method FakeImageEnv.step(action: np.ndarray | int) → GymStepReturn

source method FakeImageEnv.render(mode: str = 'human') → None

source class IdentityEnv[T: (int, np.ndarray)](dim: int | None = None, space: spaces.Space | None = None, ep_length: int = 100)

Bases : gym.Env

Identity environment for testing purposes.

Parameters

• dim : int | None — the size of the action and observation dimension you want to learn. Provide at most one of dim and space. If both are None, then initialization proceeds with dim=1 and space=None.

• space : spaces.Space | None — the action and observation space. Provide at most one of dim and space.

• ep_length : int — the length of each episode in timesteps

Attributes

• unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

• np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

• np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

Methods

• reset

• step

• render

source method IdentityEnv.reset(*, seed: int | None = None, options: dict | None = None) → tuple[T, dict]

source method IdentityEnv.step(action: T) → tuple[T, float, bool, bool, dict[str, Any]]

source method IdentityEnv.render(mode: str = 'human') → None

source class IdentityEnvBox(low: float = -1.0, high: float = 1.0, eps: float = 0.05, ep_length: int = 100)

Bases : IdentityEnv[np.ndarray]

Identity environment for testing purposes.

Parameters

• low : float — the lower bound of the box dim

• high : float — the upper bound of the box dim

• eps : float — the epsilon bound for correct value

• ep_length : int — the length of each episode in timesteps

Attributes

• unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

• np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

• np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

Methods

• step

source method IdentityEnvBox.step(action: np.ndarray) → tuple[np.ndarray, float, bool, bool, dict[str, Any]]

source class IdentityEnvMultiBinary(dim: int = 1, ep_length: int = 100)

Bases : IdentityEnv[np.ndarray]

Identity environment for testing purposes.

Parameters

• dim : int — the size of the dimensions you want to learn

• ep_length : int — the length of each episode in timesteps

Attributes

• unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

• np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

• np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

source class IdentityEnvMultiDiscrete(dim: int = 1, ep_length: int = 100)

Bases : IdentityEnv[np.ndarray]

Identity environment for testing purposes.

Parameters

• dim : int — the size of the dimensions you want to learn

• ep_length : int — the length of each episode in timesteps

Attributes

• unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

• np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

• np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

source class SimpleMultiObsEnv(num_col: int = 4, num_row: int = 4, random_start: bool = True, discrete_actions: bool = True, channel_last: bool = True)

Bases : gym.Env

Base class for GridWorld-based MultiObs Environments 4x4 grid world.

 ____________
| 0  1  2   3|
| 4|¯5¯¯6¯| 7|
| 8|_9_10_|11|
|12 13  14 15|
¯¯¯¯¯¯¯¯¯¯¯¯¯¯

start is 0 states 5, 6, 9, and 10 are blocked goal is 15 actions are = [left, down, right, up]

simple linear state env of 15 states but encoded with a vector and an image observation

each column is represented by a random vector and each row is represented by a random image, both sampled once at creation time.

Parameters

• num_col : int — Number of columns in the grid

• num_row : int — Number of rows in the grid

• random_start : bool — If true, agent starts in random position

• channel_last : bool — If true, the image will be channel last, else it will be channel first

Attributes

• unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

• np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

• np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

Methods

• init_state_mapping — Initializes the state_mapping array which holds the observation values for each state.

• get_state_mapping — Uses the state to get the observation mapping.

• init_possible_transitions — Initializes the transitions of the environment.

• step — Run one timestep of the environment's dynamics. When end of.

• render — Prints the log of the environment.

• reset — Resets the environment state and step count and returns reset observation.

source method SimpleMultiObsEnv.init_state_mapping(num_col: int, num_row: int) → None

Initializes the state_mapping array which holds the observation values for each state.

Parameters

• num_col : int — Number of columns.

• num_row : int — Number of rows.

source method SimpleMultiObsEnv.get_state_mapping() → dict[str, np.ndarray]

Uses the state to get the observation mapping.

Returns

• observation dict {'vec' — ..., 'img': ...}

source method SimpleMultiObsEnv.init_possible_transitions() → None

Initializes the transitions of the environment.

The environment exploits the cardinal directions of the grid by noting that they correspond to simple addition and subtraction from the cell id within the grid.

• up => means moving up a row => means subtracting the length of a column
• down => means moving down a row => means adding the length of a column
• left => means moving left by one => means subtracting 1
• right => means moving right by one => means adding 1

Thus one only needs to specify in which states each action is possible in order to define the transitions of the environment

source method SimpleMultiObsEnv.step(action: int | np.ndarray) → GymStepReturn

Run one timestep of the environment's dynamics. When end of.

episode is reached, you are responsible for calling reset() to reset this environment's state. Accepts an action and returns a tuple (observation, reward, terminated, truncated, info).

Parameters

• action : int | np.ndarray

• action : int | np.ndarray — The action to execute in the environment.

Returns

• GymStepReturn — tuple (observation, reward, terminated, truncated, info).

source method SimpleMultiObsEnv.render(mode: str = 'human') → None

Prints the log of the environment.

Parameters

• mode : str

• mode : str — The render mode to use.

source method SimpleMultiObsEnv.reset(*, seed: int | None = None, options: dict | None = None) → tuple[dict[str, np.ndarray], dict]

Resets the environment state and step count and returns reset observation.

Parameters

• seed : int | None — Random seed for reproducibility.

• options : dict | None — Additional options passed to the environment reset.

Returns

• observation dict {'vec' — ..., 'img': ...}