Skip to content

lit_sb3.envs.bit_flipping_env

source module lit_sb3.envs.bit_flipping_env

Classes

source class BitFlippingEnv(n_bits: int = 10, continuous: bool = False, max_steps: int | None = None, discrete_obs_space: bool = False, image_obs_space: bool = False, channel_first: bool = True, render_mode: str = 'human')

Bases : Env

Simple bit flipping env, useful to test HER.

The goal is to flip all the bits to get a vector of ones. In the continuous variant, if the ith action component has a value > 0, then the ith bit will be flipped. Uses a MultiBinary observation space by default.

Parameters

  • n_bits : int — Number of bits to flip

  • continuous : bool — Whether to use the continuous actions version or not, by default, it uses the discrete one

  • max_steps : int | None — Max number of steps, by default, equal to n_bits

  • discrete_obs_space : bool — Whether to use the discrete observation version or not, ie a one-hot encoding of all possible states

  • image_obs_space : bool — Whether to use an image observation version or not, ie a greyscale image of the state

  • channel_first : bool — Whether to use channel-first or last image.

Attributes

  • unwrapped : Env[ObsType, ActType] — Returns the base non-wrapped environment.

  • np_random_seed : int — attr:_np_random_seed that if not set will first initialise with a random int as seed.

  • np_random : np.random.Generator — attr:_np_random that if not set will initialise with a random seed.

Methods

source method BitFlippingEnv.seed(seed: int) → None

source method BitFlippingEnv.convert_if_needed(state: np.ndarray) → int | np.ndarray

Convert to discrete space if needed.

Parameters

  • state : np.ndarray

  • state : np.ndarray — The state to restore.

source method BitFlippingEnv.convert_to_bit_vector(state: int | np.ndarray, batch_size: int) → np.ndarray

Convert to bit vector if needed.

Parameters

  • state : int | np.ndarray — The state to be converted, which can be either an integer or a numpy array.

  • batch_size : int — The batch size.

Returns

  • np.ndarray — The state converted into a bit vector.

source method BitFlippingEnv.reset(*, seed: int | None = None, options: dict | None = None) → tuple[dict[str, int | np.ndarray], dict]

source method BitFlippingEnv.step(action: np.ndarray | int) → GymStepReturn

Step into the env.

Parameters

  • action : np.ndarray | int

  • action : np.ndarray | int — The action to execute in the environment.

source method BitFlippingEnv.compute_reward(achieved_goal: int | np.ndarray, desired_goal: int | np.ndarray, _info: dict[str, Any] | None) → np.float32

source method BitFlippingEnv.render() → np.ndarray | None

source method BitFlippingEnv.close() → None