lit_sb3.utils.save

source module lit_sb3.utils.save

Save util taken from stable_baselines.

used to serialize data (class parameters) of model classes.

Functions

• recursive_getattr — Recursive version of getattr.

• recursive_setattr — Recursive version of setattr.

• is_json_serializable — Test if an object is serializable into JSON.

• data_to_json — Turn data (class parameters) into a JSON string for storing.

• json_to_data — Turn JSON serialization of class-parameters back into dictionary.

• open_path — Opens a path for reading or writing with a preferred suffix and raises debug information.

• open_path_str — Open a path given by a string. If writing to the path, the function ensures.

• open_path_pathlib — Open a path given by a string. If writing to the path, the function ensures.

• save_to_zip_file — Save model data to a zip archive.

• save_to_pkl — Save an object to path creating the necessary folders along the way.

• load_from_pkl — Load an object from the path. If a suffix is provided in the path, it will use that suffix.

• load_from_zip_file — Load model data from a .zip archive.

source recursive_getattr(obj: Any, attr: str, *args) → Any

Recursive version of getattr.

taken from https://stackoverflow.com/questions/31174295.

Ex

MyObject.sub_object = SubObject(name='test') recursive_getattr(MyObject, 'sub_object.name') # return test

Parameters

• obj : Any — Object to retrieve attribute from.

• attr : str — Attribute to retrieve.

• *args — Default value if attribute is not found.

Returns

• Any — The attribute

source recursive_setattr(obj: Any, attr: str, val: Any) → None

Recursive version of setattr.

taken from https://stackoverflow.com/questions/31174295.

Ex

MyObject.sub_object = SubObject(name='test') recursive_setattr(MyObject, 'sub_object.name', 'hello')

Parameters

• obj : Any

• attr : str — Attribute to set

• val : Any — New value of the attribute

• obj : Any — The obj parameter.

source is_json_serializable(item: Any) → bool

Test if an object is serializable into JSON.

Parameters

• item : Any — The object to be tested for JSON serialization.

Returns

• bool — True if object is JSON serializable, false otherwise.

source data_to_json(data: dict[str, Any]) → str

Turn data (class parameters) into a JSON string for storing.

Parameters

• data : dict[str, Any] — Dictionary of class parameters to be stored. Items that are not JSON serializable will be pickled with Cloudpickle and stored as bytearray in the JSON file

Returns

• str — JSON string of the data serialized.

source json_to_data(json_string: str, custom_objects: dict[str, Any] | None = None) → dict[str, Any]

Turn JSON serialization of class-parameters back into dictionary.

Parameters

• json_string : str — JSON serialization of the class-parameters that should be loaded.

• custom_objects : dict[str, Any] | None — Dictionary of objects to replace upon loading. If a variable is present in this dictionary as a key, it will not be deserialized and the corresponding item will be used instead. Similar to custom_objects in keras.models.load_model. Useful when you have an object in file that can not be deserialized.

Returns

• dict[str, Any] — Loaded class parameters.

Raises

• ValueError

source open_path(path: str | pathlib.Path | io.BufferedIOBase, mode: str, verbose: int = 0, suffix: str | None = None) → io.BufferedWriter | io.BufferedReader | io.BytesIO | io.BufferedRandom

Opens a path for reading or writing with a preferred suffix and raises debug information.

If the provided path is a derivative of io.BufferedIOBase it ensures that the file matches the provided mode, i.e. If the mode is read ("r", "read") it checks that the path is readable. If the mode is write ("w", "write") it checks that the file is writable.

If the provided path is a string or a pathlib.Path, it ensures that it exists. If the mode is "read" it checks that it exists, if it doesn't exist it attempts to read path.suffix if a suffix is provided. If the mode is "write" and the path does not exist, it creates all the parent folders. If the path points to a folder, it changes the path to path_2. If the path already exists and verbose >= 2, it raises a warning.

Parameters

• path : str | pathlib.Path | io.BufferedIOBase — the path to open. if save_path is a str or pathlib.Path and mode is "w", single dispatch ensures that the path actually exists. If path is a io.BufferedIOBase the path exists.

• mode : str — how to open the file. "w"|"write" for writing, "r"|"read" for reading.

• verbose : int — Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages

• suffix : str | None — The preferred suffix. If mode is "w" then the opened file has the suffix. If mode is "r" then we attempt to open the path. If an error is raised and the suffix is not None, we attempt to open the path with the suffix.

Raises

• TypeError

• ValueError

source open_path_str(path: str, mode: str, verbose: int = 0, suffix: str | None = None) → io.BufferedIOBase

Open a path given by a string. If writing to the path, the function ensures.

that the path exists.

Parameters

• path : str — the path to open. If mode is "w" then it ensures that the path exists by creating the necessary folders and renaming path if it points to a folder.

• mode : str — how to open the file. "w" for writing, "r" for reading.

• verbose : int — Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages

• suffix : str | None — The preferred suffix. If mode is "w" then the opened file has the suffix. If mode is "r" then we attempt to open the path. If an error is raised and the suffix is not None, we attempt to open the path with the suffix.

source open_path_pathlib(path: pathlib.Path, mode: str, verbose: int = 0, suffix: str | None = None) → io.BufferedIOBase

Open a path given by a string. If writing to the path, the function ensures.

that the path exists.

Parameters

• path : pathlib.Path — the path to check. If mode is "w" then it ensures that the path exists by creating the necessary folders and renaming path if it points to a folder.

• mode : str — how to open the file. "w" for writing, "r" for reading.

• verbose : int — Verbosity level: 0 for no output, 2 for indicating if path without suffix is not found when mode is "r"

• suffix : str | None — The preferred suffix. If mode is "w" then the opened file has the suffix. If mode is "r" then we attempt to open the path. If an error is raised and the suffix is not None, we attempt to open the path with the suffix.

Raises

• ValueError

source save_to_zip_file(save_path: str | pathlib.Path | io.BufferedIOBase, data: dict[str, Any] | None = None, params: dict[str, Any] | None = None, pytorch_variables: dict[str, Any] | None = None, verbose: int = 0) → None

Save model data to a zip archive.

Parameters

• save_path : str | pathlib.Path | io.BufferedIOBase — Where to store the model. if save_path is a str or pathlib.Path ensures that the path actually exists.

• data : dict[str, Any] | None — Class parameters being stored (non-PyTorch variables)

• params : dict[str, Any] | None — Model parameters being stored expected to contain an entry for every state_dict with its name and the state_dict.

• pytorch_variables : dict[str, Any] | None — Other PyTorch variables expected to contain name and value of the variable.

• verbose : int — Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages

source save_to_pkl(path: str | pathlib.Path | io.BufferedIOBase, obj: Any, verbose: int = 0) → None

Save an object to path creating the necessary folders along the way.

If the path exists and is a directory, it will raise a warning and rename the path. If a suffix is provided in the path, it will use that suffix, otherwise, it will use '.pkl'.

Parameters

• path : str | pathlib.Path | io.BufferedIOBase — the path to open. if save_path is a str or pathlib.Path and mode is "w", single dispatch ensures that the path actually exists. If path is a io.BufferedIOBase the path exists.

• obj : Any — The object to save.

• verbose : int — Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages

source load_from_pkl(path: str | pathlib.Path | io.BufferedIOBase, verbose: int = 0) → Any

Load an object from the path. If a suffix is provided in the path, it will use that suffix.

If the path does not exist, it will attempt to load using the .pkl suffix.

Parameters

• path : str | pathlib.Path | io.BufferedIOBase — the path to open. if save_path is a str or pathlib.Path and mode is "w", single dispatch ensures that the path actually exists. If path is a io.BufferedIOBase the path exists.

• verbose : int — Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages

source load_from_zip_file(load_path: str | pathlib.Path | io.BufferedIOBase, load_data: bool = True, custom_objects: dict[str, Any] | None = None, device: torch.device | str = 'auto', verbose: int = 0, print_system_info: bool = False) → tuple[dict[str, Any] | None, TensorDict, TensorDict | None]

Load model data from a .zip archive.

Parameters

• load_path : str | pathlib.Path | io.BufferedIOBase — Where to load the model from

• load_data : bool — Whether we should load and return data (class parameters). Mainly used by 'load_parameters' to only load model parameters (weights)

• custom_objects : dict[str, Any] | None — Dictionary of objects to replace upon loading. If a variable is present in this dictionary as a key, it will not be deserialized and the corresponding item will be used instead. Similar to custom_objects in keras.models.load_model. Useful when you have an object in file that can not be deserialized.

• device : torch.device | str — Device on which the code should run.

• verbose : int — Verbosity level: 0 for no output, 1 for info messages, 2 for debug messages

• print_system_info : bool — Whether to print or not the system info about the saved model.

Returns

• tuple[dict[str, Any] | None, TensorDict, TensorDict | None] — Class parameters, model state_dicts (aka "params", dict of state_dict) and dict of pytorch variables

Raises

• ValueError