lit_sb3.callbacks.eval
source module lit_sb3.callbacks.eval
Classes
-
EvalCallback — Callback for evaluating an agent.
source class EvalCallback(eval_env: gym.Env | VecEnv, callback_on_new_best: BaseCallback | None = None, callback_after_eval: BaseCallback | None = None, n_eval_episodes: int = 5, eval_freq: int = 10000, log_path: str | Path | None = None, best_model_save_path: str | Path | None = None, deterministic: bool = True, render: bool = False, verbose: int = 1, warn: bool = True)
Bases : EventCallback
Callback for evaluating an agent.
.. warning::
When using multiple environments, each call to env.step()
will effectively correspond to n_envs steps.
To account for that, you can use eval_freq = max(eval_freq // n_envs, 1)
Parameters
-
eval_env : gym.Env | VecEnv — The environment used for initialization
-
callback_on_new_best : BaseCallback | None — Callback to trigger when there is a new best model according to the
mean_reward -
callback_after_eval : BaseCallback | None — Callback to trigger after every evaluation
-
n_eval_episodes : int — The number of episodes to test the agent
-
eval_freq : int — Evaluate the agent every
eval_freqcall of the callback. -
log_path : str | Path | None — Path to a folder where the evaluations (
evaluations.npz) will be saved. It will be updated at each evaluation. -
best_model_save_path : str | Path | None — Path to a folder where the best model according to performance on the eval env will be saved.
-
deterministic : bool — Whether the evaluation should use a stochastic or deterministic actions.
-
render : bool — Whether to render or not the environment during evaluation
-
verbose : int — Verbosity level: 0 for no output, 1 for indicating information about evaluation results
-
warn : bool — Passed to
evaluate_policy(warns ifeval_envhas not been wrapped with a Monitor wrapper)
Attributes
-
state_key : str — Identifier for the state of the callback.
Methods
-
update_child_locals — Update the references to the local variables.
source method EvalCallback.on_train_start(trainer: L.Trainer, pl_module: L.LightningModule) → None
source method EvalCallback.update_child_locals(locals_: dict[str, Any]) → None
Update the references to the local variables.
Parameters
-
locals_ : dict[str, Any] — the local variables during rollout collection