class maze.train.trainers.imitation.bc_validation_evaluator.BCValidationEvaluator(loss: maze.train.trainers.imitation.bc_loss.BCLoss, model_selection: Optional[maze.train.trainers.common.model_selection.model_selection_base.ModelSelectionBase], data_loader: torch.utils.data.DataLoader, logging_prefix: str = 'eval')

Evaluates a given policy on validation data.

Expects that the first two items returned in the dataset tuple are the observation_dict and action_dict.

  • data_loader – The data used for evaluation.

  • loss – Loss function to be used.

  • model_selection – Model selection interface that will be notified of the recorded rewards.

evaluate(policy: maze.core.agent.torch_policy.TorchPolicy)None

(overrides Evaluator)

Evaluate given policy (results are stored in stat logs) and dump the model if the reward improved.

param policy

Policy to evaluate