StructuredSpacesRecord¶

class maze.core.trajectory_recording.records.structured_spaces_record.StructuredSpacesRecord(substep_records: List[maze.core.trajectory_recording.records.spaces_record.SpacesRecord] = None, event_log: Optional[maze.core.log_events.step_event_log.StepEventLog] = None, step_stats: Optional[Dict[maze.core.log_stats.log_stats.LogStatsKey, Union[int, float, numpy.ndarray, dict]]] = None, episode_stats: Optional[Dict[maze.core.log_stats.log_stats.LogStatsKey, Union[int, float, numpy.ndarray, dict]]] = None)¶

Records spaces (i.e., raw actions and observations) from a single environment step.

Individual items are structured in dictionaries, with keys corresponding to the structured sub-step IDs.

Provides helpers methods for batching, i.e. stacking of multiple spaces records to be processed by a model in a single batch.

property actions¶: List of actions from the individual sub-steps.

property actions_dict¶: Dict of actions from the sub-steps, keyed by the sub-step ID (not suitable in multi-agent scenarios).

property actor_ids¶: List of actor IDs for the individual sub-steps.

append(substep_record: maze.core.trajectory_recording.records.spaces_record.SpacesRecord) → None ¶: Append a sub-step record.

property batch_shape¶: Return whether this record is batched or not.

classmethod converted_from(state_record: maze.core.trajectory_recording.records.state_record.StateRecord, conversion_env: maze.core.env.maze_env.MazeEnv, first_step_in_episode: bool) → maze.core.trajectory_recording.records.structured_spaces_record.StructuredSpacesRecord ¶

Convert a state record (containing a Maze state and Maze action) into a spaces record (containing raw actions and observations for each sub-step).

Maze states and actions are converted to spaces using the supplied conversion env – it’s action and observation interfaces, as well as the wrapper stack determine the format of the converted actions and observations.

This is useful e.g. for behavioral cloning, when we have recorded Maze states and actions from teacher runs, and now need to convert these into raw actions and observations to be fed to a model.

Note that multi-agent scenarios are not supported yet (the conversion only support a single action-observation pair per sub-step key).

Parameters

state_record – State record to convert.
conversion_env – Environment to use for the conversion. Determines the format of the resulting spaces.
first_step_in_episode – Flag whether this is the first step in an episode (to resets stateful wrapper)

Returns

Converted spaces record.

property discounted_returns¶: List of discounted returns from the individual sub-steps.

property discounted_returns_dict¶: Dict of discounted returns from the sub-steps, keyed by the sub-step ID (not suitable in multi-agent scenarios).

property dones¶: List of dones from the individual sub-steps.

property dones_dict¶: Dict of dones from the sub-steps, keyed by the sub-step ID (not suitable in multi-agent scenarios).

episode_stats: Optional[Dict[maze.core.log_stats.log_stats.LogStatsKey, Union[int, float, numpy.ndarray, dict]]] = None¶: Aggregated statistics from the last episode. Expected to be attached only to terminal steps of episodes.

event_log: Optional[maze.core.log_events.step_event_log.StepEventLog] = None¶: Log of events recorded during the whole step.

is_batched() → bool ¶

Return whether this record is batched or not.

Returns: whether this record is batched or not

is_done() → bool ¶

Return true if the episode ended during this structured step.

Returns: true if the episode ended during this structured step

property logits¶: List of logits from the individual sub-steps.

property logits_dict¶: Dict of logits from the sub-steps, keyed by the sub-step ID (not suitable in multi-agent scenarios).

property next_observations¶: List of next observations from the individual sub-steps.

property next_observations_dict¶: Dict of next observations from the sub-steps, keyed by the sub-step ID (not suitable in multi-agent scenarios).

property observations¶: List of observations from the individual sub-steps.

property observations_dict¶: Dict of observations from the sub-steps, keyed by the sub-step ID (not suitable in multi-agent scenarios).

property rewards¶: List of rewards from the individual sub-steps.

property rewards_dict¶: Dict of rewards from the sub-steps, keyed by the sub-step ID (not suitable in multi-agent scenarios).

classmethod stack_records(records: List[StructuredSpacesRecord]) → maze.core.trajectory_recording.records.structured_spaces_record.StructuredSpacesRecord ¶

Stack multiple records into a single spaces record. Useful for processing multiple records in a batch.

All the records should be in numpy and have the same structure of the spaces (i.e. come from the same environment etc.).

Parameters: records – Records to stack.
Returns: Single stacked record, containing all the given records, and having the corresponding batch shape.

step_stats: Optional[Dict[maze.core.log_stats.log_stats.LogStatsKey, Union[int, float, numpy.ndarray, dict]]] = None¶: Statistics recorded during the whole step.

property substep_keys¶: List of sub-step keys for the individual sub-steps.

substep_records: List[maze.core.trajectory_recording.records.spaces_record.SpacesRecord] = None¶: Records for individual sub-steps (containing individual observations, action etc.)

to_numpy() → maze.core.trajectory_recording.records.structured_spaces_record.StructuredSpacesRecord ¶: Convert the record to numpy.

to_torch(device: str) → maze.core.trajectory_recording.records.structured_spaces_record.StructuredSpacesRecord ¶

Convert the record to Torch.

Parameters: device – Device to move the tensors to.
Returns: Self after conversion.