Environment Wrappers¶
Environment wrappers are an elegant way to modify and customize environments for RL training and experimentation. As the name already suggests, they wrap an existing environment and allow to modify different parts of the agent-environment interaction loop including observations, actions, the reward or any other internals of the environment itself.
To gain access to the functionality of Maze environment wrappers you simply have to add a wrapper stack in your hydra configuration. To get started just copy one of our hydra config snippets below or use it directly within Python.
Note
Wrappers have been already introduced in OpenAi’s Gym and elegantly expose methods and attributes of all nested envs. However, wrapping destroys the class hierarchy, querying the base classes is not straight-forward. Maze environment wrappers fix the behaviour of isinstance() for arbitrarily nested wrappers.
List of Features¶
Maze environment wrappers allows the following:
Easy customization of environments: (observations, actions, reward, internals)
Convenient development of concepts such as observation pre-processing and observation normalization.
Preserves the class hierarchy of nested environments.
Example 1: Customizing Environments with Wrappers¶
To make use of Maze environment wrappers just add a config snippet as listed below.
# @package wrappers
RandomResetWrapper:
min_skip_steps: 0
max_skip_steps: 100
maze.core.wrappers.time_limit_wrapper.TimeLimitWrapper:
max_episode_steps: 1000
Details:
It applies the specified wrappers in the defined order from top to bottom.
Adds a RandomResetWrapper randomly skipping the first 0 to 100 frames
Adds a TimeLimitWrapper restricting the maximum temporal horizon of the environment
Example 2: Using Custom Wrappers¶
In case the built-in wrappers provided with Maze are not sufficient for your use case you can of course implement and add additional custom wrappers.
# @package wrappers
my_project.wrappers.custom_wrapper.CustomObserverWrapper:
parameter_1: 0.5
parameter_2: 1000
When adding a new environment wrappers you (1) have to implement the Wrapper interface and (2) make sure that it is accessible within your Python path. Besides that you only have to provide the reference path of the wrapper to use, plus any parameters the wrapper initializer needs.
Example 3: Plain Python Configuration¶
If you are not working with the Maze command line interface but still want to use wrappers directly within Python you can start with the code snippet below.
"""Contains an example showing how to add wrappers."""
from maze.core.wrappers.random_reset_wrapper import RandomResetWrapper
from maze.core.wrappers.time_limit_wrapper import TimeLimitWrapper
# instantiate the environment
env = ...
# apply wrappers
env = RandomResetWrapper.wrap(env, min_skip_steps=0, max_skip_steps=100)
env = TimeLimitWrapper.wrap(env, max_episode_steps=1000)
Built-in Wrappers¶
Maze already comes with built-in environment wrappers. You can find a list and further details on the functionality of the respective wrappers in the reference documentation.
For the following wrappers we also provide a more extensive documentation:
Where to Go Next¶
For further details please see the reference documentation.
Special wrapper for observation pre-processing and observation normalization.
You might also want to read up on the Maze environment hierarchy.