Perception Module

This page contains the reference documentation of Maze Perception Module.

maze.perception.blocks

These are basic neural network building blocks and interfaces:

PerceptionBlock

Interface for all perception blocks.

ShapeNormalizationBlock

Perception block normalizing the input and de-normalizing the output tensor dimensions.

InferenceBlock

An inference block combining multiple perception blocks into one prediction module.

InferenceGraph

Models a perception module inference graph.

Feed Forward: these are built-in feed forward building blocks:

DenseBlock

A block containing multiple subsequent dense layers.

VGGConvolutionBlock

A block containing multiple subsequent vgg style convolutions.

StridedConvolutionBlock

A block containing multiple subsequent strided convolution layers.

GraphConvBlock

A block containing multiple subsequent graph convolution stacks.

GraphAttentionBlock

A block containing multiple subsequent graph (multi-head) attention stacks.

MultiHeadAttentionBlock

Implementation of a torch MultiHeadAttention block.

PointNetFeatureBlock

PointNet block allowing to embed a variable sized set of point observations into a fixed size feature vector via the PointNet mechanics.

Recurrent: these are built-in recurrent building blocks:

LSTMBlock

A block containing multiple subsequent LSTM layers followed by a final time-distributed dense layer with explicit non-linearity.

General: these are build-in general purpose building blocks:

FlattenBlock

A flattening block.

CorrelationBlock

A feature correlation block.

ConcatenationBlock

A feature concatenation block.

FunctionalBlock

A block applying a custom callable.

GlobalAveragePoolingBlock

A global average pooling block.

MaskedGlobalPoolingBlock

A block applying global pooling with optional masking.

MultiIndexSlicingBlock

A multi-index-slicing block.

RepeatToMatchBlock

A repeat-to-match block.

SelfAttentionConvBlock

Implementation of a self-attention block as described by reference: https://arxiv.org/abs/1805.08318

SelfAttentionSeqBlock

Implementation of a self-attention block as described by reference: https://arxiv.org/abs/1706.03762

SliceBlock

A slicing block.

ActionMaskingBlock

An action masking block.

TorchModelBlock

A block transforming a common nn.Module to a shape-normalized Maze perception block.

Joint: these are build in joint building blocks combining multiple perception blocks:

FlattenDenseBlock

A block containing a flattening stage followed by a dense layer block.

VGGConvolutionDenseBlock

A block containing multiple subsequent vgg style convolution stacks followed by flattening and a dense layer block.

VGGConvolutionGAPBlock

A block containing multiple subsequent vgg style convolution stacks followed by global average pooling.

StridedConvolutionDenseBlock

A block containing multiple subsequent strided convolutions followed by flattening and a dense layer block.

LSTMLastStepBlock

A block containing a LSTM perception block followed by a Slicing Block keeping only the output of the final time step.

maze.perception.builders

These are template model builders:

BaseModelBuilder

Base class for perception default model builders.

ConcatModelBuilder

A model builder that first processes individual observations, concatenates the resulting latent spaces and then processes this concatenated output to action and value outputs.

maze.perception.models

These are model composers and components:

BaseModelComposer

Abstract baseclass and interface definitions for model composers.

TemplateModelComposer

Composes template models from configs.

CustomModelComposer

Composes models from explicit model definitions.

SpacesConfig

Represents configuration of environment spaces (action & observation) used for model config.

These are maze.perception.models.policies

BasePolicyComposer

Interface for policy (actor) network composers.

ProbabilisticPolicyComposer

Composes networks for probabilistic policies.

There are maze.perception.models.critics

CriticComposerInterface

Interface for critic (value function) network composers.

BaseStateCriticComposer

Interface for critic (value function) network composers.

SharedStateCriticComposer

One critic is shared across all sub-steps or actors (default to use for standard gym-style environments).

StepStateCriticComposer

Each sub-step or actor gets its individual critic.

DeltaStateCriticComposer

First sub step gets a regular critic, subsequent sub-steps predict a delta w.r.t.

StateCriticComposer

alias of maze.perception.models.critics.step_state_critic_composer.StepStateCriticComposer

BaseStateActionCriticComposer

Interface for state action (Q) critic network composers.

SharedStateActionCriticComposer

One critic is shared across all sub-steps or actors (default to use for standard gym-style environments).

StepStateActionCriticComposer

Each sub-step or actor gets its individual critic.

StateActionCriticComposer

alias of maze.perception.models.critics.step_state_action_critic_composer.StepStateActionCriticComposer

These are maze.perception.models.build_in models

FlattenConcatBaseNet

Base flatten and concatenation model for policies and critics.

FlattenConcatPolicyNet

Flatten and concatenation policy model.

FlattenConcatStateValueNet

Flatten and concatenation state value model.

maze.perception.perception_utils

These are some helper functions when working with the perception module:

observation_spaces_to_in_shapes

Convert an observation space to the input shapes for the neural networks

flatten_spaces

Merges an iterable of dictionary spaces (usually observations or actions from subsequent sub-steps) into a single dictionary containing all the items.

stack_and_flatten_spaces

Merges an iterable of dictionary spaces (usually observations or actions from subsequent sub-steps) into a single dictionary containing all the items.

convert_to_torch

Converts any struct to torch.Tensors.

convert_to_numpy

Convert torch to np

maze.perception.weight_init

These are some helper functions for initializing model weights:

make_module_init_normc

Compiles normc weight initialization function initializing module weights with normc_initializer and biases with zeros.

compute_sigmoid_bias

Compute the bias value for a sigmoid activation function such as in multi-binary action spaces (Bernoulli distributions).