burlap.domain.singleagent.gridworld

Class GridWorldDomain

java.lang.Object

burlap.domain.singleagent.gridworld.GridWorldDomain

All Implemented Interfaces:

DomainGenerator

Direct Known Subclasses:

MacroCellGridWorld

public class GridWorldDomain
extends java.lang.Object
implements DomainGenerator

A domain generator for basic grid worlds. This domain generator allows for the creation of arbitrarily sized grid worlds with user defined layouts. The grid world supports classic north, south, east, west movement actions that may be either deterministic or stochastic with user defined stochastic failures.
The domain consists of only two object classes: an agent class and a location class, each of which is defined by and x and y position. Locations also have an attribute defining which type of location it is. The number of possible types can be set using the setNumberOfLocationTypes(int) method. Setting location types may be useful if terminating "pits" and goal locations exist in the world
Walls are not considered objects. Instead walls are considered part of the transition dynamics. There are 2 types of walls supported. Walls that are more like obstacles and occupy an entire cell of the map and 1D walls. 1D walls are specified as either a horizontal wall on the north side of a cell or a vertical wall on the east side of the wall. A cell may also have a 1D north and east wall in it. The type of wall for each cell is specified by a 2D int matrix provided to the constructor. Cells in the matrix with a 0 are clear of any walls and obstacle; 1s indicate a full cell obstacle; 2s a 1D north wall; 3s a 1D east wall; and 4s indicate that the cell has both a 1D north wall and 1D east wall.
Note that if you change the stochastic transition dynamics or the map of the domain generator *after* generating a domain with generateDomain(), the previously generated domain will use the settings prior to its generation. To use the new settings, you will need to generate a new domain object.
There are five propositional functions supported: atLocation(agent, location), wallToNorth(agent), wallToSouth(agent), wallToEast(agent), and wallToWest(agent).

Author:

James MacGlashan

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	GridWorldDomain.AtLocationPF Propositional function for determining if the agent is at the same position as a given location object
class	GridWorldDomain.MovementAction Action class for movement actions in grid world.
class	GridWorldDomain.WallToPF Propositional function for indicating if a wall is in a given position relative to the agent position

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	ACTIONEAST Constant for the name of the east action
static java.lang.String	ACTIONNORTH Constant for the name of the north action
static java.lang.String	ACTIONSOUTH Constant for the name of the south action
static java.lang.String	ACTIONWEST Constant for the name of the west action
static java.lang.String	ATTLOCTYPE Constant for the name of attribute for location object type
static java.lang.String	ATTX Constant for the name of the x attribute
static java.lang.String	ATTY Constant for the name of the y attribute
static java.lang.String	CLASSAGENT Constant for the name of the agent class
static java.lang.String	CLASSLOCATION Constant for the name of the location class
protected int	height The height of grid world
protected int[][]	map The wall map where the first index is the x position and the second index is the y position.
protected int	numLocationTypes The number of possible location types
static java.lang.String	PFATLOCATION Constant for the name of the at location propositional function
static java.lang.String	PFWALLEAST Constant for the name of the wall to east propositional function
static java.lang.String	PFWALLNORTH Constant for the name of the wall to north propositional function
static java.lang.String	PFWALLSOUTH Constant for the name of the wall to south propositional function
static java.lang.String	PFWALLWEST Constant for the name of the wall to west propositional function
protected double[][]	transitionDynamics Matrix specifying the transition dynamics in terms of movement directions.
protected int	width The width of the grid world

Constructor Summary

Constructors

Constructor and Description
GridWorldDomain(int[][] map) Constructs a deterministic world based on the provided map.
GridWorldDomain(int width, int height) Constructs an empty map with deterministic transitions

Method Summary

Methods

Modifier and Type	Method and Description
void	clearLocationOfWalls(int x, int y) Removes any obstacles or walls at the specified location.
Domain	generateDomain() Returns a newly instanced Domain object
static ValueFunctionVisualizerGUI	getGridWorldValueFunctionVisualization(java.util.List<State> states, ValueFunction valueFunction, Policy p) Creates and returns a ValueFunctionVisualizerGUI object for a grid world.
int	getHeight() Returns this grid world's height
int[][]	getMap() Returns a deep copy of the map being used for the domain
static State	getOneAgentNLocationState(Domain d, int n) Will return a state object with a single agent object and n location objects
static State	getOneAgentNoLocationState(Domain d) Will return a state object with a single agent object and no location objects
static State	getOneAgentNoLocationState(Domain d, int ax, int ay) Will return a state object with a single agent object and no location objects
static State	getOneAgentOneLocationState(Domain d) Will return a state object with a single agent object and a single location object
int	getWidth() Returns this grid world's width
void	horizontal1DNorthWall(int xi, int xf, int y) Creates a sequence of 1D north walls spanning the specified start and end x coordinates.
void	horizontalWall(int xi, int xf, int y) Creates a sequence of complete cell walls spanning the specified start and end x coordinates.
static void	main(java.lang.String[] args) Creates a visual explorer or terminal explorer.
void	makeEmptyMap() Makes the map empty
protected State	move(State s, int xd, int yd, int[][] map) Attempts to move the agent into the given position, taking into account walls and blocks
protected int[]	movementDirectionFromIndex(int i) Returns the change in x and y position for a given direction number.
void	set1DEastWall(int x, int y) Sets a specified location to have a 1D east wall.
void	set1DNorthWall(int x, int y) Sets a specified location to have a 1D north wall.
static void	setAgent(State s, int x, int y) Sets the first agent object in s to the specified x and y position.
void	setCellWallState(int x, int y, int wallType) Sets the map at the specified location to have the specified wall configuration.
void	setDeterministicTransitionDynamics() Will set the domain to use deterministic action transitions.
static void	setLocation(State s, int i, int x, int y) Sets the i'th location object to the specified x and y position.
static void	setLocation(State s, int i, int x, int y, int locType) Sets the i'th location object to the specified x and y position and location type.
void	setMap(int[][] map) Set the map of the world.
void	setMapToFourRooms() Will set the map of the world to the classic Four Rooms map used the original options work (Sutton, R.S.
void	setNumberOfLocationTypes(int numLocationTypes) Sets the number of possible location types to which a location object can belong.
void	setObstacleInCell(int x, int y) Sets a complete cell obstacle in the designated location.
void	setProbSucceedTransitionDynamics(double probSucceed) Sets the domain to use probabilistic transitions.
void	setTransitionDynamics(double[][] transitionDynamics) Will set the movement direction probabilities based on the action chosen.
void	vertical1DEastWall(int yi, int yf, int x) Creates a sequence of 1D east walls spanning the specified start and end y coordinates.
void	verticalWall(int yi, int yf, int x) Creates a sequence of complete cell walls spanning the specified start and end y coordinates

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ATTX

public static final java.lang.String ATTX

Constant for the name of the x attribute

See Also:

Constant Field Values

ATTY

public static final java.lang.String ATTY

Constant for the name of the y attribute

See Also:

Constant Field Values

ATTLOCTYPE

public static final java.lang.String ATTLOCTYPE

Constant for the name of attribute for location object type

See Also:

Constant Field Values

CLASSAGENT

public static final java.lang.String CLASSAGENT

Constant for the name of the agent class

See Also:

Constant Field Values

CLASSLOCATION

public static final java.lang.String CLASSLOCATION

Constant for the name of the location class

See Also:

Constant Field Values

ACTIONNORTH

public static final java.lang.String ACTIONNORTH

Constant for the name of the north action

See Also:

Constant Field Values

ACTIONSOUTH

public static final java.lang.String ACTIONSOUTH

Constant for the name of the south action

See Also:

Constant Field Values

ACTIONEAST

public static final java.lang.String ACTIONEAST

Constant for the name of the east action

See Also:

Constant Field Values

ACTIONWEST

public static final java.lang.String ACTIONWEST

Constant for the name of the west action

See Also:

Constant Field Values

PFATLOCATION

public static final java.lang.String PFATLOCATION

Constant for the name of the at location propositional function

See Also:

Constant Field Values

PFWALLNORTH

public static final java.lang.String PFWALLNORTH

Constant for the name of the wall to north propositional function

See Also:

Constant Field Values

PFWALLSOUTH

public static final java.lang.String PFWALLSOUTH

Constant for the name of the wall to south propositional function

See Also:

Constant Field Values

PFWALLEAST

public static final java.lang.String PFWALLEAST

Constant for the name of the wall to east propositional function

See Also:

Constant Field Values

PFWALLWEST

public static final java.lang.String PFWALLWEST

Constant for the name of the wall to west propositional function

See Also:

Constant Field Values

width

protected int width

The width of the grid world

height

protected int height

The height of grid world

numLocationTypes

protected int numLocationTypes

The number of possible location types

map

protected int[][] map

The wall map where the first index is the x position and the second index is the y position. Values of 1 indicate a wall is there, values of 0 indicate an empty cell

transitionDynamics

protected double[][] transitionDynamics

Matrix specifying the transition dynamics in terms of movement directions. The first index indicates the action direction attempted (ordered north, south, east, west) the second index indicates the actual resulting direction the agent will go (assuming there is no wall in the way). The value is the probability of that outcome. The existence of walls does not affect the probability of the direction the agent will actually go, but if a wall is in the way, it will affect the outcome. For instance, if the agent selects north, but there is a 0.2 probability of actually going east and there is a wall to the east, then with 0.2 probability, the agent will stay in place.

Constructor Detail

GridWorldDomain

public GridWorldDomain(int width,
               int height)

Constructs an empty map with deterministic transitions

Parameters:

width - width of the map

height - height of the map

GridWorldDomain

public GridWorldDomain(int[][] map)

Constructs a deterministic world based on the provided map.

Parameters:

map - the first index is the x index, the second the y; 1 entries indicate a wall

Method Detail

setNumberOfLocationTypes

public void setNumberOfLocationTypes(int numLocationTypes)

Sets the number of possible location types to which a location object can belong. The default is 1.

Parameters:

numLocationTypes - the number of possible location types to which a location object can belong.

setDeterministicTransitionDynamics

public void setDeterministicTransitionDynamics()

Will set the domain to use deterministic action transitions.

setProbSucceedTransitionDynamics

public void setProbSucceedTransitionDynamics(double probSucceed)

Sets the domain to use probabilistic transitions. Agent will move in the intended direction with probability probSucceed. Agent will move in a random direction with probability 1 - probSucceed

Parameters:

probSucceed - probability to move the in intended direction

setTransitionDynamics

public void setTransitionDynamics(double[][] transitionDynamics)

Will set the movement direction probabilities based on the action chosen. The index (0,1,2,3) indicates the direction north,south,east,west, respectively and the matrix is organized by transitionDynamics[selectedDirection][actualDirection]. For instance, the probability of the agent moving east when selecting north would be specified in the entry transitionDynamics[0][2]

Parameters:

transitionDynamics - entries indicate the probability of movement in the given direction (second index) for the given action selected (first index).

makeEmptyMap

public void makeEmptyMap()

Makes the map empty

setMap

public void setMap(int[][] map)

Set the map of the world.

Parameters:

map - the first index is the x index, the second the y; 1 entries indicate a wall

setMapToFourRooms

public void setMapToFourRooms()

Will set the map of the world to the classic Four Rooms map used the original options work (Sutton, R.S. and Precup, D. and Singh, S., 1999).

horizontalWall

public void horizontalWall(int xi,
                  int xf,
                  int y)

Creates a sequence of complete cell walls spanning the specified start and end x coordinates.

Parameters:

xi - The starting x coordinate of the wall

xf - The ending x coordinate of the wall

y - The y coordinate of the wall

verticalWall

public void verticalWall(int yi,
                int yf,
                int x)

Creates a sequence of complete cell walls spanning the specified start and end y coordinates

Parameters:

yi - The stating y coordinate of the wall

yf - The ending y coordinate of the wall

x - The x coordinate of the wall

horizontal1DNorthWall

public void horizontal1DNorthWall(int xi,
                         int xf,
                         int y)

Creates a sequence of 1D north walls spanning the specified start and end x coordinates. If any of the cells spanned already have a east wall set in that location, then the cell is set to have both an east wall and a north wall.

Parameters:

xi - The starting x coordinate of the wall

xf - The ending x coordinate of the wall

y - The y coordinate of the wall

vertical1DEastWall

public void vertical1DEastWall(int yi,
                      int yf,
                      int x)

Creates a sequence of 1D east walls spanning the specified start and end y coordinates. If any of the cells spanned already have a 1D north wall set in that location, then the cell is set to have both a north wall and an east wall.

Parameters:

yi - The stating y coordinate of the wall

yf - The ending y coordinate of the wall

x - The x coordinate of the wall

setObstacleInCell

public void setObstacleInCell(int x,
                     int y)

Sets a complete cell obstacle in the designated location.

Parameters:

x - the x coordinate of the obstacle

y - the y coordinate of the obstacle

set1DNorthWall

public void set1DNorthWall(int x,
                  int y)

Sets a specified location to have a 1D north wall. If the specified cell already has a 1D east wall set in that location, then the cell is set to have both an east wall and a north wall.

Parameters:

x - the x coordinate of the location to have the north wall

y - the y coordinate of the location to have the north wall

set1DEastWall

public void set1DEastWall(int x,
                 int y)

Sets a specified location to have a 1D east wall. If the specified cell already has a 1D north wall set in that location, then the cell is set to have both a north wall and an east wall.

Parameters:

x - the x coordinate of the location to have the east wall

y - the y coordinate of the location to have the east wall

clearLocationOfWalls

public void clearLocationOfWalls(int x,
                        int y)

Removes any obstacles or walls at the specified location.

Parameters:

x - the x coordinate of the location

y - the y coordinate of the location

setCellWallState

public void setCellWallState(int x,
                    int y,
                    int wallType)

Sets the map at the specified location to have the specified wall configuration.

Parameters:

x - the x coordinate of the location

y - the y coordinate of the location

wallType - the wall configuration for this location. 0 = no walls; 1 = complete cell wall/obstacle; 2 = 1D north wall; 3 = 1D east wall; 4 = 1D north *and* east wall

getMap

public int[][] getMap()

Returns a deep copy of the map being used for the domain

Returns:

a deep copy of the map being used in the domain

getWidth

public int getWidth()

Returns this grid world's width

Returns:

this grid world's width

getHeight

public int getHeight()

Returns this grid world's height

Returns:

this grid world's height

generateDomain

public Domain generateDomain()

Description copied from interface: DomainGenerator

Returns a newly instanced Domain object

Specified by:

generateDomain in interface DomainGenerator

Returns:

the newly instantiated Domain object.

getOneAgentNoLocationState

public static State getOneAgentNoLocationState(Domain d)

Will return a state object with a single agent object and no location objects

Parameters:

d - the domain object that is used to specify the min/max dimensions

Returns:

a state object with a single agent object and no location objects

getOneAgentNoLocationState

public static State getOneAgentNoLocationState(Domain d,
                               int ax,
                               int ay)

Will return a state object with a single agent object and no location objects

Parameters:

d - the domain object that is used to specify the min/max dimensions

ax - the starting x position of the agent

ay - the starting y position of the agent

Returns:

a state object with a single agent object and no location objects

getOneAgentOneLocationState

public static State getOneAgentOneLocationState(Domain d)

Will return a state object with a single agent object and a single location object

Parameters:

d - the domain object that is used to specify the min/max dimensions

Returns:

a state object with a single agent object and a single location object

getOneAgentNLocationState

public static State getOneAgentNLocationState(Domain d,
                              int n)

Will return a state object with a single agent object and n location objects

Parameters:

d - the domain object that is used to specify the min/max dimensions

n - the number of location objects

Returns:

a state object with a single agent object and n location objects

setAgent

public static void setAgent(State s,
            int x,
            int y)

Sets the first agent object in s to the specified x and y position.

Parameters:

s - the state with the agent whose position to set

x - the x position of the agent

y - the y position of the agent

setLocation

public static void setLocation(State s,
               int i,
               int x,
               int y)

Sets the i'th location object to the specified x and y position. The location type will be set to 0.

Parameters:

s - the state with the location object

i - specifies which location object index to set

x - the x position of the location

y - the y position of the location

setLocation

public static void setLocation(State s,
               int i,
               int x,
               int y,
               int locType)

Sets the i'th location object to the specified x and y position and location type.

Parameters:

s - the state with the location object

i - specifies which location object index to set

x - the x position of the location

y - the y position of the location

locType - the location type of the location

getGridWorldValueFunctionVisualization

public static ValueFunctionVisualizerGUI getGridWorldValueFunctionVisualization(java.util.List<State> states,
                                                                ValueFunction valueFunction,
                                                                Policy p)

Creates and returns a ValueFunctionVisualizerGUI object for a grid world. The value of states will be represented by colored cells from red (lowest value) to blue (highest value). North-south-east-west actions will be rendered with arrows using ArrowActionGlyph objects. The GUI will not be launched by default; call the ValueFunctionVisualizerGUI.initGUI() on the returned object to start it.

Parameters:

states - the states whose value should be rendered.

valueFunction - the value Function that can return the state values.

p - the policy to render

Returns:

a gridworld-based ValueFunctionVisualizerGUI object.

move

protected State move(State s,
         int xd,
         int yd,
         int[][] map)

Attempts to move the agent into the given position, taking into account walls and blocks

Parameters:

s - the current state

xd - the attempted new X position of the agent

yd - the attempted new Y position of the agent

movementDirectionFromIndex

protected int[] movementDirectionFromIndex(int i)

Returns the change in x and y position for a given direction number.

Parameters:

i - the direction number (0,1,2,3 indicates north,south,east,west, respectively)

Returns:

the change in direction for x and y; the first index of the returned double is change in x, the second index is change in y.

main

public static void main(java.lang.String[] args)

Creates a visual explorer or terminal explorer. By default a visual explorer is presented; use the "t" argument to create terminal explorer. Will create a 4 rooms grid world with the agent in lower left corner and a location in the upper right. Use w-a-s-d to move.

Parameters:

args -