burlap.domain.singleagent.gridworld

Class GridWorldRewardFunction

java.lang.Object

burlap.domain.singleagent.gridworld.GridWorldRewardFunction

All Implemented Interfaces:

RewardFunction

public class GridWorldRewardFunction
extends java.lang.Object
implements RewardFunction

This class is used for defining reward functions in grid worlds that are a function of cell of the world to which the agent transitions. That is, a double matrix (called rewardMatrix) the size of the grid world is stored. In an agent transitions to cell x,y, then they will receive the double value stored in rewardMatrix[x][y]. The rewards returned for transitioning to an agent position may be set with the setReward(int, int, double) method.

This reward function is useful for simple grid worlds without any location objects or worlds for which the rewards are independent of location objects. An alternative to this class is to define worlds with location objects and use the atLocation propositional function and location types to define rewards.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected int	height
protected double[][]	rewardMatrix
protected int	width

Constructor Summary

Constructors

Constructor and Description
GridWorldRewardFunction(Domain domain) Initializes the reward function for a grid world with a size defined by the attributes of the domain object to 0.
GridWorldRewardFunction(Domain domain, double initializingReward) Initializes the reward function for a grid world with a size defined by the attributes of the domain object to initializingReward.
GridWorldRewardFunction(int width, int height) Initializes the reward function for a grid world of size width and height and initializes the reward values everywhere to 0.
GridWorldRewardFunction(int width, int height, double initializingReward) Initializes the reward function for a grid world of size width and height and initializes the reward values everywhere to initializingReward.

Method Summary

Methods

Modifier and Type	Method and Description
double	getRewardForTransitionsTo(int x, int y) Returns the reward this reward function will return when the agent transitions to position x, y.
double[][]	getRewardMatrix() Returns the reward matrix this reward function uses.
protected void	initialize(int width, int height, double initializingReward) Initializes the reward matrix.
double	reward(State s, GroundedAction a, State sprime) Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
void	setReward(int x, int y, double r) Sets the reward the agent will receive to transitioning to position x, y

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

rewardMatrix

protected double[][] rewardMatrix

width

protected int width

height

protected int height

Constructor Detail

GridWorldRewardFunction

public GridWorldRewardFunction(int width,
                       int height,
                       double initializingReward)

Initializes the reward function for a grid world of size width and height and initializes the reward values everywhere to initializingReward. The reward returned from specific agent positions may be changed with the setReward(int, int, double) method.

Parameters:

width - the width of the grid world

height - the height of the grid world

initializingReward - the reward to which all agent position transitions are initialized to return.

GridWorldRewardFunction

public GridWorldRewardFunction(int width,
                       int height)

Initializes the reward function for a grid world of size width and height and initializes the reward values everywhere to 0. The reward returned from specific agent positions may be changed with the setReward(int, int, double) method.

Parameters:

width - the width of the grid world

height - the height of the grid world

GridWorldRewardFunction

public GridWorldRewardFunction(Domain domain,
                       double initializingReward)

Initializes the reward function for a grid world with a size defined by the attributes of the domain object to initializingReward. The reward returned from specific agent positions may be changed with the setReward(int, int, double) method.

Parameters:

domain - the domain whose x and y attributes will be used to determine the size of the grid world.

initializingReward - the reward to which all agent position transitions are initialized to return.

GridWorldRewardFunction

public GridWorldRewardFunction(Domain domain)

Initializes the reward function for a grid world with a size defined by the attributes of the domain object to 0. The reward returned from specific agent positions may be changed with the setReward(int, int, double) method.

Parameters:

domain - the domain whose x and y attributes will be used to determine the size of the grid world.

Method Detail

initialize

protected void initialize(int width,
              int height,
              double initializingReward)

Initializes the reward matrix.

Parameters:

width - the width of the grid world

height - the height of the grid world

initializingReward - the reward to which all agent position transitions are initialized to return.

getRewardMatrix

public double[][] getRewardMatrix()

Returns the reward matrix this reward function uses. Changes to the returned matrix *will* change this reward function. rewardMatrix[x][y] specifies the reward the agent will receive for transitioning to position x,y.

Returns:

the reward matrix this reward function uses

setReward

public void setReward(int x,
             int y,
             double r)

Sets the reward the agent will receive to transitioning to position x, y

Parameters:

x - the x position

y - the y position

r - the reward the agent will receive to transitioning to position x, y

getRewardForTransitionsTo

public double getRewardForTransitionsTo(int x,
                               int y)

Returns the reward this reward function will return when the agent transitions to position x, y.

Parameters:

x - the x position

y - the y position

Returns:

the reward this reward function will return when the agent transitions to position x, y.

reward

public double reward(State s,
            GroundedAction a,
            State sprime)

Description copied from interface: RewardFunction

Returns the reward received when action a is executed in state s and the agent transitions to state sprime.

Specified by:

reward in interface RewardFunction

Parameters:

s - the state in which the action was executed

a - the action executed

sprime - the state to which the agent transitioned

Returns:

the reward received when action a is executed in state s and the agent transitions to state sprime.