burlap.mdp.stochasticgames.world

Class World

java.lang.Object

burlap.mdp.stochasticgames.world.World

public class World
extends java.lang.Object

This class provides a means to have agents play against each other and synchronize all of their actions and observations. Any number of agents can join a World instance and they will be told when a game is starting, when a game ends, when they need to provide an action, and what happened to all agents after every agent made their action selection. The world may also make use of an optional StateMapping object so that agents are provided an abstract and simpler representation of the world. A game can be run until a terminal state is hit, or for a specific number of stages, the latter of which is useful for repeated games.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected StateMapping	abstractionForAgents
protected HashedAggregator<java.lang.String>	agentCumulativeReward
protected java.util.List<SGAgent>	agents
protected GameEpisode	currentGameEpisodeRecord
protected State	currentState
protected int	debugId
protected SGDomain	domain
protected StateGenerator	initialStateGenerator
protected boolean	isRecordingGame
protected JointRewardFunction	jointRewardFunction
protected JointAction	lastJointAction
protected double[]	lastRewards
protected TerminalFunction	tf
protected JointModel	worldModel
protected java.util.List<WorldObserver>	worldObservers

Constructor Summary

Constructors

Constructor and Description
World(SGDomain domain, JointRewardFunction jr, TerminalFunction tf, State initialState) Initializes the world.
World(SGDomain domain, JointRewardFunction jr, TerminalFunction tf, StateGenerator sg) Initializes the world.
World(SGDomain domain, JointRewardFunction jr, TerminalFunction tf, StateGenerator sg, StateMapping abstractionForAgents) Initializes the world

Method Summary

Modifier and Type	Method and Description
void	addWorldObserver(WorldObserver ob) Adds a world observer to this world
SGAgent	agentWithName(java.lang.String name) Returns the agent with the given name, or null if there is no agent with that name.
void	clearAllWorldObserver() Clears all world observers from this world.
void	executeJointAction(JointAction ja) Manually attempts to execute a joint action in the current world state, if a game is currently not running.
boolean	gameIsRunning() Returns whether a game in this world is currently running.
void	generateNewCurrentState() Causes the world to set the current state to a state generated by the provided StateGenerator object if a game is not currently running.
JointModel	getActionModel() Returns the JointModel used in this world.
java.util.List<SGAgentType>	getAgentDefinitions() Returns the agent definitions for the agents registered in this world.
double	getCumulativeRewardForAgent(java.lang.String aname) Returns the cumulative reward that the agent with name aname has received across all interactions in this world.
State	getCurrentWorldState() Returns the current world state
int	getDebugId() This class will report execution information as games are played using the DPrint class.
SGDomain	getDomain()
JointAction	getLastJointAction() Returns the last joint action taken in this world; null if none have been taken yet.
double[]	getLastRewards() Returns the last rewards received.
int	getPlayerNumberForAgent(java.lang.String aname) Returns the player index for the agent with the given name.
java.util.List<SGAgent>	getRegisteredAgents() Returns the list of agents participating in this world.
JointRewardFunction	getRewardFunction() Returns the JointRewardFunction function used in this world.
TerminalFunction	getTF() Returns the TerminalFunction used in this world.
protected void	init(SGDomain domain, JointModel jam, JointRewardFunction jr, TerminalFunction tf, StateGenerator sg, StateMapping abstractionForAgents)
void	join(SGAgent a) Causes the provided agent to join the world
void	removeWorldObserver(WorldObserver ob) Removes the specified world observer from this world
GameEpisode	rolloutJointPolicy(JointPolicy jp, int maxStages) Rollsout a joint policy until a terminate state is reached for a maximum number of stages.
GameEpisode	rolloutJointPolicyFromState(JointPolicy jp, State s, int maxStages) Rollsout a joint policy from a given state until a terminate state is reached for a maximum number of stages.
protected void	rolloutOneStageOfJointPolicy(JointPolicy jp) Runs a single stage following a joint policy for the current world state
GameEpisode	runGame() Runs a game until a terminal state is hit.
GameEpisode	runGame(int maxStages) Runs a game until a terminal state is hit for maxStages have occurred
GameEpisode	runGame(int maxStages, State s) Runs a game starting in the input state until a terminal state is hit.
void	runStage() Runs a single stage of this game.
void	setCurrentState(State s) Sets the world state to the provided state if the a game is not currently running.
void	setDebugId(int id) Sets the debug code that is use for printing with DPrint.
void	setDomain(SGDomain domain)
boolean	worldStateIsTerminal() Returns whether the current state in the world is a terminal state or not.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

domain

protected SGDomain domain

currentState

protected State currentState

agents

protected java.util.List<SGAgent> agents

agentCumulativeReward

protected HashedAggregator<java.lang.String> agentCumulativeReward

worldModel

protected JointModel worldModel

jointRewardFunction

protected JointRewardFunction jointRewardFunction

tf

protected TerminalFunction tf

initialStateGenerator

protected StateGenerator initialStateGenerator

abstractionForAgents

protected StateMapping abstractionForAgents

lastJointAction

protected JointAction lastJointAction

worldObservers

protected java.util.List<WorldObserver> worldObservers

currentGameEpisodeRecord

protected GameEpisode currentGameEpisodeRecord

isRecordingGame

protected boolean isRecordingGame

debugId

protected int debugId

lastRewards

protected double[] lastRewards

Constructor Detail

World

public World(SGDomain domain,
             JointRewardFunction jr,
             TerminalFunction tf,
             State initialState)

Initializes the world.

Parameters:

domain - the SGDomain the world will use

jr - the joint reward function

tf - the terminal function

initialState - the initial state of the world every time a new game starts

World

public World(SGDomain domain,
             JointRewardFunction jr,
             TerminalFunction tf,
             StateGenerator sg)

Initializes the world.

Parameters:

domain - the SGDomain the world will use

jr - the joint reward function

tf - the terminal function

sg - a state generator for generating initial states of a game

World

public World(SGDomain domain,
             JointRewardFunction jr,
             TerminalFunction tf,
             StateGenerator sg,
             StateMapping abstractionForAgents)

Initializes the world

Parameters:

domain - the SGDomain the world will use

jr - the joint reward function

tf - the terminal function

sg - a state generator for generating initial states of a game

abstractionForAgents - the abstract state representation that agents will be provided

Method Detail

init

protected void init(SGDomain domain,
                    JointModel jam,
                    JointRewardFunction jr,
                    TerminalFunction tf,
                    StateGenerator sg,
                    StateMapping abstractionForAgents)

getDomain

public SGDomain getDomain()

setDomain

public void setDomain(SGDomain domain)

getDebugId

public int getDebugId()

This class will report execution information as games are played using the DPrint class. If the user wishes to suppress these messages, they can retrieve this code and suppress DPrint from printing messages that correspond to this code.

Returns:

the debug code used with DPrint.

setDebugId

public void setDebugId(int id)

Sets the debug code that is use for printing with DPrint.

Parameters:

id - the debug code to use when printing messages

getCumulativeRewardForAgent

public double getCumulativeRewardForAgent(java.lang.String aname)

Returns the cumulative reward that the agent with name aname has received across all interactions in this world.

Parameters:

aname - the name of the agent

Returns:

the cumulative reward the agent has received in this world.

join

public void join(SGAgent a)

Causes the provided agent to join the world

Parameters:

a - the agent to join

agentWithName

public SGAgent agentWithName(java.lang.String name)

Returns the agent with the given name, or null if there is no agent with that name.

Parameters:

name - the name of the agent

Returns:

the SGAgent with the name in this world, or null if there is none.

getCurrentWorldState

public State getCurrentWorldState()

Returns the current world state

Returns:

the current world state

generateNewCurrentState

public void generateNewCurrentState()

Causes the world to set the current state to a state generated by the provided StateGenerator object if a game is not currently running. If a game is currently running, then the state will not be changed.

worldStateIsTerminal

public boolean worldStateIsTerminal()

Returns whether the current state in the world is a terminal state or not.

Returns:

true if the current world state is terminal; false otherwise.

setCurrentState

public void setCurrentState(State s)

Sets the world state to the provided state if the a game is not currently running. If a game is running, then no change will occur.

Parameters:

s - the state to which the world will be set.

getLastJointAction

public JointAction getLastJointAction()

Returns the last joint action taken in this world; null if none have been taken yet.

Returns:

the last joint action taken in this world; null if none have been taken yet

getLastRewards

public double[] getLastRewards()

Returns the last rewards received.

Returns:

the last rewards reward delivered to each agent.

addWorldObserver

public void addWorldObserver(WorldObserver ob)

Adds a world observer to this world

Parameters:

ob - the observer to add

removeWorldObserver

public void removeWorldObserver(WorldObserver ob)

Removes the specified world observer from this world

Parameters:

ob - the world observer to remove

clearAllWorldObserver

public void clearAllWorldObserver()

Clears all world observers from this world.

executeJointAction

public void executeJointAction(JointAction ja)

Manually attempts to execute a joint action in the current world state, if a game is currently not running. If a game is running, then no action will be taken. Additionally, if the world is currently in a terminal state, then no action will be taken either.

Parameters:

ja - the JointAction to execute.

runGame

public GameEpisode runGame()

Runs a game until a terminal state is hit.

Returns:

a GameEpisode of the game.

runGame

public GameEpisode runGame(int maxStages)

Runs a game until a terminal state is hit for maxStages have occurred

Parameters:

maxStages - the maximum number of stages to play in the game before its forced to end. If set to -1, then run until a terminal state is hit.

Returns:

a GameEpisode of the game.

runGame

public GameEpisode runGame(int maxStages,
                           State s)

Runs a game starting in the input state until a terminal state is hit.

Parameters:

maxStages - the maximum number of stages to play in the game before its forced to end. If set to -1, then run until a terminal state is hit.

s - the input State from which the game will start

Returns:

a GameEpisode of the game.

rolloutJointPolicy

public GameEpisode rolloutJointPolicy(JointPolicy jp,
                                      int maxStages)

Rollsout a joint policy until a terminate state is reached for a maximum number of stages.

Parameters:

jp - the joint policy to rollout

maxStages - the maximum number of stages

Returns:

a GameEpisode that has recorded the result.

rolloutJointPolicyFromState

public GameEpisode rolloutJointPolicyFromState(JointPolicy jp,
                                               State s,
                                               int maxStages)

Rollsout a joint policy from a given state until a terminate state is reached for a maximum number of stages.

Parameters:

jp - the joint policy to rollout

s - the state from which the joint policy should be rolled out

maxStages - the maximum number of stages

Returns:

a GameEpisode that has recorded the result.

runStage

public void runStage()

Runs a single stage of this game.

rolloutOneStageOfJointPolicy

protected void rolloutOneStageOfJointPolicy(JointPolicy jp)

Runs a single stage following a joint policy for the current world state

Parameters:

jp - the joint policy to follow

getActionModel

public JointModel getActionModel()

Returns the JointModel used in this world.

Returns:

the JointModel used in this world.

getRewardFunction

public JointRewardFunction getRewardFunction()

Returns the JointRewardFunction function used in this world.

Returns:

the JointRewardFunction function used in this world.

getTF

public TerminalFunction getTF()

Returns the TerminalFunction used in this world.

Returns:

the TerminalFunction used in this world.

getRegisteredAgents

public java.util.List<SGAgent> getRegisteredAgents()

Returns the list of agents participating in this world.

Returns:

the list of agents participating in this world.

getAgentDefinitions

public java.util.List<SGAgentType> getAgentDefinitions()

Returns the agent definitions for the agents registered in this world.

Returns:

the agent definitions for the agents registered in this world.

getPlayerNumberForAgent

public int getPlayerNumberForAgent(java.lang.String aname)

Returns the player index for the agent with the given name.

Parameters:

aname - the name of the agent

Returns:

the player index of the agent with the given name.

gameIsRunning

public boolean gameIsRunning()

Returns whether a game in this world is currently running.

Returns:

true if a game is running, false otherwise.