GameEpisode

java.lang.Object
- burlap.behavior.stochasticgames.GameEpisode

```
public class GameEpisode
extends java.lang.Object
```
This class provides a means to record all the interactions in a stochastic game; specifically, the sequence of states, joint actions taken, and joint reward received. It also has methods to converting everything to a string/file and parsing a string/file back into this object so that results can be recorded and saved to disk.
This class should be used either by constructing with an initial state (GameEpisode(State)) or by constructing with the default constructor and then using the initializeInState(State) method before recording any further transitions. Transitions should then be recorded with the transition(JointAction, State, double[]) method which takes as input the next state to which the agent transitions, the joint action taken in the previously recorded state that causes the transition, and the joint reward received for the transition.
When querying about the state, joint action, or joint rewards, use the methods state(int), jointAction(int), and jointReward(int) respectively. These methods take as input the time step of the element you want. Note that t = 0 refers to the initial state step so calling getState(0) and getJointAction(0) will return the initial state and the joint action taken in the initial state, respectively. However, joint rewards are always received in the next time step from the state and action that produced them. Therefore, getJointReward(0) is undefined. Instead, the first reward received will be at time step 1: getReward(1). Additionally, the action and reward for a specific agent in a specific time step can be queried with agentAction(int, int) and agentReward(int, int), respectively.

Author:

James MacGlashan

Field Summary

Fields
Modifier and Type	Field and Description
`java.util.List<JointAction>`	`jointActions` The sequence of joint actions
`java.util.List<double[]>`	`jointRewards` The sequence of joint rewards
`java.util.List<State>`	`states` The sequence of states

Constructor Summary

Constructors
Constructor and Description

GameEpisode()
Initialzes the datastructures.

GameEpisode(State initialState)
Initializes with an initial state of the game.

Constructors
Constructor and Description
`GameEpisode()` Initialzes the datastructures.
`GameEpisode(State initialState)` Initializes with an initial state of the game.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`Action`	`agentAction(int t, int agentNum)` Returns the action taken for the given agent at the given time step where t=0 refers to the joint action taken in the initial state.
`double`	`agentReward(int t, int agentNum)` Returns the reward received for the agent with the given name at the given step.
`java.util.List<JointAction>`	`getJointActions()` Returns the joint action sequence list object
`java.util.List<double[]>`	`getJointRewards()` Returns the joint reward sequence list object
`java.util.List<State>`	`getStates()` Returns the state sequence list object
`void`	`initializeInState(State initialState)` Clears out any already recorded states, joint actions, and rewards, and sets the initial state of the game.
`JointAction`	`jointAction(int t)` Returns the joint action taken in time step t where t=0 refers to the joint action taken in the initial state.
`double[]`	`jointReward(int t)` Returns the joint reward received in time step t.
`static void`	`main(java.lang.String[] args)`
`int`	`maxTimeStep()` Returns the max time step index in this game which equals `numTimeSteps()`-1.
`int`	`numTimeSteps()` Returns the number of time steps recorded which is equal to the number of states observed.
`static GameEpisode`	`parse(java.lang.String episodeString)`
`static GameEpisode`	`read(java.lang.String path)` Reads a game that was written to a file and turns into a `GameEpisode` object.
`java.lang.String`	`serialize()`
`State`	`state(int t)` Returns the state stored at time step t where t=0 refers to the initial state
`void`	`transition(JointAction jointAction, State nextState, double[] jointReward)` Records a transition from the last recorded state in this object using the specififed joint action to the specified next state and with the specified joint reward being received as a result.
`void`	`write(java.lang.String path)` Writes this game to a file.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - states
```
public java.util.List<State> states
```
    The sequence of states
  - jointActions
```
public java.util.List<JointAction> jointActions
```
    The sequence of joint actions
  - jointRewards
```
public java.util.List<double[]> jointRewards
```
    The sequence of joint rewards
- Constructor Detail
  - GameEpisode
```
public GameEpisode()
```
    Initialzes the datastructures. Note that the method initializeInState(State) should be called to set the initial state of the game before any transitions are recorded.
  - GameEpisode
```
public GameEpisode(State initialState)
```
    Initializes with an initial state of the game.
    
    Parameters:
    
    initialState - the initial state of the game.
- Method Detail
  - initializeInState
```
public void initializeInState(State initialState)
```
    Clears out any already recorded states, joint actions, and rewards, and sets the initial state of the game.
    
    Parameters:
    
    initialState - the initial state of the game.
  - state
```
public State state(int t)
```
    Returns the state stored at time step t where t=0 refers to the initial state
    
    Parameters:
    
    t - the time step
    
    Returns:
    
    the state at time step t
  - jointAction
```
public JointAction jointAction(int t)
```
    Returns the joint action taken in time step t where t=0 refers to the joint action taken in the initial state.
    
    Parameters:
    
    t - the time step
    
    Returns:
    
    the joint action taken in time step t
  - jointReward
```
public double[] jointReward(int t)
```
    Returns the joint reward received in time step t. Note that rewards are always returned in the time step after the joint action that generated them; therefore, this method is undefined for t = 0. Instead, the first time step with a reward is t=1 which refers to the joint reward received after the first joint action is taken in the initial state.
    
    Parameters:
    
    t - the time step
    
    Returns:
    
    the joint reward received at time step t
  - agentAction
```
public Action agentAction(int t,
                          int agentNum)
```
    Returns the action taken for the given agent at the given time step where t=0 refers to the joint action taken in the initial state. if there is no action taken by the agent with the given name, then a runtime exception is thrown.
    
    Parameters:
    
    t - the time step
    
    agentNum - the agent number for the action to be returned
    
    Returns:
    
    the action taken by the specified agent in the given time step
  - agentReward
```
public double agentReward(int t,
                          int agentNum)
```
    Returns the reward received for the agent with the given name at the given step. Note that rewards are always returned in the time step after the joint action that generated them; therefore, this method is undefined for t = 0. Instead, the first time step with a reward is t=1 which refers to the reward received after the first joint action is taken in the initial state.
    
    Parameters:
    
    t - the time step
    
    agentNum - the agent for whom the reward should be returned
    
    Returns:
    
    the reward received by the agent
  - numTimeSteps
```
public int numTimeSteps()
```
    Returns the number of time steps recorded which is equal to the number of states observed.
    
    Returns:
    
    the number of time steps recorded.
  - maxTimeStep
```
public int maxTimeStep()
```
    Returns the max time step index in this game which equals numTimeSteps()-1.
    
    Returns:
    
    the max time step index in this game
  - transition
```
public void transition(JointAction jointAction,
                       State nextState,
                       double[] jointReward)
```
    Records a transition from the last recorded state in this object using the specififed joint action to the specified next state and with the specified joint reward being received as a result.
    
    Parameters:
    
    jointAction - the joint action taken in the last recorded state in this object
    
    nextState - the next state to which the agents transition
    
    jointReward - the joint reward received for the transiton
  - getStates
```
public java.util.List<State> getStates()
```
    Returns the state sequence list object
    
    Returns:
    
    the state sequence list object
  - getJointActions
```
public java.util.List<JointAction> getJointActions()
```
    Returns the joint action sequence list object
    
    Returns:
    
    the joint action sequence list object
  - getJointRewards
```
public java.util.List<double[]> getJointRewards()
```
    Returns the joint reward sequence list object
    
    Returns:
    
    the joint reward sequence list object
  - serialize
```
public java.lang.String serialize()
```
  - parse
```
public static GameEpisode parse(java.lang.String episodeString)
```
  - write
```
public void write(java.lang.String path)
```
    Writes this game to a file. If the the directory for the specified file path do not exist, then they will be created. If the file extension is not ".game" will automatically be added. States must be serializable.
    
    Parameters:
    
    path - the path to the file in which to write this game.
  - read
```
public static GameEpisode read(java.lang.String path)
```
    Reads a game that was written to a file and turns into a GameEpisode object.
    
    Parameters:
    
    path - the path to the game file.
    
    Returns:
    
    an GameEpisode object.
  - main
```
public static void main(java.lang.String[] args)
```

Class GameEpisode

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

states

jointActions

jointRewards

Constructor Detail

GameEpisode

GameEpisode

Method Detail

initializeInState

state

jointAction

jointReward

agentAction

agentReward

numTimeSteps

maxTimeStep

transition

getStates

getJointActions

getJointRewards

serialize

parse

write

read

main