public class EpisodeAnalysis
extends java.lang.Object
initializeEpisideWithInitialState(State)
method to set the initil state of the episode, before recording
any transitions. It is then advised that transitions are recorded with the recordTransitionTo(GroundedAction, State, double)
method, which takes as input
the next state to which the agent transitioned, the action applied in the last recorded state, and the reward received fro the transition.
When querying about the state, action, and reward sequences, use the getState(int)
, getAction(int)
, and getReward(int)
methods.
These methods take as input the time step of the element you want. Note that t = 0 refers to the initial state step so calling getState(0) and getAction(0)
will return the initial state and the action taken in the initial state, respectively. However, rewards are always received in the next time step
from the state and action that produced them. Therefore, getReward(0) is undefined. Instead, the first reward received will be at time step 1: getReward(1).Modifier and Type | Field and Description |
---|---|
java.util.List<GroundedAction> |
actionSequence
The sequence of actions taken
|
java.util.List<java.lang.Double> |
rewardSequence
The sequence of rewards received.
|
java.util.List<State> |
stateSequence
The sequence of states observed
|
Constructor and Description |
---|
EpisodeAnalysis()
Creates a new EpisodeAnalysis object.
|
EpisodeAnalysis(State initialState)
Initializes a new EpisodeAnalysis object with the initial state in which the episode started.
|
Modifier and Type | Method and Description |
---|---|
void |
addAction(GroundedAction ga)
Adds a GroundedAction to the action sequence.
|
void |
addReward(double r)
Adds a reward to the reward sequence.
|
void |
addState(State s)
Adds a state to the state sequence.
|
void |
appendAndMergeEpisodeAnalysis(EpisodeAnalysis e)
This method will append execution results in e to this object's results.
|
GroundedAction |
getAction(int t)
Returns the action taken in the state at time step t.
|
java.lang.String |
getActionSequenceString()
Returns a string representing the actions taken in this episode.
|
java.lang.String |
getActionSequenceString(java.lang.String delimiter)
Returns a string representing the actions taken in this episode.
|
double |
getDiscountedReturn(double discountFactor)
Will return the discounted return received from the first state in the episode to the last state in the episode.
|
double |
getReward(int t)
Returns the reward received at timestep t.
|
State |
getState(int t)
Returns the state observed at time step t.
|
protected void |
initializeDatastructures() |
void |
initializeEpisideWithInitialState(State initialState)
Initializes this object with the initial state in which the episode started.
|
int |
maxTimeStep()
Returns the maximimum time step index in this episode which is the
numTimeSteps() -1. |
int |
numTimeSteps()
Returns the number of time steps in this episode, which is equivalent to the number of states.
|
static EpisodeAnalysis |
parseFileIntoEA(java.lang.String path,
Domain d,
StateParser sp)
Reads an episode that was written to a file and turns into an EpisodeAnalysis object.
|
static java.util.List<EpisodeAnalysis> |
parseFilesIntoEAList(java.lang.String directoryPath,
Domain d,
StateParser sp)
Takes a path to a directory containing .episode files and reads them all into a
List
of EpisodeAnalysis objects. |
java.lang.String |
parseIntoString(StateParser sp)
Converts this episode into a string representation.
|
static EpisodeAnalysis |
parseStringIntoEA(java.lang.String str,
Domain d,
StateParser sp)
Parses a string representation of an episode into an EpisodeAnalysis object.
|
void |
recordTransitionTo(GroundedAction usingAction,
State nextState,
double r)
Records an transition event where the agent applied the usingAction action in the last
state in this object's state sequence, transitioned to state nextState, and received reward r,.
|
void |
recordTransitionTo(State next,
GroundedAction usingAction,
double r)
Deprecated.
|
static void |
writeEpisodesToDisk(java.util.List<EpisodeAnalysis> episodes,
java.lang.String directoryPath,
java.lang.String baseFileName,
int zeroPaddedDigits,
StateParser sp)
Takes a
List of EpisodeAnalysis objects and writes them to a directory. |
void |
writeToFile(java.lang.String path,
StateParser sp)
Writes this episode to a file.
|
public java.util.List<State> stateSequence
public java.util.List<GroundedAction> actionSequence
public java.util.List<java.lang.Double> rewardSequence
public EpisodeAnalysis()
initializeEpisideWithInitialState(State)
method
should be called to set the initial state of the episode.public EpisodeAnalysis(State initialState)
initialState
- the initial state of the episodepublic void initializeEpisideWithInitialState(State initialState)
initialState
- the initial state of the episodeprotected void initializeDatastructures()
public void addState(State s)
initializeEpisideWithInitialState(State)
method
along with subsequent calls to the recordTransitionTo(GroundedAction, State, double)
method is used instead, but this
method can be used to manually add a state.s
- the state to addpublic void addAction(GroundedAction ga)
initializeEpisideWithInitialState(State)
method
along with subsequent calls to the recordTransitionTo(GroundedAction, State, double)
method is used instead, but this
method can be used to manually add a GroundedAction.ga
- the GroundedAction to addpublic void addReward(double r)
initializeEpisideWithInitialState(State)
method
along with subsequent calls to the recordTransitionTo(GroundedAction, State, double)
method is used instead, but this
method can be used to manually add a reward.r
- the reward to add@Deprecated public void recordTransitionTo(State next, GroundedAction usingAction, double r)
next
- the next state to which the agent transitionedusingAction
- the action the agent used that caused the transitionr
- the reward the agent received for this transition.public void recordTransitionTo(GroundedAction usingAction, State nextState, double r)
usingAction
- the action the agent used that caused the transitionnextState
- the next state to which the agent transitionedr
- the reward the agent received for this transition.public State getState(int t)
t
- the time step of the episodepublic GroundedAction getAction(int t)
t
- the time step of the episodepublic double getReward(int t)
t
- the time step of the episodepublic int numTimeSteps()
public int maxTimeStep()
numTimeSteps()
-1.public double getDiscountedReturn(double discountFactor)
discountFactor
- the discount factor to compute the discounted return; should be on [0, 1]public void appendAndMergeEpisodeAnalysis(EpisodeAnalysis e)
e
- the execution results to append to this episode.public java.lang.String getActionSequenceString()
public java.lang.String getActionSequenceString(java.lang.String delimiter)
delimiter
- the delimiter to separate actions in the string.public static void writeEpisodesToDisk(java.util.List<EpisodeAnalysis> episodes, java.lang.String directoryPath, java.lang.String baseFileName, int zeroPaddedDigits, StateParser sp)
List
of EpisodeAnalysis
objects and writes them to a directory.
The format of the file names will be "baseFileName{index}.episode" where {index} represents the index of the
episode in the list. Furthermore, zeroPaddedDigits specifies the minimum number of digits in the {index} string
(using leading zeros to force the size). For example,
if baseName = "ep" and zeroPaddedDigits = 3, then the episode at index 2 will have the name: "ep002.episode"
If you set zeroPaddedDigits = 0, then the number of digits will be forced to fit the maximum number of digits
necessary to represent the last episode index in the list. For example, if the list has 22 episodes,
then zeroPaddedDigits = 0 will behave like zeroPaddedDigits = 2.episodes
- the list of episodes to write to diskdirectoryPath
- the directory path in which the episodes will be writtenbaseFileName
- the base file name to use for the episode fileszeroPaddedDigits
- the minimum number of digitssp
- the state parse used to convert states objects t ostring representationspublic void writeToFile(java.lang.String path, StateParser sp)
path
- the path to the file in which to write this episode.sp
- the state parser to use to convert state objects to string representations.public java.lang.String parseIntoString(StateParser sp)
sp
- the state parser to use to convert state objects to string representations.public static java.util.List<EpisodeAnalysis> parseFilesIntoEAList(java.lang.String directoryPath, Domain d, StateParser sp)
List
of EpisodeAnalysis
objects.directoryPath
- the path to the directory containing the episode filesd
- the domain to which the episode states and actions belongsp
- a state parser that can parse the state string representation in each fileList
of EpisodeAnalysis
objects.public static EpisodeAnalysis parseFileIntoEA(java.lang.String path, Domain d, StateParser sp)
path
- the path to the episode file.d
- the domain to which the states and actions belongsp
- a state parser that can parse the state string representation in the filepublic static EpisodeAnalysis parseStringIntoEA(java.lang.String str, Domain d, StateParser sp)
str
- a string represenation of the episode.d
- the domain to which the states and actions belongsp
- a state parser that can parse the state string representation in the file