public class EpisodeAnalysis
extends java.lang.Object
initializeEpisideWithInitialState(State)
method to set the initial state of the episode, before recording
any transitions. It is then advised that transitions are recorded with the recordTransitionTo(GroundedAction, State, double)
method, which takes as input
the next state to which the agent transitioned, the action applied in the last recorded state, and the reward received fro the transition.
When querying about the state, action, and reward sequences, use the getState(int)
, getAction(int)
, and getReward(int)
methods.
These methods take as input the time step of the element you want. Note that t = 0 refers to the initial state step so calling getState(0) and getAction(0)
will return the initial state and the action taken in the initial state, respectively. However, rewards are always received in the next time step
from the state and action that produced them. Therefore, getReward(0) is undefined. Instead, the first reward received will be at time step 1: getReward(1).Modifier and Type | Field and Description |
---|---|
java.util.List<GroundedAction> |
actionSequence
The sequence of actions taken
|
java.util.List<java.lang.Double> |
rewardSequence
The sequence of rewards received.
|
java.util.List<State> |
stateSequence
The sequence of states observed
|
Constructor and Description |
---|
EpisodeAnalysis()
Creates a new EpisodeAnalysis object.
|
EpisodeAnalysis(State initialState)
Initializes a new EpisodeAnalysis object with the initial state in which the episode started.
|
Modifier and Type | Method and Description |
---|---|
void |
addAction(GroundedAction ga)
Adds a GroundedAction to the action sequence.
|
void |
addReward(double r)
Adds a reward to the reward sequence.
|
void |
addState(State s)
Adds a state to the state sequence.
|
void |
appendAndMergeEpisodeAnalysis(EpisodeAnalysis e)
This method will append execution results in e to this object's results.
|
GroundedAction |
getAction(int t)
Returns the action taken in the state at time step t.
|
java.lang.String |
getActionSequenceString()
Returns a string representing the actions taken in this episode.
|
java.lang.String |
getActionSequenceString(java.lang.String delimiter)
Returns a string representing the actions taken in this episode.
|
double |
getDiscountedReturn(double discountFactor)
Will return the discounted return received from the first state in the episode to the last state in the episode.
|
double |
getReward(int t)
Returns the reward received at timestep t.
|
State |
getState(int t)
Returns the state observed at time step t.
|
protected void |
initializeDatastructures() |
void |
initializeEpisideWithInitialState(State initialState)
Initializes this object with the initial state in which the episode started.
|
static EpisodeAnalysis |
legacyParseFileIntoEA(java.lang.String path,
Domain d,
StateParser sp)
Parses a legacy BURLAP 1.0 string representation from a file and turns it into an EpisodeAnalysis object.
|
static java.util.List<EpisodeAnalysis> |
legacyParseFilesIntoEAList(java.lang.String directoryPath,
Domain d,
StateParser sp)
Parses episode files using the legacy BURLAP 1.0 string representation.
|
static EpisodeAnalysis |
legacyParseStringIntoEA(java.lang.String str,
Domain d,
StateParser sp)
Parses a legacy BURLAP 1.0 string representation and turns it into an EpisodeAnalysis object.
|
static void |
main(java.lang.String[] args) |
int |
maxTimeStep()
Returns the maximimum time step index in this episode which is the
numTimeSteps() -1. |
int |
numTimeSteps()
Returns the number of time steps in this episode, which is equivalent to the number of states.
|
static EpisodeAnalysis |
parseEpisode(Domain domain,
java.lang.String episodeString) |
static EpisodeAnalysis |
parseFileIntoEA(java.lang.String path,
Domain d)
Reads an episode that was written to a file and turns into an EpisodeAnalysis object.
|
static java.util.List<EpisodeAnalysis> |
parseFilesIntoEAList(java.lang.String directoryPath,
Domain d)
Takes a path to a directory containing .episode files and reads them all into a
List
of EpisodeAnalysis objects. |
void |
recordTransitionTo(GroundedAction usingAction,
State nextState,
double r)
Records an transition event where the agent applied the usingAction action in the last
state in this object's state sequence, transitioned to state nextState, and received reward r,.
|
void |
recordTransitionTo(State next,
GroundedAction usingAction,
double r)
Deprecated.
|
java.lang.String |
serialize() |
java.lang.String |
serialize(SerializableStateFactory serializableStateFactory) |
static void |
writeEpisodesToDisk(java.util.List<EpisodeAnalysis> episodes,
java.lang.String directoryPath,
java.lang.String baseFileName)
Takes a
List of EpisodeAnalysis objects and writes them to a directory. |
static void |
writeEpisodesToDisk(java.util.List<EpisodeAnalysis> episodes,
java.lang.String directoryPath,
java.lang.String baseFileName,
SerializableStateFactory serializableStateFactory)
Takes a
List of EpisodeAnalysis objects and writes them to a directory. |
void |
writeToFile(java.lang.String path)
Writes this episode to a file.
|
void |
writeToFile(java.lang.String path,
SerializableStateFactory serializableStateFactory)
Writes this episode to a file.
|
public java.util.List<State> stateSequence
public java.util.List<GroundedAction> actionSequence
public java.util.List<java.lang.Double> rewardSequence
public EpisodeAnalysis()
initializeEpisideWithInitialState(State)
method
should be called to set the initial state of the episode.public EpisodeAnalysis(State initialState)
initialState
- the initial state of the episodepublic void initializeEpisideWithInitialState(State initialState)
initialState
- the initial state of the episodeprotected void initializeDatastructures()
public void addState(State s)
initializeEpisideWithInitialState(State)
method
along with subsequent calls to the recordTransitionTo(GroundedAction, State, double)
method is used instead, but this
method can be used to manually add a state.s
- the state to addpublic void addAction(GroundedAction ga)
initializeEpisideWithInitialState(State)
method
along with subsequent calls to the recordTransitionTo(GroundedAction, State, double)
method is used instead, but this
method can be used to manually add a GroundedAction.ga
- the GroundedAction to addpublic void addReward(double r)
initializeEpisideWithInitialState(State)
method
along with subsequent calls to the recordTransitionTo(GroundedAction, State, double)
method is used instead, but this
method can be used to manually add a reward.r
- the reward to add@Deprecated public void recordTransitionTo(State next, GroundedAction usingAction, double r)
next
- the next state to which the agent transitionedusingAction
- the action the agent used that caused the transitionr
- the reward the agent received for this transition.public void recordTransitionTo(GroundedAction usingAction, State nextState, double r)
usingAction
- the action the agent used that caused the transitionnextState
- the next state to which the agent transitionedr
- the reward the agent received for this transition.public State getState(int t)
t
- the time step of the episodepublic GroundedAction getAction(int t)
t
- the time step of the episodepublic double getReward(int t)
t
- the time step of the episodepublic int numTimeSteps()
public int maxTimeStep()
numTimeSteps()
-1.public double getDiscountedReturn(double discountFactor)
discountFactor
- the discount factor to compute the discounted return; should be on [0, 1]public void appendAndMergeEpisodeAnalysis(EpisodeAnalysis e)
e
- the execution results to append to this episode.public java.lang.String getActionSequenceString()
public java.lang.String getActionSequenceString(java.lang.String delimiter)
delimiter
- the delimiter to separate actions in the string.public static void writeEpisodesToDisk(java.util.List<EpisodeAnalysis> episodes, java.lang.String directoryPath, java.lang.String baseFileName)
List
of EpisodeAnalysis
objects and writes them to a directory.
The format of the file names will be "baseFileName{index}.episode" where {index} represents the index of the
episode in the list. States will be serialized with a SimpleSerializableStateFactory
episodes
- the list of episodes to write to diskdirectoryPath
- the directory path in which the episodes will be writtenbaseFileName
- the base file name to use for the episode filespublic static void writeEpisodesToDisk(java.util.List<EpisodeAnalysis> episodes, java.lang.String directoryPath, java.lang.String baseFileName, SerializableStateFactory serializableStateFactory)
List
of EpisodeAnalysis
objects and writes them to a directory.
The format of the file names will be "baseFileName{index}.episode" where {index} represents the index of the
episode in the list. States will be serialized with a SimpleSerializableStateFactory
episodes
- the list of episodes to write to diskdirectoryPath
- the directory path in which the episodes will be writtenbaseFileName
- the base file name to use for the episode filespublic void writeToFile(java.lang.String path)
SimpleSerializableStateFactory
.path
- the path to the file in which to write this episode.public void writeToFile(java.lang.String path, SerializableStateFactory serializableStateFactory)
path
- the path to the file in which to write this episode.serializableStateFactory
- the SerializableStateFactory
that defines how states are to be serialized.public static java.util.List<EpisodeAnalysis> parseFilesIntoEAList(java.lang.String directoryPath, Domain d)
List
of EpisodeAnalysis
objects.directoryPath
- the path to the directory containing the episode filesd
- the domain to which the episode states and actions belongList
of EpisodeAnalysis
objects.public static EpisodeAnalysis parseFileIntoEA(java.lang.String path, Domain d)
path
- the path to the episode file.d
- the domain to which the states and actions belongpublic java.lang.String serialize()
public java.lang.String serialize(SerializableStateFactory serializableStateFactory)
public static EpisodeAnalysis parseEpisode(Domain domain, java.lang.String episodeString)
public static java.util.List<EpisodeAnalysis> legacyParseFilesIntoEAList(java.lang.String directoryPath, Domain d, StateParser sp)
List
of EpisodeAnalysis
objects.directoryPath
- the path to the directory containing the episode filesd
- the domain to which the episode states and actions belongsp
- a state parser that can parse the state string representation in each fileList
of EpisodeAnalysis
objects.public static EpisodeAnalysis legacyParseFileIntoEA(java.lang.String path, Domain d, StateParser sp)
path
- the path to the episode file.d
- the domain to which the states and actions belongsp
- a state parser that can parse the state string representation in the filepublic static EpisodeAnalysis legacyParseStringIntoEA(java.lang.String str, Domain d, StateParser sp)
str
- a string representation of the episode.d
- the domain to which the states and actions belongsp
- a state parser that can parse the state string representation in the filepublic static void main(java.lang.String[] args)