public class DeterministicTerminationOption extends Option
PolicyDefinedSubgoalOption
when the client needs more control over in which states the option can be initiated and in which states it terminates.
One other advantage of this option is that it can be passed a valueFunction from which its policy will be created. The valueFunction
will be called from each initiation state provided to the option constructor.FullActionModel.FullActionModelHelper
cachedExpectations, cachedExpectedRewards, cumulativeDiscount, discountFactor, expectationSearchCutoffProb, expectationStateHashingFactory, externalTerminalFunction, keepTrackOfReward, lastCumulativeReward, lastNumSteps, lastOptionExecutionResults, rand, rf, shouldAnnotateExecution, shouldRecordResults, stateMapping, terminateMapper
actionObservers, domain, name
Constructor and Description |
---|
DeterministicTerminationOption(java.lang.String name,
Policy p,
StateConditionTest init,
StateConditionTest terminationStates)
Initializes.
|
DeterministicTerminationOption(java.lang.String name,
StateConditionTestIterable init,
StateConditionTest terminaitonStates,
Planner planner,
SolverDerivedPolicy p)
Initializes the option by creating the policy uses some provided option.
|
DeterministicTerminationOption(java.lang.String name,
StateConditionTest init,
StateConditionTest terminationStates,
java.util.List<State> seedStatesForPlanning,
Planner planner,
SolverDerivedPolicy p)
Initializes the option by creating the policy uses some provided option.
|
Modifier and Type | Method and Description |
---|---|
boolean |
applicableInState(State st,
GroundedAction groundedAction)
Returns true if this action can be applied in this specified state with the parameters
specified by the provided
GroundedAction
Default behavior is that an action can be applied in any state,
but this will need be overridden if that is not the case. |
boolean |
enumerable()
Returns true if the initiation states and termination states of this option are iterable; false if either of them are not.
|
java.util.List<Policy.ActionProb> |
getActionDistributionForState(State s,
GroundedAction groundedAction)
Returns the option's policy distribution for a given state.
|
java.util.List<GroundedAction> |
getAllApplicableGroundedActions(State s)
Returns all possible groundings of this action that can be applied in the provided
State . |
GroundedAction |
getAssociatedGroundedAction()
Returns a
GroundedAction instance that points to this Action ,
but does not have any parameters--if any--set. |
StateConditionTest |
getInitiationTest()
Returns the object defining the initiation states.
|
StateConditionTest |
getTerminiationStates()
Returns the object defining the termination states.
|
void |
initiateInStateHelper(State s,
GroundedAction groundedAction)
This method is always called when an option is initiated and begins execution.
|
boolean |
isMarkov()
Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.
|
boolean |
isParameterized()
Returns true if this action is parameterized; false otherwise.
|
GroundedAction |
oneStepActionSelection(State s,
GroundedAction groundedAction)
This method causes the option to select a single step in the given state, when the option was initiated with the provided parameters.
|
double |
probabilityOfTermination(State s,
GroundedAction groundedAction)
Returns the probability that this option (executed with the given parameters) will terminate in the given state
|
boolean |
usesDeterministicPolicy()
Returns whether this option's policy is deterministic or stochastic
|
boolean |
usesDeterministicTermination()
Returns whether this option's termination conditions are deterministic or stochastic
|
accumulateDiscountedProb, continueFromState, getDeterministicPolicy, getExpectedRewards, getLastCumulativeReward, getLastExecutionResults, getLastNumSteps, getTransitions, initiateInState, isAnnotatingExecutionResults, isPrimitive, isRecordingExecutionResults, iterateExpectationScan, keepTrackOfRewardWith, map, oneStep, oneStep, performActionHelper, performInEnvironment, setExernalTermination, setExpectationCalculationProbabilityCutoff, setExpectationHashingFactory, setStateMapping, setTerminateMapper, toggleShouldAnnotateResults, toggleShouldRecordResults
addActionObserver, clearAllActionsObservers, deterministicTransition, equals, getAllApplicableGroundedActionsFromActionList, getDomain, getGroundedAction, getName, hashCode, performAction
public DeterministicTerminationOption(java.lang.String name, Policy p, StateConditionTest init, StateConditionTest terminationStates)
name
- the name of the optionp
- the option's policyinit
- the initiation states of the optionterminationStates
- the deterministic termination states of the option.public DeterministicTerminationOption(java.lang.String name, StateConditionTestIterable init, StateConditionTest terminaitonStates, Planner planner, SolverDerivedPolicy p)
StateConditionTestIterable
) and then
sets this options policy to the valueFunction derived policy that is provided.name
- the name of the optioninit
- the iterable initiation statesterminaitonStates
- the termination states of the optionplanner
- the valueFunction to be used to create the policy for this optionp
- the valueFunction derived policy to use after planning from each initial state is performed.public DeterministicTerminationOption(java.lang.String name, StateConditionTest init, StateConditionTest terminationStates, java.util.List<State> seedStatesForPlanning, Planner planner, SolverDerivedPolicy p)
seedStatesForPlanning
and then
sets this options policy to the valueFunction derived policy that is provided.name
- the name of the optioninit
- the initiation conditions of the optionterminationStates
- the termination states of the optionseedStatesForPlanning
- the states that should be used as initial states for the valueFunctionplanner
- the valueFunction that is used to create the policy for this optionp
- the valueFunction derived policy to use after planning from each initial state is performed.public boolean isParameterized()
Action
isParameterized
in class Action
Action
is parameterized; false if it is not.public GroundedAction getAssociatedGroundedAction()
Action
GroundedAction
instance that points to this Action
,
but does not have any parameters--if any--set.getAssociatedGroundedAction
in class Action
GroundedAction
instance.public java.util.List<GroundedAction> getAllApplicableGroundedActions(State s)
Action
State
. To check if a grounded
action is applicable in the state, the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction)
method is checked.
The default behavior of this method is to treat the parameters as possible object bindings, finding all bindings
that satisfy the object class typing specified and then checking them against the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction)
method. However, this class can also be overridden to provide custom
grounding behavior or non-object based parametrization.getAllApplicableGroundedActions
in class Action
s
- the State
in which all applicable grounded actions of this Action
object should be returned.GroundedAction
s of this Action
object in in the given State
public StateConditionTest getInitiationTest()
public StateConditionTest getTerminiationStates()
public boolean enumerable()
public boolean isMarkov()
Option
public boolean usesDeterministicTermination()
Option
usesDeterministicTermination
in class Option
public boolean usesDeterministicPolicy()
Option
usesDeterministicPolicy
in class Option
public double probabilityOfTermination(State s, GroundedAction groundedAction)
Option
probabilityOfTermination
in class Option
s
- the state to test for terminationgroundedAction
- the parameters in which this option was initiatedpublic boolean applicableInState(State st, GroundedAction groundedAction)
Action
GroundedAction
Default behavior is that an action can be applied in any state,
but this will need be overridden if that is not the case.applicableInState
in class Action
st
- the state to perform the action ongroundedAction
- the GroundedAction
specifying the parameters to usepublic void initiateInStateHelper(State s, GroundedAction groundedAction)
Option
Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction)
For Markov options, this method probably does not need to do anything, but for non-Markov options, like Macro actions, it may need
to initialize some structures for determining termination and action selection.initiateInStateHelper
in class Option
s
- the state in which the option was initiatedgroundedAction
- the parameters in which this option will be initiatedpublic GroundedAction oneStepActionSelection(State s, GroundedAction groundedAction)
Option
Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction)
method until it is determined that the option terminates.oneStepActionSelection
in class Option
s
- the state in which an action should be selected.groundedAction
- the parameters in which this option was initiateds
public java.util.List<Policy.ActionProb> getActionDistributionForState(State s, GroundedAction groundedAction)
Option
getActionDistributionForState
in class Option
s
- the state for which this option's policy distribution should be returnedgroundedAction
- the parameters in which this option was initiated