burlap.behavior.singleagent.options

Class DeterministicTerminationOption

java.lang.Object

burlap.oomdp.singleagent.Action

burlap.behavior.singleagent.options.Option

burlap.behavior.singleagent.options.DeterministicTerminationOption

All Implemented Interfaces:

FullActionModel

public class DeterministicTerminationOption
extends Option

An option with a defined policy, initiation state set and termination state set. This option has deterministic termination conditions in any state where the policy is undefined, or in any state that is in the termination set. In general, this method should be used instead of the PolicyDefinedSubgoalOption when the client needs more control over in which states the option can be initiated and in which states it terminates. One other advantage of this option is that it can be passed a valueFunction from which its policy will be created. The valueFunction will be called from each initiation state provided to the option constructor.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.oomdp.singleagent.FullActionModel

FullActionModel.FullActionModelHelper

Field Summary

Fields inherited from class burlap.behavior.singleagent.options.Option

cachedExpectations, cachedExpectedRewards, cumulativeDiscount, discountFactor, expectationSearchCutoffProb, expectationStateHashingFactory, externalTerminalFunction, keepTrackOfReward, lastCumulativeReward, lastNumSteps, lastOptionExecutionResults, rand, rf, shouldAnnotateExecution, shouldRecordResults, stateMapping, terminateMapper

Fields inherited from class burlap.oomdp.singleagent.Action

actionObservers, domain, name

Constructor Summary

Constructors

Constructor and Description
DeterministicTerminationOption(java.lang.String name, Policy p, StateConditionTest init, StateConditionTest terminationStates) Initializes.
DeterministicTerminationOption(java.lang.String name, StateConditionTestIterable init, StateConditionTest terminaitonStates, Planner planner, SolverDerivedPolicy p) Initializes the option by creating the policy uses some provided option.
DeterministicTerminationOption(java.lang.String name, StateConditionTest init, StateConditionTest terminationStates, java.util.List<State> seedStatesForPlanning, Planner planner, SolverDerivedPolicy p) Initializes the option by creating the policy uses some provided option.

Method Summary

Methods

Modifier and Type	Method and Description
boolean	applicableInState(State st, GroundedAction groundedAction) Returns true if this action can be applied in this specified state with the parameters specified by the provided GroundedAction Default behavior is that an action can be applied in any state, but this will need be overridden if that is not the case.
boolean	enumerable() Returns true if the initiation states and termination states of this option are iterable; false if either of them are not.
java.util.List<Policy.ActionProb>	getActionDistributionForState(State s, GroundedAction groundedAction) Returns the option's policy distribution for a given state.
java.util.List<GroundedAction>	getAllApplicableGroundedActions(State s) Returns all possible groundings of this action that can be applied in the provided State.
GroundedAction	getAssociatedGroundedAction() Returns a GroundedAction instance that points to this Action, but does not have any parameters--if any--set.
StateConditionTest	getInitiationTest() Returns the object defining the initiation states.
StateConditionTest	getTerminiationStates() Returns the object defining the termination states.
void	initiateInStateHelper(State s, GroundedAction groundedAction) This method is always called when an option is initiated and begins execution.
boolean	isMarkov() Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.
boolean	isParameterized() Returns true if this action is parameterized; false otherwise.
GroundedAction	oneStepActionSelection(State s, GroundedAction groundedAction) This method causes the option to select a single step in the given state, when the option was initiated with the provided parameters.
double	probabilityOfTermination(State s, GroundedAction groundedAction) Returns the probability that this option (executed with the given parameters) will terminate in the given state
boolean	usesDeterministicPolicy() Returns whether this option's policy is deterministic or stochastic
boolean	usesDeterministicTermination() Returns whether this option's termination conditions are deterministic or stochastic

Methods inherited from class burlap.behavior.singleagent.options.Option

accumulateDiscountedProb, continueFromState, getDeterministicPolicy, getExpectedRewards, getLastCumulativeReward, getLastExecutionResults, getLastNumSteps, getTransitions, initiateInState, isAnnotatingExecutionResults, isPrimitive, isRecordingExecutionResults, iterateExpectationScan, keepTrackOfRewardWith, map, oneStep, oneStep, performActionHelper, performInEnvironment, setExernalTermination, setExpectationCalculationProbabilityCutoff, setExpectationHashingFactory, setStateMapping, setTerminateMapper, toggleShouldAnnotateResults, toggleShouldRecordResults

Methods inherited from class burlap.oomdp.singleagent.Action

addActionObserver, clearAllActionsObservers, deterministicTransition, equals, getAllApplicableGroundedActionsFromActionList, getDomain, getGroundedAction, getName, hashCode, performAction

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DeterministicTerminationOption

public DeterministicTerminationOption(java.lang.String name,
                              Policy p,
                              StateConditionTest init,
                              StateConditionTest terminationStates)

Initializes.

Parameters:

name - the name of the option

p - the option's policy

init - the initiation states of the option

terminationStates - the deterministic termination states of the option.

DeterministicTerminationOption

public DeterministicTerminationOption(java.lang.String name,
                              StateConditionTestIterable init,
                              StateConditionTest terminaitonStates,
                              Planner planner,
                              SolverDerivedPolicy p)

Initializes the option by creating the policy uses some provided option. The valueFunction is called repeatedly on each state in the initiation state set (which needs to be a StateConditionTestIterable) and then sets this options policy to the valueFunction derived policy that is provided.

Parameters:

name - the name of the option

init - the iterable initiation states

terminaitonStates - the termination states of the option

planner - the valueFunction to be used to create the policy for this option

p - the valueFunction derived policy to use after planning from each initial state is performed.

DeterministicTerminationOption

public DeterministicTerminationOption(java.lang.String name,
                              StateConditionTest init,
                              StateConditionTest terminationStates,
                              java.util.List<State> seedStatesForPlanning,
                              Planner planner,
                              SolverDerivedPolicy p)

Initializes the option by creating the policy uses some provided option. The valueFunction is called repeatedly on each state in the the list seedStatesForPlanning and then sets this options policy to the valueFunction derived policy that is provided.

Parameters:

name - the name of the option

init - the initiation conditions of the option

terminationStates - the termination states of the option

seedStatesForPlanning - the states that should be used as initial states for the valueFunction

planner - the valueFunction that is used to create the policy for this option

p - the valueFunction derived policy to use after planning from each initial state is performed.

Method Detail

isParameterized

public boolean isParameterized()

Description copied from class: Action

Returns true if this action is parameterized; false otherwise.

Specified by:

isParameterized in class Action

Returns:

true if this Action is parameterized; false if it is not.

getAssociatedGroundedAction

public GroundedAction getAssociatedGroundedAction()

Description copied from class: Action

Returns a GroundedAction instance that points to this Action, but does not have any parameters--if any--set.

Specified by:

getAssociatedGroundedAction in class Action

Returns:

a GroundedAction instance.

getAllApplicableGroundedActions

public java.util.List<GroundedAction> getAllApplicableGroundedActions(State s)

Description copied from class: Action

Returns all possible groundings of this action that can be applied in the provided State. To check if a grounded action is applicable in the state, the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction) method is checked. The default behavior of this method is to treat the parameters as possible object bindings, finding all bindings that satisfy the object class typing specified and then checking them against the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction) method. However, this class can also be overridden to provide custom grounding behavior or non-object based parametrization.

Specified by:

getAllApplicableGroundedActions in class Action

Parameters:

s - the State in which all applicable grounded actions of this Action object should be returned.

Returns:

a list of all applicable GroundedActions of this Action object in in the given State

getInitiationTest

public StateConditionTest getInitiationTest()

Returns the object defining the initiation states.

Returns:

the object defining the initiation states.

getTerminiationStates

public StateConditionTest getTerminiationStates()

Returns the object defining the termination states.

Returns:

the object defining the termination states.

enumerable

public boolean enumerable()

Returns true if the initiation states and termination states of this option are iterable; false if either of them are not.

Returns:

true if the initiation states and termination states of this option are iterable; false if either of them are not.

isMarkov

public boolean isMarkov()

Description copied from class: Option

Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.

Specified by:

isMarkov in class Option

Returns:

True if this option is Markov ; false otherwise.

usesDeterministicTermination

public boolean usesDeterministicTermination()

Description copied from class: Option

Returns whether this option's termination conditions are deterministic or stochastic

Specified by:

usesDeterministicTermination in class Option

Returns:

true if this option's termination conditions are deterministic; false if stochastic.

usesDeterministicPolicy

public boolean usesDeterministicPolicy()

Description copied from class: Option

Returns whether this option's policy is deterministic or stochastic

Specified by:

usesDeterministicPolicy in class Option

Returns:

true if this option's policy is deterministic; false if stochastic

probabilityOfTermination

public double probabilityOfTermination(State s,
                              GroundedAction groundedAction)

Description copied from class: Option

Returns the probability that this option (executed with the given parameters) will terminate in the given state

Specified by:

probabilityOfTermination in class Option

Parameters:

s - the state to test for termination

groundedAction - the parameters in which this option was initiated

Returns:

the probability that this option (executed with the given parameters) will terminate in the given state

applicableInState

public boolean applicableInState(State st,
                        GroundedAction groundedAction)

Description copied from class: Action

Returns true if this action can be applied in this specified state with the parameters specified by the provided GroundedAction Default behavior is that an action can be applied in any state, but this will need be overridden if that is not the case.

Specified by:

applicableInState in class Action

Parameters:

st - the state to perform the action on

groundedAction - the GroundedAction specifying the parameters to use

Returns:

whether this action can be performed on the given state with the given parameters

initiateInStateHelper

public void initiateInStateHelper(State s,
                         GroundedAction groundedAction)

Description copied from class: Option

This method is always called when an option is initiated and begins execution. Specifically, it is called from the Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction) For Markov options, this method probably does not need to do anything, but for non-Markov options, like Macro actions, it may need to initialize some structures for determining termination and action selection.

Specified by:

initiateInStateHelper in class Option

Parameters:

s - the state in which the option was initiated

groundedAction - the parameters in which this option will be initiated

oneStepActionSelection

public GroundedAction oneStepActionSelection(State s,
                                    GroundedAction groundedAction)

Description copied from class: Option

This method causes the option to select a single step in the given state, when the option was initiated with the provided parameters. This method will be called by the Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction) method until it is determined that the option terminates.

Specified by:

oneStepActionSelection in class Option

Parameters:

s - the state in which an action should be selected.

groundedAction - the parameters in which this option was initiated

Returns:

the action the option has selected to take in State s

getActionDistributionForState

public java.util.List<Policy.ActionProb> getActionDistributionForState(State s,
                                                              GroundedAction groundedAction)

Description copied from class: Option

Returns the option's policy distribution for a given state. This method is primarily used by the methods for computing transition dynamics. Note that if this is a non-Markov option, the returned distribution should be with respect to the state in which the option was executed and any previous actions it took that influence behavior.

Specified by:

getActionDistributionForState in class Option

Parameters:

s - the state for which this option's policy distribution should be returned

groundedAction - the parameters in which this option was initiated

Returns:

this options policy distribution for the given state.