burlap.behavior.singleagent.options

Class PolicyDefinedSubgoalOption

java.lang.Object

burlap.oomdp.singleagent.Action

burlap.behavior.singleagent.options.Option

burlap.behavior.singleagent.options.PolicyDefinedSubgoalOption

All Implemented Interfaces:

FullActionModel

public class PolicyDefinedSubgoalOption
extends Option

This is a subgoal option whose initiation states are defined by the state in which the policy is defined. If the agent enters a state outside where the policy is defined, that is a termination state with probability 1, as are any subgoal states.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.oomdp.singleagent.FullActionModel

FullActionModel.FullActionModelHelper

Field Summary

Fields inherited from class burlap.behavior.singleagent.options.Option

cachedExpectations, cachedExpectedRewards, cumulativeDiscount, discountFactor, expectationSearchCutoffProb, expectationStateHashingFactory, externalTerminalFunction, keepTrackOfReward, lastCumulativeReward, lastNumSteps, lastOptionExecutionResults, rand, rf, shouldAnnotateExecution, shouldRecordResults, stateMapping, terminateMapper

Fields inherited from class burlap.oomdp.singleagent.Action

actionObservers, domain, name

Constructor Summary

Constructors

Constructor and Description
PolicyDefinedSubgoalOption(java.lang.String name, Policy p, StateConditionTest sg) Initializes.

Method Summary

Methods

Modifier and Type	Method and Description
boolean	applicableInState(State st, GroundedAction groundedAction) Returns true if this action can be applied in this specified state with the parameters specified by the provided GroundedAction Default behavior is that an action can be applied in any state, but this will need be overridden if that is not the case.
java.util.List<Policy.ActionProb>	getActionDistributionForState(State s, GroundedAction groundedAction) Returns the option's policy distribution for a given state.
java.util.List<GroundedAction>	getAllApplicableGroundedActions(State s) Returns all possible groundings of this action that can be applied in the provided State.
GroundedAction	getAssociatedGroundedAction() Returns a GroundedAction instance that points to this Action, but does not have any parameters--if any--set.
void	initiateInStateHelper(State s, GroundedAction groundedAction) This method is always called when an option is initiated and begins execution.
boolean	isMarkov() Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.
boolean	isParameterized() Returns true if this action is parameterized; false otherwise.
GroundedAction	oneStepActionSelection(State s, GroundedAction groundedAction) This method causes the option to select a single step in the given state, when the option was initiated with the provided parameters.
double	probabilityOfTermination(State s, GroundedAction groundedAction) Returns the probability that this option (executed with the given parameters) will terminate in the given state
boolean	usesDeterministicPolicy() Returns whether this option's policy is deterministic or stochastic
boolean	usesDeterministicTermination() Returns whether this option's termination conditions are deterministic or stochastic

Methods inherited from class burlap.behavior.singleagent.options.Option

accumulateDiscountedProb, continueFromState, getDeterministicPolicy, getExpectedRewards, getLastCumulativeReward, getLastExecutionResults, getLastNumSteps, getTransitions, initiateInState, isAnnotatingExecutionResults, isPrimitive, isRecordingExecutionResults, iterateExpectationScan, keepTrackOfRewardWith, map, oneStep, oneStep, performActionHelper, performInEnvironment, setExernalTermination, setExpectationCalculationProbabilityCutoff, setExpectationHashingFactory, setStateMapping, setTerminateMapper, toggleShouldAnnotateResults, toggleShouldRecordResults

Methods inherited from class burlap.oomdp.singleagent.Action

addActionObserver, clearAllActionsObservers, deterministicTransition, equals, getAllApplicableGroundedActionsFromActionList, getDomain, getGroundedAction, getName, hashCode, performAction

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PolicyDefinedSubgoalOption

public PolicyDefinedSubgoalOption(java.lang.String name,
                          Policy p,
                          StateConditionTest sg)

Initializes.

Parameters:

name - the name of the option

p - the policy of the option

sg - the subgoals it is meant to reach

Method Detail

isParameterized

public boolean isParameterized()

Description copied from class: Action

Returns true if this action is parameterized; false otherwise.

Specified by:

isParameterized in class Action

Returns:

true if this Action is parameterized; false if it is not.

getAssociatedGroundedAction

public GroundedAction getAssociatedGroundedAction()

Description copied from class: Action

Returns a GroundedAction instance that points to this Action, but does not have any parameters--if any--set.

Specified by:

getAssociatedGroundedAction in class Action

Returns:

a GroundedAction instance.

getAllApplicableGroundedActions

public java.util.List<GroundedAction> getAllApplicableGroundedActions(State s)

Description copied from class: Action

Returns all possible groundings of this action that can be applied in the provided State. To check if a grounded action is applicable in the state, the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction) method is checked. The default behavior of this method is to treat the parameters as possible object bindings, finding all bindings that satisfy the object class typing specified and then checking them against the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction) method. However, this class can also be overridden to provide custom grounding behavior or non-object based parametrization.

Specified by:

getAllApplicableGroundedActions in class Action

Parameters:

s - the State in which all applicable grounded actions of this Action object should be returned.

Returns:

a list of all applicable GroundedActions of this Action object in in the given State

isMarkov

public boolean isMarkov()

Description copied from class: Option

Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.

Specified by:

isMarkov in class Option

Returns:

True if this option is Markov ; false otherwise.

usesDeterministicTermination

public boolean usesDeterministicTermination()

Description copied from class: Option

Returns whether this option's termination conditions are deterministic or stochastic

Specified by:

usesDeterministicTermination in class Option

Returns:

true if this option's termination conditions are deterministic; false if stochastic.

usesDeterministicPolicy

public boolean usesDeterministicPolicy()

Description copied from class: Option

Returns whether this option's policy is deterministic or stochastic

Specified by:

usesDeterministicPolicy in class Option

Returns:

true if this option's policy is deterministic; false if stochastic

probabilityOfTermination

public double probabilityOfTermination(State s,
                              GroundedAction groundedAction)

Description copied from class: Option

Returns the probability that this option (executed with the given parameters) will terminate in the given state

Specified by:

probabilityOfTermination in class Option

Parameters:

s - the state to test for termination

groundedAction - the parameters in which this option was initiated

Returns:

the probability that this option (executed with the given parameters) will terminate in the given state

applicableInState

public boolean applicableInState(State st,
                        GroundedAction groundedAction)

Description copied from class: Action

Returns true if this action can be applied in this specified state with the parameters specified by the provided GroundedAction Default behavior is that an action can be applied in any state, but this will need be overridden if that is not the case.

Specified by:

applicableInState in class Action

Parameters:

st - the state to perform the action on

groundedAction - the GroundedAction specifying the parameters to use

Returns:

whether this action can be performed on the given state with the given parameters

initiateInStateHelper

public void initiateInStateHelper(State s,
                         GroundedAction groundedAction)

Description copied from class: Option

This method is always called when an option is initiated and begins execution. Specifically, it is called from the Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction) For Markov options, this method probably does not need to do anything, but for non-Markov options, like Macro actions, it may need to initialize some structures for determining termination and action selection.

Specified by:

initiateInStateHelper in class Option

Parameters:

s - the state in which the option was initiated

groundedAction - the parameters in which this option will be initiated

oneStepActionSelection

public GroundedAction oneStepActionSelection(State s,
                                    GroundedAction groundedAction)

Description copied from class: Option

This method causes the option to select a single step in the given state, when the option was initiated with the provided parameters. This method will be called by the Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction) method until it is determined that the option terminates.

Specified by:

oneStepActionSelection in class Option

Parameters:

s - the state in which an action should be selected.

groundedAction - the parameters in which this option was initiated

Returns:

the action the option has selected to take in State s

getActionDistributionForState

public java.util.List<Policy.ActionProb> getActionDistributionForState(State s,
                                                              GroundedAction groundedAction)

Description copied from class: Option

Returns the option's policy distribution for a given state. This method is primarily used by the methods for computing transition dynamics. Note that if this is a non-Markov option, the returned distribution should be with respect to the state in which the option was executed and any previous actions it took that influence behavior.

Specified by:

getActionDistributionForState in class Option

Parameters:

s - the state for which this option's policy distribution should be returned

groundedAction - the parameters in which this option was initiated

Returns:

this options policy distribution for the given state.