burlap.behavior.singleagent.options

Class MacroAction

java.lang.Object

burlap.oomdp.singleagent.Action

burlap.behavior.singleagent.options.Option

burlap.behavior.singleagent.options.MacroAction

All Implemented Interfaces:

FullActionModel

public class MacroAction
extends Option

A macro action is an action that always executes a sequence of actions.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.oomdp.singleagent.FullActionModel

FullActionModel.FullActionModelHelper

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<GroundedAction>	actionSequence The list of actions that will be executed in order when this macro-action is called.
protected int	curIndex the current execution index of the macro-action sequence.

Fields inherited from class burlap.behavior.singleagent.options.Option

cachedExpectations, cachedExpectedRewards, cumulativeDiscount, discountFactor, expectationSearchCutoffProb, expectationStateHashingFactory, externalTerminalFunction, keepTrackOfReward, lastCumulativeReward, lastNumSteps, lastOptionExecutionResults, rand, rf, shouldAnnotateExecution, shouldRecordResults, stateMapping, terminateMapper

Fields inherited from class burlap.oomdp.singleagent.Action

actionObservers, domain, name

Constructor Summary

Constructors

Constructor and Description
MacroAction(java.lang.String name, java.util.List<GroundedAction> actionSequence) Instantiates a macro action with a given name and action sequence.

Method Summary

Methods

Modifier and Type	Method and Description
boolean	applicableInState(State s, GroundedAction groundedAction) Returns true if this action can be applied in this specified state with the parameters specified by the provided GroundedAction Default behavior is that an action can be applied in any state, but this will need be overridden if that is not the case.
java.util.List<Policy.ActionProb>	getActionDistributionForState(State s, GroundedAction groundedAction) Returns the option's policy distribution for a given state.
java.util.List<GroundedAction>	getAllApplicableGroundedActions(State s) Returns all possible groundings of this action that can be applied in the provided State.
GroundedAction	getAssociatedGroundedAction() Returns a GroundedAction instance that points to this Action, but does not have any parameters--if any--set.
void	initiateInStateHelper(State s, GroundedAction groundedAction) This method is always called when an option is initiated and begins execution.
boolean	isMarkov() Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.
boolean	isParameterized() Returns true if this action is parameterized; false otherwise.
GroundedAction	oneStepActionSelection(State s, GroundedAction groundedAction) This method causes the option to select a single step in the given state, when the option was initiated with the provided parameters.
double	probabilityOfTermination(State s, GroundedAction groundedAction) Returns the probability that this option (executed with the given parameters) will terminate in the given state
boolean	usesDeterministicPolicy() Returns whether this option's policy is deterministic or stochastic
boolean	usesDeterministicTermination() Returns whether this option's termination conditions are deterministic or stochastic

Methods inherited from class burlap.behavior.singleagent.options.Option

accumulateDiscountedProb, continueFromState, getDeterministicPolicy, getExpectedRewards, getLastCumulativeReward, getLastExecutionResults, getLastNumSteps, getTransitions, initiateInState, isAnnotatingExecutionResults, isPrimitive, isRecordingExecutionResults, iterateExpectationScan, keepTrackOfRewardWith, map, oneStep, oneStep, performActionHelper, performInEnvironment, setExernalTermination, setExpectationCalculationProbabilityCutoff, setExpectationHashingFactory, setStateMapping, setTerminateMapper, toggleShouldAnnotateResults, toggleShouldRecordResults

Methods inherited from class burlap.oomdp.singleagent.Action

addActionObserver, clearAllActionsObservers, deterministicTransition, equals, getAllApplicableGroundedActionsFromActionList, getDomain, getGroundedAction, getName, hashCode, performAction

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

actionSequence

protected java.util.List<GroundedAction> actionSequence

The list of actions that will be executed in order when this macro-action is called.

curIndex

protected int curIndex

the current execution index of the macro-action sequence. Every time this action is executed, it will start at index 0.

Constructor Detail

MacroAction

public MacroAction(java.lang.String name,
           java.util.List<GroundedAction> actionSequence)

Instantiates a macro action with a given name and action sequence. The name of the macro action should be unique from any other action name.

Parameters:

name - the name of the macro action.

actionSequence - the sequence of actions the macro action will execute.

Method Detail

applicableInState

public boolean applicableInState(State s,
                        GroundedAction groundedAction)

Description copied from class: Action

Returns true if this action can be applied in this specified state with the parameters specified by the provided GroundedAction Default behavior is that an action can be applied in any state, but this will need be overridden if that is not the case.

Specified by:

applicableInState in class Action

Parameters:

s - the state to perform the action on

groundedAction - the GroundedAction specifying the parameters to use

Returns:

whether this action can be performed on the given state with the given parameters

isParameterized

public boolean isParameterized()

Description copied from class: Action

Returns true if this action is parameterized; false otherwise.

Specified by:

isParameterized in class Action

Returns:

true if this Action is parameterized; false if it is not.

getAssociatedGroundedAction

public GroundedAction getAssociatedGroundedAction()

Description copied from class: Action

Returns a GroundedAction instance that points to this Action, but does not have any parameters--if any--set.

Specified by:

getAssociatedGroundedAction in class Action

Returns:

a GroundedAction instance.

getAllApplicableGroundedActions

public java.util.List<GroundedAction> getAllApplicableGroundedActions(State s)

Description copied from class: Action

Returns all possible groundings of this action that can be applied in the provided State. To check if a grounded action is applicable in the state, the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction) method is checked. The default behavior of this method is to treat the parameters as possible object bindings, finding all bindings that satisfy the object class typing specified and then checking them against the Action.applicableInState(State, burlap.oomdp.singleagent.GroundedAction) method. However, this class can also be overridden to provide custom grounding behavior or non-object based parametrization.

Specified by:

getAllApplicableGroundedActions in class Action

Parameters:

s - the State in which all applicable grounded actions of this Action object should be returned.

Returns:

a list of all applicable GroundedActions of this Action object in in the given State

isMarkov

public boolean isMarkov()

Description copied from class: Option

Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.

Specified by:

isMarkov in class Option

Returns:

True if this option is Markov ; false otherwise.

usesDeterministicTermination

public boolean usesDeterministicTermination()

Description copied from class: Option

Returns whether this option's termination conditions are deterministic or stochastic

Specified by:

usesDeterministicTermination in class Option

Returns:

true if this option's termination conditions are deterministic; false if stochastic.

usesDeterministicPolicy

public boolean usesDeterministicPolicy()

Description copied from class: Option

Returns whether this option's policy is deterministic or stochastic

Specified by:

usesDeterministicPolicy in class Option

Returns:

true if this option's policy is deterministic; false if stochastic

probabilityOfTermination

public double probabilityOfTermination(State s,
                              GroundedAction groundedAction)

Description copied from class: Option

Returns the probability that this option (executed with the given parameters) will terminate in the given state

Specified by:

probabilityOfTermination in class Option

Parameters:

s - the state to test for termination

groundedAction - the parameters in which this option was initiated

Returns:

the probability that this option (executed with the given parameters) will terminate in the given state

initiateInStateHelper

public void initiateInStateHelper(State s,
                         GroundedAction groundedAction)

Description copied from class: Option

This method is always called when an option is initiated and begins execution. Specifically, it is called from the Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction) For Markov options, this method probably does not need to do anything, but for non-Markov options, like Macro actions, it may need to initialize some structures for determining termination and action selection.

Specified by:

initiateInStateHelper in class Option

Parameters:

s - the state in which the option was initiated

groundedAction - the parameters in which this option will be initiated

oneStepActionSelection

public GroundedAction oneStepActionSelection(State s,
                                    GroundedAction groundedAction)

Description copied from class: Option

This method causes the option to select a single step in the given state, when the option was initiated with the provided parameters. This method will be called by the Option.performActionHelper(burlap.oomdp.core.states.State, burlap.oomdp.singleagent.GroundedAction) method until it is determined that the option terminates.

Specified by:

oneStepActionSelection in class Option

Parameters:

s - the state in which an action should be selected.

groundedAction - the parameters in which this option was initiated

Returns:

the action the option has selected to take in State s

getActionDistributionForState

public java.util.List<Policy.ActionProb> getActionDistributionForState(State s,
                                                              GroundedAction groundedAction)

Description copied from class: Option

Returns the option's policy distribution for a given state. This method is primarily used by the methods for computing transition dynamics. Note that if this is a non-Markov option, the returned distribution should be with respect to the state in which the option was executed and any previous actions it took that influence behavior.

Specified by:

getActionDistributionForState in class Option

Parameters:

s - the state for which this option's policy distribution should be returned

groundedAction - the parameters in which this option was initiated

Returns:

this options policy distribution for the given state.