burlap.behavior.singleagent.options

Class MacroAction

java.lang.Object

burlap.oomdp.singleagent.Action

burlap.behavior.singleagent.options.Option

burlap.behavior.singleagent.options.MacroAction

public class MacroAction
extends Option

A macro action is an action that always executes a sequence of actions.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<GroundedAction>	actionSequence The list of actions that will be executed in order when this macro-action is called.
protected int	curIndex the current execution index of the macro-action sequence.

Fields inherited from class burlap.behavior.singleagent.options.Option

cachedExpectations, cachedExpectedRewards, cumulativeDiscount, discountFactor, expectationSearchCutoffProb, expectationStateHashingFactory, externalTerminalFunction, keepTrackOfReward, lastCumulativeReward, lastNumSteps, lastOptionExecutionResults, rand, rf, shouldAnnotateExecution, shouldRecordResults, stateMapping, terminateMapper

Fields inherited from class burlap.oomdp.singleagent.Action

actionObservers, domain, name, parameterClasses, parameterOrderGroup

Constructor Summary

Constructors

Constructor and Description
MacroAction(java.lang.String name, java.util.List<GroundedAction> actionSequence) Instantiates a macro action with a given name and action sequence.

Method Summary

Methods

Modifier and Type	Method and Description
java.util.List<Policy.ActionProb>	getActionDistributionForState(State s, java.lang.String[] params) Returns the option's policy distribution for a given state.
void	initiateInStateHelper(State s, java.lang.String[] params) This method is always called when an option is initated and begins execution.
boolean	isMarkov() Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.
GroundedAction	oneStepActionSelection(State s, java.lang.String[] params) This method causes the option to take a single step in the given state, when the option was initiated with the provided parameters.
double	probabilityOfTermination(State s, java.lang.String[] params) Returns the probability that this option (executed with the given parameters) will terminate in the given state
boolean	usesDeterministicPolicy() Returns whether this option's policy is deterministic or stochastic
boolean	usesDeterministicTermination() Returns whether this option's termination conditions are deterministic or stochastic

Methods inherited from class burlap.behavior.singleagent.options.Option

accumulateDiscountedProb, continueFromState, getDeterministicPolicy, getExpectedRewards, getLastCumulativeReward, getLastExecutionResults, getLastNumSteps, getTransitions, init, initiateInState, isAnnotatingExecutionResults, isPrimitive, isRecordingExecutionResults, iterateExpectationScan, keepTrackOfRewardWith, map, oneStep, performActionHelper, setExernalTermination, setExpectationCalculationProbabilityCutoff, setExpectationHashingFactory, setStateMapping, setTerminateMapper, toggleShouldAnnotateResults, toggleShouldRecordResults

Methods inherited from class burlap.oomdp.singleagent.Action

addActionObserver, applicableInState, applicableInState, clearAllActionsObservers, deterministicTransition, equals, getAllApplicableGroundedActions, getAllApplicableGroundedActionsFromActionList, getDomain, getName, getParameterClasses, getParameterOrderGroups, getTransitions, hashCode, parametersAreObjects, performAction, performAction

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

actionSequence

protected java.util.List<GroundedAction> actionSequence

The list of actions that will be executed in order when this macro-action is called.

curIndex

protected int curIndex

the current execution index of the macro-action sequence. Every time this action is executed, it will start at index 0.

Constructor Detail

MacroAction

public MacroAction(java.lang.String name,
           java.util.List<GroundedAction> actionSequence)

Instantiates a macro action with a given name and action sequence. The name of the macro action should be unique from any other action name.

Parameters:

name - the name of the macro action.

actionSequence - the sequence of actions the macro action will execute.

Method Detail

isMarkov

public boolean isMarkov()

Description copied from class: Option

Returns whether this option is Markov or not; that is, whether action selection and termination only depends on the current state.

Specified by:

isMarkov in class Option

Returns:

True if this option is Markov ; false otherwise.

usesDeterministicTermination

public boolean usesDeterministicTermination()

Description copied from class: Option

Returns whether this option's termination conditions are deterministic or stochastic

Specified by:

usesDeterministicTermination in class Option

Returns:

true if this option's termination conditions are deterministic; false if stochastic.

usesDeterministicPolicy

public boolean usesDeterministicPolicy()

Description copied from class: Option

Returns whether this option's policy is deterministic or stochastic

Specified by:

usesDeterministicPolicy in class Option

Returns:

true if this option's policy is deterministic; false if stochastic

probabilityOfTermination

public double probabilityOfTermination(State s,
                              java.lang.String[] params)

Description copied from class: Option

Returns the probability that this option (executed with the given parameters) will terminate in the given state

Specified by:

probabilityOfTermination in class Option

Parameters:

s - the state to test for termination

params - any parameters that were applied to this option when it was initiated

Returns:

the probability that this option (executed with the given parameters) will terminate in the given state

initiateInStateHelper

public void initiateInStateHelper(State s,
                         java.lang.String[] params)

Description copied from class: Option

This method is always called when an option is initated and begins execution. Specifically, it is called from the Option.performActionHelper(State, String []) For Markov options, this method probably does not need to do anything, but for non-Markov options, like Macro actions, it may need to initialize some structures for determining termination and action selection.

Specified by:

initiateInStateHelper in class Option

Parameters:

s - the state in which the option was initiated

params - the parameters that were passed to the option for execution

oneStepActionSelection

public GroundedAction oneStepActionSelection(State s,
                                    java.lang.String[] params)

Description copied from class: Option

This method causes the option to take a single step in the given state, when the option was initiated with the provided parameters. This method will be called by the Option.performActionHelper(State, String []) method until it is determined that the option terminates.

Specified by:

oneStepActionSelection in class Option

Parameters:

s - the state in which an action should be selected.

params - the parameters that were passed to the option when it was initiated

Returns:

the action the option has selected to take in State s

getActionDistributionForState

public java.util.List<Policy.ActionProb> getActionDistributionForState(State s,
                                                              java.lang.String[] params)

Description copied from class: Option

Returns the option's policy distribution for a given state. This method is primarily used by the methods for computing transition dynamics. Note that if this is a non-Markov option, the returned distribution should be with respect to the state in which the option was executed and any previous actions it took that influence behavior.

Specified by:

getActionDistributionForState in class Option

Parameters:

s - the state for which this option's policy distribution should be returned

params - the parameters that were passed to the option when it was initiated

Returns:

this options policy distribution for the given state.