burlap.behavior.singleagent.planning

Class OOMDPPlanner

java.lang.Object

burlap.behavior.singleagent.planning.OOMDPPlanner

Direct Known Subclasses:

ActorCritic, ARTDP, DeterministicPlanner, DifferentiableSparseSampling, FittedVI, GradientDescentSarsaLam, LSPI, PotentialShapedRMax, QLearning, SparseSampling, UCT, ValueFunctionPlanner

public abstract class OOMDPPlanner
extends java.lang.Object

The super class to use for all planning algorithms. It provides the common data members that most all planning algorithms will need to use for planning and provides methods for manipulating them that are common. This class also defines the interface that all planners should use.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<Action>	actions The list of actions this planner can use.
protected boolean	containsParameterizedActions Indicates whether the action set for this planner includes parametrized actions
protected int	debugCode The debug code use for calls to DPrint
protected Domain	domain The domain in which planning will be performed
protected double	gamma The discount factor
protected StateHashFactory	hashingFactory The hashing factory to use for hashing states
protected java.util.Map<StateHashTuple,StateHashTuple>	mapToStateIndex A mapping to internal states that are stored.
protected RewardFunction	rf The reward function used for planning
protected TerminalFunction	tf The terminal function for identifying terminal states

Constructor Summary

Constructors

Constructor and Description
OOMDPPlanner()

Method Summary

Methods

Modifier and Type	Method and Description
void	addNonDomainReferencedAction(Action a) Adds an additional action the planner that is not included in the domain definition.
java.util.List<Action>	getActions() Returns a copy of all actions this planner uses for reasoning; including added actions that are not part of the domain specification (e.g., Options).
protected java.util.List<GroundedAction>	getAllGroundedActions(State s) Returns all grounded actions in the provided state for all the actions that this planner can use.
int	getDebugCode() Returns the debug code used by this planner for calls to DPrint
Domain	getDomain()
double	getGamma() Returns gamma, the discount factor used by this planner
StateHashFactory	getHashingFactory() Returns the StateHashFactory this planner uses.
RewardFunction	getRf()
RewardFunction	getRF() Returns the RewardFunction this planner uses.
TerminalFunction	getTf()
TerminalFunction	getTF() Returns the TerminalFunction this planner uses.
abstract void	planFromState(State initialState) This method will cause the planner to begin planning from the specified initial state
void	plannerInit(Domain domain, RewardFunction rf, TerminalFunction tf, double gamma, StateHashFactory hashingFactory) Initializes the planner with the common planning elements
abstract void	resetPlannerResults() Use this method to reset all planner results so that planning can be started fresh with a call to planFromState(State) as if no planning had ever been performed before.
void	setActions(java.util.List<Action> actions) Sets the action set the planner should use.
void	setDebugCode(int code) Sets the debug code to be used by calls to DPrint
void	setDomain(Domain domain) Sets the domain of this planner.
void	setGamma(double gamma) Sets gamma, the discount factor used by this planner
void	setRf(RewardFunction rf) Sets the reward function used by this planner
void	setTf(TerminalFunction tf) Sets the terminal state function used by this planner
StateHashTuple	stateHash(State s) A shorthand method for hashing a state.
void	toggleDebugPrinting(boolean toggle) Toggles whether the planner's calls to DPrint should be printed.
protected GroundedAction	translateAction(GroundedAction a, java.util.Map<java.lang.String,java.lang.String> matching) Takes a source parameterized GroundedAction and a matching between object instances of two different states and returns a GroudnedAction with parameters using the matched parameters.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

domain

protected Domain domain

The domain in which planning will be performed

hashingFactory

protected StateHashFactory hashingFactory

The hashing factory to use for hashing states

rf

protected RewardFunction rf

The reward function used for planning

tf

protected TerminalFunction tf

The terminal function for identifying terminal states

gamma

protected double gamma

The discount factor

actions

protected java.util.List<Action> actions

The list of actions this planner can use. May include non-domain specified actions like options.

mapToStateIndex

protected java.util.Map<StateHashTuple,StateHashTuple> mapToStateIndex

A mapping to internal states that are stored. Useful since two identical states may have different object instance name identifiers that can affect the parameters in GroundedActions.

containsParameterizedActions

protected boolean containsParameterizedActions

Indicates whether the action set for this planner includes parametrized actions

debugCode

protected int debugCode

The debug code use for calls to DPrint

Constructor Detail

OOMDPPlanner

public OOMDPPlanner()

Method Detail

planFromState

public abstract void planFromState(State initialState)

This method will cause the planner to begin planning from the specified initial state

Parameters:

initialState - the initial state of the planning problem

resetPlannerResults

public abstract void resetPlannerResults()

Use this method to reset all planner results so that planning can be started fresh with a call to planFromState(State) as if no planning had ever been performed before. Specifically, data produced from calls to the planFromState(State) will be cleared, but all other planner settings should remain the same. This is useful if the reward function or transition dynamics have changed, thereby requiring new results to be computed. If there were other objects this planner was provided that may have changed and need to be reset, you will need to reset them yourself. For instance, if you told a planner to follow a policy that had a temperature parameter decrease with time, you will need to reset the policy's temperature yourself.

plannerInit

public void plannerInit(Domain domain,
               RewardFunction rf,
               TerminalFunction tf,
               double gamma,
               StateHashFactory hashingFactory)

Initializes the planner with the common planning elements

Parameters:

domain - the domain in which planning will be performed

rf - the reward function

tf - the terminal state function

gamma - the discount factor

hashingFactory - the hashing factory used to store states (may be set to null if the planner is not tabular)

addNonDomainReferencedAction

public void addNonDomainReferencedAction(Action a)

Adds an additional action the planner that is not included in the domain definition. For instance, an Option should be added using this method.

Parameters:

a - the action to add to the planner

setActions

public void setActions(java.util.List<Action> actions)

Sets the action set the planner should use.

Parameters:

actions - the actions the planner should use.

getActions

public java.util.List<Action> getActions()

Returns a copy of all actions this planner uses for reasoning; including added actions that are not part of the domain specification (e.g., Options). Modifying the returned list will not modify the action list this planner uses.

Returns:

a List of all actions this planner uses.

getTF

public TerminalFunction getTF()

Returns the TerminalFunction this planner uses.

Returns:

the TerminalFunction this planner uses.

getRF

public RewardFunction getRF()

Returns the RewardFunction this planner uses.

Returns:

the RewardFunction this planner uses.

getHashingFactory

public StateHashFactory getHashingFactory()

Returns the StateHashFactory this planner uses.

Returns:

the StateHashFactory this planner uses.

setRf

public void setRf(RewardFunction rf)

Sets the reward function used by this planner

Parameters:

rf - the reward function to be used by this planner

setTf

public void setTf(TerminalFunction tf)

Sets the terminal state function used by this planner

Parameters:

tf - the terminal function to be used by this planner

getGamma

public double getGamma()

Returns gamma, the discount factor used by this planner

Returns:

gamma, the discount factor used by this planner

setGamma

public void setGamma(double gamma)

Sets gamma, the discount factor used by this planner

Parameters:

gamma - the discount factor used by this planner

setDebugCode

public void setDebugCode(int code)

Sets the debug code to be used by calls to DPrint

Parameters:

code - the code to be used by DPrint

getDebugCode

public int getDebugCode()

Returns the debug code used by this planner for calls to DPrint

Returns:

the debug code used by this planner for calls to DPrint

toggleDebugPrinting

public void toggleDebugPrinting(boolean toggle)

Toggles whether the planner's calls to DPrint should be printed.

Parameters:

toggle - whether to print the calls to DPrint

setDomain

public void setDomain(Domain domain)

Sets the domain of this planner. NOTE: this will also reset the actions this planner uses to the actions of the provided domain. If you have previously added non-domain referenced actiosn through the addNonDomainReferencedAction(burlap.oomdp.singleagent.Action) method, you will have to do so again.

Parameters:

domain - the domain this planner should use.

getDomain

public Domain getDomain()

getRf

public RewardFunction getRf()

getTf

public TerminalFunction getTf()

translateAction

protected GroundedAction translateAction(GroundedAction a,
                             java.util.Map<java.lang.String,java.lang.String> matching)

Takes a source parameterized GroundedAction and a matching between object instances of two different states and returns a GroudnedAction with parameters using the matched parameters. This method is useful a stored state and action pair in the planner data structure has different object name identifiers than a query state that is otherwise identical. The matching is from the state in which the source action is applied to some target state that is not provided to this method.

Parameters:

a - the source action that needs to be translated

matching - a map from object instance names to other object instance names.

Returns:

and new GroundedAction with object parameterizations that follow from the matching

stateHash

public StateHashTuple stateHash(State s)

A shorthand method for hashing a state.

Parameters:

s - the state to hash

Returns:

a StateHashTuple produce from this planners StateHashFactory.

getAllGroundedActions

protected java.util.List<GroundedAction> getAllGroundedActions(State s)

Returns all grounded actions in the provided state for all the actions that this planner can use.

Parameters:

s - the source state for which to get all GroundedActions.

Returns:

all GroundedActions.