burlap.behavior.singleagent

Class MDPSolver

java.lang.Object

burlap.behavior.singleagent.MDPSolver

All Implemented Interfaces:

MDPSolverInterface

Direct Known Subclasses:

ActorCritic, ARTDP, BeliefSparseSampling, DeterministicPlanner, DifferentiableSparseSampling, DynamicProgramming, FittedVI, GradientDescentSarsaLam, LSPI, PotentialShapedRMax, QLearning, QMDP, SparseSampling, TDLambda, UCT

public abstract class MDPSolver
extends java.lang.Object
implements MDPSolverInterface

The abstract super class to use for various MDP solving algorithms, including both planning and learning algorithms. It implements the MDPSolverInterface and provides the common data members and method implementations that most all algorithms will need to use and provides methods for manipulating them that are common.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<ActionType>	actionTypes The list of actions this solver can use.
protected int	debugCode The debug code use for calls to DPrint
protected SADomain	domain The domain to solve
protected double	gamma The MDP discount factor
protected HashableStateFactory	hashingFactory The hashing factory to use for hashing states in tabular solvers
protected SampleModel	model
protected boolean	usingOptionModel

Constructor Summary

Constructors

Constructor and Description
MDPSolver()

Method Summary

Modifier and Type	Method and Description
void	addActionType(ActionType a) Adds an additional action the solver that is not included in the domain definition.
protected java.util.List<Action>	applicableActions(State s) Returns all applicable actions in the provided state for all the actions that this MDP Solver can use.
java.util.List<ActionType>	getActionTypes() Returns a copy of all actions this solver uses for reasoning; including added actions that are not part of the domain specification (e.g., Options).
int	getDebugCode() Returns the debug code used by this solver for calls to DPrint
Domain	getDomain() Returns the Domain this solver solves.
double	getGamma() Returns gamma, the discount factor used by this solver
HashableStateFactory	getHashingFactory() Returns the HashableStateFactory this solver uses.
SampleModel	getModel() Returns the model being used by this solver
abstract void	resetSolver() This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.
void	setActionTypes(java.util.List<ActionType> actionTypes) Sets the action set the solver should use.
void	setDebugCode(int code) Sets the debug code to be used by calls to DPrint
void	setDomain(SADomain domain) Sets the domain of this solver.
void	setGamma(double gamma) Sets gamma, the discount factor used by this solver
void	setHashingFactory(HashableStateFactory hashingFactory) Sets the HashableStateFactory used to hash states for tabular solvers.
void	setModel(SampleModel model) Sets the model to use for this solver
void	solverInit(SADomain domain, double gamma, HashableStateFactory hashingFactory) Initializes the solver with the common elements.
HashableState	stateHash(State s) A shorthand method for hashing a state.
void	toggleDebugPrinting(boolean toggle) Toggles whether the solver's calls to DPrint should be printed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

domain

protected SADomain domain

The domain to solve

model

protected SampleModel model

hashingFactory

protected HashableStateFactory hashingFactory

The hashing factory to use for hashing states in tabular solvers

gamma

protected double gamma

The MDP discount factor

actionTypes

protected java.util.List<ActionType> actionTypes

The list of actions this solver can use. May include non-domain specified actions like Options.

debugCode

protected int debugCode

The debug code use for calls to DPrint

usingOptionModel

protected boolean usingOptionModel

Constructor Detail

MDPSolver

public MDPSolver()

Method Detail

resetSolver

public abstract void resetSolver()

Description copied from interface: MDPSolverInterface

This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.

Specified by:

resetSolver in interface MDPSolverInterface

solverInit

public void solverInit(SADomain domain,
                       double gamma,
                       HashableStateFactory hashingFactory)

Description copied from interface: MDPSolverInterface

Initializes the solver with the common elements.

Specified by:

solverInit in interface MDPSolverInterface

Parameters:

domain - the domain to be solved.

gamma - the MDP discount factor

hashingFactory - the hashing factory used to store states (may be set to null if the solver is not tabular)

addActionType

public void addActionType(ActionType a)

Description copied from interface: MDPSolverInterface

Adds an additional action the solver that is not included in the domain definition. For instance, an Option should be added using this method.

Specified by:

addActionType in interface MDPSolverInterface

Parameters:

a - the action to add to the solver

setModel

public void setModel(SampleModel model)

Description copied from interface: MDPSolverInterface

Sets the model to use for this solver

Specified by:

setModel in interface MDPSolverInterface

Parameters:

model - the model to use

getModel

public SampleModel getModel()

Description copied from interface: MDPSolverInterface

Returns the model being used by this solver

Specified by:

getModel in interface MDPSolverInterface

Returns:

a SampleModel

setActionTypes

public void setActionTypes(java.util.List<ActionType> actionTypes)

Description copied from interface: MDPSolverInterface

Sets the action set the solver should use.

Specified by:

setActionTypes in interface MDPSolverInterface

Parameters:

actionTypes - the actions the solver should use.

getActionTypes

public java.util.List<ActionType> getActionTypes()

Description copied from interface: MDPSolverInterface

Returns a copy of all actions this solver uses for reasoning; including added actions that are not part of the domain specification (e.g., Options). Modifying the returned list will not modify the action list this solver uses.

Specified by:

getActionTypes in interface MDPSolverInterface

Returns:

a List of all actions this solver uses.

setHashingFactory

public void setHashingFactory(HashableStateFactory hashingFactory)

Description copied from interface: MDPSolverInterface

Sets the HashableStateFactory used to hash states for tabular solvers.

Specified by:

setHashingFactory in interface MDPSolverInterface

Parameters:

hashingFactory - the HashableStateFactory used to hash states for tabular solvers.

getHashingFactory

public HashableStateFactory getHashingFactory()

Description copied from interface: MDPSolverInterface

Returns the HashableStateFactory this solver uses.

Specified by:

getHashingFactory in interface MDPSolverInterface

Returns:

the HashableStateFactory this solver uses.

getGamma

public double getGamma()

Description copied from interface: MDPSolverInterface

Returns gamma, the discount factor used by this solver

Specified by:

getGamma in interface MDPSolverInterface

Returns:

gamma, the discount factor used by this solver

setGamma

public void setGamma(double gamma)

Description copied from interface: MDPSolverInterface

Sets gamma, the discount factor used by this solver

Specified by:

setGamma in interface MDPSolverInterface

Parameters:

gamma - the discount factor used by this solver

setDebugCode

public void setDebugCode(int code)

Description copied from interface: MDPSolverInterface

Sets the debug code to be used by calls to DPrint

Specified by:

setDebugCode in interface MDPSolverInterface

Parameters:

code - the code to be used by DPrint

getDebugCode

public int getDebugCode()

Description copied from interface: MDPSolverInterface

Returns the debug code used by this solver for calls to DPrint

Specified by:

getDebugCode in interface MDPSolverInterface

Returns:

the debug code used by this solver for calls to DPrint

toggleDebugPrinting

public void toggleDebugPrinting(boolean toggle)

Description copied from interface: MDPSolverInterface

Toggles whether the solver's calls to DPrint should be printed.

Specified by:

toggleDebugPrinting in interface MDPSolverInterface

Parameters:

toggle - whether to print the calls to DPrint

setDomain

public void setDomain(SADomain domain)

Description copied from interface: MDPSolverInterface

Sets the domain of this solver. NOTE: this will also reset the actions this solver uses to the actions of the provided domain. If you have previously added non-domain referenced actions through the MDPSolverInterface.addActionType(ActionType) method, you will have to do so again.

Specified by:

setDomain in interface MDPSolverInterface

Parameters:

domain - the domain this solver should use.

getDomain

public Domain getDomain()

Description copied from interface: MDPSolverInterface

Returns the Domain this solver solves.

Specified by:

getDomain in interface MDPSolverInterface

Returns:

the Domain this solver solves.

stateHash

public HashableState stateHash(State s)

A shorthand method for hashing a state.

Parameters:

s - the state to hash

Returns:

a StateHashTuple produce from this planners StateHashFactory.

applicableActions

protected java.util.List<Action> applicableActions(State s)

Returns all applicable actions in the provided state for all the actions that this MDP Solver can use. If this solver had ActionTypes not included in the domain added (e.g., types for Options), this method will include those as well.

Parameters:

s - the source state for which to get all GroundedActions.

Returns:

all GroundedActions.