burlap.behavior.singleagent

Interface MDPSolverInterface

All Known Subinterfaces:

ModelLearningPlanner, Planner

All Known Implementing Classes:

ActorCritic, ARTDP, AStar, BeliefSparseSampling, BestFirst, BFS, BFSRTDP, BoundedRTDP, DeterministicPlanner, DFS, DifferentiableDP, DifferentiableSparseSampling, DifferentiableVI, DynamicProgramming, DynamicProgramming.StaticVFPlanner, DynamicWeightedAStar, FittedVI, GradientDescentSarsaLam, IDAStar, LimitedMemoryDFS, LSPI, MDPSolver, PolicyEvaluation, PolicyIteration, PotentialShapedRMax, PrioritizedSweeping, QLearning, QLTutorial, QMDP, RTDP, SarsaLam, SparseSampling, StaticWeightedAStar, UCT, ValueIteration, VIModelLearningPlanner, VITutorial, WeightedGreedy

public interface MDPSolverInterface

The top-level interface for algorithms that solve MDPs. In general, classes should probably subclass the MDPSolver abstract class instead of using this interface directly, since the MDPSolver will provide relevant data members, implement these methods, and provide additional helper methods.

Author:

James MacGlashan.

Method Summary

Methods

Modifier and Type	Method and Description
void	addNonDomainReferencedAction(Action a) Adds an additional action the solver that is not included in the domain definition.
java.util.List<Action>	getActions() Returns a copy of all actions this solver uses for reasoning; including added actions that are not part of the domain specification (e.g., Options).
int	getDebugCode() Returns the debug code used by this solver for calls to DPrint
Domain	getDomain() Returns the Domain this solver solves.
double	getGamma() Returns gamma, the discount factor used by this solver
HashableStateFactory	getHashingFactory() Returns the HashableStateFactory this solver uses.
RewardFunction	getRf() Returns the RewardFunction this solver uses.
RewardFunction	getRF() Returns the RewardFunction this solver uses.
TerminalFunction	getTf() Returns the TerminalFunction this solver uses.
TerminalFunction	getTF() Returns the TerminalFunction this solver uses.
void	resetSolver() This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.
void	setActions(java.util.List<Action> actions) Sets the action set the solver should use.
void	setDebugCode(int code) Sets the debug code to be used by calls to DPrint
void	setDomain(Domain domain) Sets the domain of this solver.
void	setGamma(double gamma) Sets gamma, the discount factor used by this solver
void	setHashingFactory(HashableStateFactory hashingFactory) Sets the HashableStateFactory used to hash states for tabular solvers.
void	setRf(RewardFunction rf) Sets the reward function used by this solver
void	setTf(TerminalFunction tf) Sets the terminal state function used by this solver
void	solverInit(Domain domain, RewardFunction rf, TerminalFunction tf, double gamma, HashableStateFactory hashingFactory) Initializes the solver with the common elements.
void	toggleDebugPrinting(boolean toggle) Toggles whether the solver's calls to DPrint should be printed.

Method Detail

solverInit

void solverInit(Domain domain,
              RewardFunction rf,
              TerminalFunction tf,
              double gamma,
              HashableStateFactory hashingFactory)

Initializes the solver with the common elements.

Parameters:

domain - the domain to be solved.

rf - the reward function

tf - the terminal state function

gamma - the MDP discount factor

hashingFactory - the hashing factory used to store states (may be set to null if the solver is not tabular)

resetSolver

void resetSolver()

This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.

setDomain

void setDomain(Domain domain)

Sets the domain of this solver. NOTE: this will also reset the actions this solver uses to the actions of the provided domain. If you have previously added non-domain referenced actions through the addNonDomainReferencedAction(burlap.oomdp.singleagent.Action) method, you will have to do so again.

Parameters:

domain - the domain this solver should use.

getDomain

Domain getDomain()

Returns the Domain this solver solves.

Returns:

the Domain this solver solves.

addNonDomainReferencedAction

void addNonDomainReferencedAction(Action a)

Adds an additional action the solver that is not included in the domain definition. For instance, an Option should be added using this method.

Parameters:

a - the action to add to the solver

setActions

void setActions(java.util.List<Action> actions)

Sets the action set the solver should use.

Parameters:

actions - the actions the solver should use.

getActions

java.util.List<Action> getActions()

Returns a copy of all actions this solver uses for reasoning; including added actions that are not part of the domain specification (e.g., Options). Modifying the returned list will not modify the action list this solver uses.

Returns:

a List of all actions this solver uses.

getTF

TerminalFunction getTF()

Returns the TerminalFunction this solver uses.

Returns:

the TerminalFunction this solver uses.

getRF

RewardFunction getRF()

Returns the RewardFunction this solver uses.

Returns:

the RewardFunction this solver uses.

setHashingFactory

void setHashingFactory(HashableStateFactory hashingFactory)

Sets the HashableStateFactory used to hash states for tabular solvers.

Parameters:

hashingFactory - the HashableStateFactory used to hash states for tabular solvers.

getHashingFactory

HashableStateFactory getHashingFactory()

Returns the HashableStateFactory this solver uses.

Returns:

the HashableStateFactory this solver uses.

setRf

void setRf(RewardFunction rf)

Sets the reward function used by this solver

Parameters:

rf - the reward function to be used by this solver

getRf

RewardFunction getRf()

Returns the RewardFunction this solver uses.

Returns:

the RewardFunction this solver uses.

setTf

void setTf(TerminalFunction tf)

Sets the terminal state function used by this solver

Parameters:

tf - the terminal function to be used by this solver

getTf

TerminalFunction getTf()

Returns the TerminalFunction this solver uses.

Returns:

the TerminalFunction this solver uses.

getGamma

double getGamma()

Returns gamma, the discount factor used by this solver

Returns:

gamma, the discount factor used by this solver

setGamma

void setGamma(double gamma)

Sets gamma, the discount factor used by this solver

Parameters:

gamma - the discount factor used by this solver

setDebugCode

void setDebugCode(int code)

Sets the debug code to be used by calls to DPrint

Parameters:

code - the code to be used by DPrint

getDebugCode

int getDebugCode()

Returns the debug code used by this solver for calls to DPrint

Returns:

the debug code used by this solver for calls to DPrint

toggleDebugPrinting

void toggleDebugPrinting(boolean toggle)

Toggles whether the solver's calls to DPrint should be printed.

Parameters:

toggle - whether to print the calls to DPrint