burlap.behavior.singleagent.pomdp.wrappedmdpalgs

Class BeliefSparseSampling

java.lang.Object

burlap.behavior.singleagent.MDPSolver

burlap.behavior.singleagent.pomdp.wrappedmdpalgs.BeliefSparseSampling

All Implemented Interfaces:

MDPSolverInterface, Planner, QFunction, ValueFunction

public class BeliefSparseSampling
extends MDPSolver
implements Planner, QFunction

A POMDP planning algorithm that converts a POMDP into a Belief MDP and then uses SparseSampling to solve it. If the full transition dynamics are used (set c in the constructor to -1), then it provides and optimal finite horizon POMDP policy.

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.valuefunction.QFunction

QFunction.QFunctionHelper

Field Summary

Fields

Modifier and Type	Field and Description
protected SADomain	beliefMDP The belief MDP domain to solve.
protected RewardFunction	beliefRF The belief MDP reward function
protected SparseSampling	mdpPlanner The SparseSampling planning instance to solve the problem.

Fields inherited from class burlap.behavior.singleagent.MDPSolver

actions, debugCode, domain, gamma, hashingFactory, mapToStateIndex, rf, tf

Constructor Summary

Constructors

Constructor and Description
BeliefSparseSampling(PODomain domain, RewardFunction rf, double discount, HashableStateFactory hashingFactory, int h, int c) Initializes the planner.

Method Summary

Methods

Modifier and Type	Method and Description
SADomain	getBeliefMDP() Returns the generated Belief MDP that will be solved.
QValue	getQ(State s, AbstractGroundedAction a) Returns the QValue for the given state-action pair.
java.util.List<QValue>	getQs(State s) Returns a List of QValue objects for ever permissible action for the given input state.
SparseSampling	getSparseSamplingPlanner() Returns the SparseSampling planning used to solve the Belief MDP.
static void	main(java.lang.String[] args)
Policy	planFromState(State initialState) This method will cause the Planner to begin planning from the specified initial State.
void	resetSolver() This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.
double	value(State s) Returns the value function evaluation of the given state.

Methods inherited from class burlap.behavior.singleagent.MDPSolver

addNonDomainReferencedAction, getActions, getAllGroundedActions, getDebugCode, getDomain, getGamma, getHashingFactory, getRf, getRF, getTf, getTF, setActions, setDebugCode, setDomain, setGamma, setHashingFactory, setRf, setTf, solverInit, stateHash, toggleDebugPrinting, translateAction

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface burlap.behavior.singleagent.MDPSolverInterface

addNonDomainReferencedAction, getActions, getDebugCode, getDomain, getGamma, getHashingFactory, getRf, getRF, getTf, getTF, setActions, setDebugCode, setDomain, setGamma, setHashingFactory, setRf, setTf, solverInit, toggleDebugPrinting

Field Detail

beliefMDP

protected SADomain beliefMDP

The belief MDP domain to solve.

beliefRF

protected RewardFunction beliefRF

The belief MDP reward function

mdpPlanner

protected SparseSampling mdpPlanner

The SparseSampling planning instance to solve the problem.

Constructor Detail

BeliefSparseSampling

public BeliefSparseSampling(PODomain domain,
                    RewardFunction rf,
                    double discount,
                    HashableStateFactory hashingFactory,
                    int h,
                    int c)

Initializes the planner.

Parameters:

domain - the POMDP domain

rf - the POMDP reward function

discount - the discount factor

hashingFactory - the Belief MDP HashableStateFactory that SparseSampling will use.

h - the height of the SparseSampling tree.

c - the number of samples SparseSampling will use. Set to -1 to use the full BeliefMDP transition dynamics.

Method Detail

getBeliefMDP

public SADomain getBeliefMDP()

Returns the generated Belief MDP that will be solved.

Returns:

the generated Belief MDP that will be solved.

getSparseSamplingPlanner

public SparseSampling getSparseSamplingPlanner()

Returns the SparseSampling planning used to solve the Belief MDP.

Returns:

the SparseSampling planning used to solve the Belief MDP.

getQs

public java.util.List<QValue> getQs(State s)

Description copied from interface: QFunction

Returns a List of QValue objects for ever permissible action for the given input state.

Specified by:

getQs in interface QFunction

Parameters:

s - the state for which Q-values are to be returned.

Returns:

a List of QValue objects for ever permissible action for the given input state.

getQ

public QValue getQ(State s,
          AbstractGroundedAction a)

Description copied from interface: QFunction

Returns the QValue for the given state-action pair.

Specified by:

getQ in interface QFunction

Parameters:

s - the input state

a - the input action

Returns:

the QValue for the given state-action pair.

planFromState

public Policy planFromState(State initialState)

Description copied from interface: Planner

This method will cause the Planner to begin planning from the specified initial State. It will then return an appropriate Policy object that captured the planning results. Note that typically you can use a variety of different Policy objects in conjunction with this Planner to get varying behavior and the returned Policy is not required to be used.

Specified by:

planFromState in interface Planner

Parameters:

initialState - the initial state of the planning problem

Returns:

a Policy that captures the planning results from input State.

resetSolver

public void resetSolver()

Description copied from interface: MDPSolverInterface

This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.

Specified by:

resetSolver in interface MDPSolverInterface

Specified by:

resetSolver in class MDPSolver

value

public double value(State s)

Description copied from interface: ValueFunction

Returns the value function evaluation of the given state. If the value is not stored, then the default value specified by the ValueFunctionInitialization object of this class is returned.

Specified by:

value in interface ValueFunction

Parameters:

s - the state to evaluate.

Returns:

the value function evaluation of the given state.

main

public static void main(java.lang.String[] args)