burlap.behavior.singleagent.pomdp.wrappedmdpalgs

Class BeliefSparseSampling

java.lang.Object

burlap.behavior.singleagent.MDPSolver

burlap.behavior.singleagent.pomdp.wrappedmdpalgs.BeliefSparseSampling

All Implemented Interfaces:

MDPSolverInterface, Planner, QFunction, QProvider, ValueFunction

public class BeliefSparseSampling
extends MDPSolver
implements Planner, QProvider

A POMDP planning algorithm that converts a POMDP into a Belief MDP and then uses SparseSampling to solve it. If the full transition dynamics are used (set c in the constructor to -1), then it provides and optimal finite horizon POMDP policy.

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.valuefunction.QProvider

QProvider.Helper

Field Summary

Fields

Modifier and Type	Field and Description
protected SADomain	beliefMDP The belief MDP domain to solve.
protected SparseSampling	mdpPlanner The SparseSampling planning instance to solve the problem.

Fields inherited from class burlap.behavior.singleagent.MDPSolver

actionTypes, debugCode, domain, gamma, hashingFactory, model, usingOptionModel

Constructor Summary

Constructors

Constructor and Description
BeliefSparseSampling(PODomain domain, double discount, HashableStateFactory hashingFactory, int h, int c) Initializes the planner.

Method Summary

Modifier and Type	Method and Description
SADomain	getBeliefMDP() Returns the generated Belief MDP that will be solved.
SparseSampling	getSparseSamplingPlanner() Returns the SparseSampling planning used to solve the Belief MDP.
static void	main(java.lang.String[] args)
Policy	planFromState(State initialState) This method will cause the Planner to begin planning from the specified initial State.
double	qValue(State s, Action a) Returns the QValue for the given state-action pair.
java.util.List<QValue>	qValues(State s) Returns a List of QValue objects for ever permissible action for the given input state.
void	resetSolver() This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.
double	value(State s) Returns the value function evaluation of the given state.

Methods inherited from class burlap.behavior.singleagent.MDPSolver

addActionType, applicableActions, getActionTypes, getDebugCode, getDomain, getGamma, getHashingFactory, getModel, setActionTypes, setDebugCode, setDomain, setGamma, setHashingFactory, setModel, solverInit, stateHash, toggleDebugPrinting

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface burlap.behavior.singleagent.MDPSolverInterface

addActionType, getActionTypes, getDebugCode, getDomain, getGamma, getHashingFactory, getModel, setActionTypes, setDebugCode, setDomain, setGamma, setHashingFactory, setModel, solverInit, toggleDebugPrinting

Field Detail

beliefMDP

protected SADomain beliefMDP

The belief MDP domain to solve.

mdpPlanner

protected SparseSampling mdpPlanner

The SparseSampling planning instance to solve the problem.

Constructor Detail

BeliefSparseSampling

public BeliefSparseSampling(PODomain domain,
                            double discount,
                            HashableStateFactory hashingFactory,
                            int h,
                            int c)

Initializes the planner.

Parameters:

domain - the POMDP domain

discount - the discount factor

hashingFactory - the Belief MDP HashableStateFactory that SparseSampling will use.

h - the height of the SparseSampling tree.

c - the number of samples SparseSampling will use. Set to -1 to use the full BeliefMDP transition dynamics.

Method Detail

getBeliefMDP

public SADomain getBeliefMDP()

Returns the generated Belief MDP that will be solved.

Returns:

the generated Belief MDP that will be solved.

getSparseSamplingPlanner

public SparseSampling getSparseSamplingPlanner()

Returns the SparseSampling planning used to solve the Belief MDP.

Returns:

the SparseSampling planning used to solve the Belief MDP.

qValues

public java.util.List<QValue> qValues(State s)

Description copied from interface: QProvider

Returns a List of QValue objects for ever permissible action for the given input state.

Specified by:

qValues in interface QProvider

Parameters:

s - the state for which Q-values are to be returned.

Returns:

a List of QValue objects for ever permissible action for the given input state.

qValue

public double qValue(State s,
                     Action a)

Description copied from interface: QFunction

Returns the QValue for the given state-action pair.

Specified by:

qValue in interface QFunction

Parameters:

s - the input state

a - the input action

Returns:

the QValue for the given state-action pair.

planFromState

public Policy planFromState(State initialState)

Description copied from interface: Planner

This method will cause the Planner to begin planning from the specified initial State. It will then return an appropriate Policy object that captured the planning results. Note that typically you can use a variety of different Policy objects in conjunction with this Planner to get varying behavior and the returned Policy is not required to be used.

Specified by:

planFromState in interface Planner

Parameters:

initialState - the initial state of the planning problem

Returns:

a Policy that captures the planning results from input State.

resetSolver

public void resetSolver()

Description copied from interface: MDPSolverInterface

This method resets all solver results so that a solver can be restarted fresh as if had never solved the MDP.

Specified by:

resetSolver in interface MDPSolverInterface

Specified by:

resetSolver in class MDPSolver

value

public double value(State s)

Description copied from interface: ValueFunction

Returns the value function evaluation of the given state. If the value is not stored, then the default value specified by the ValueFunctionInitialization object of this class is returned.

Specified by:

value in interface ValueFunction

Parameters:

s - the state to evaluate.

Returns:

the value function evaluation of the given state.

main

public static void main(java.lang.String[] args)