burlap.oomdp.singleagent.pomdp

Class BeliefMDPGenerator.BeliefRF

java.lang.Object

burlap.oomdp.singleagent.pomdp.BeliefMDPGenerator.BeliefRF

All Implemented Interfaces:

RewardFunction

Enclosing class:

BeliefMDPGenerator

public static class BeliefMDPGenerator.BeliefRF
extends java.lang.Object
implements RewardFunction

A class for turning a POMDP reward function into a Belief MDP reward function. This class requires that the input State objects are classes that implement BeliefState and EnumerableBeliefState. If the POMDP reward function does not depend on the next state, then this can be declared with the srcRFIsNextStateIndependent flag in the burlap.oomdp.singleagent.pomdp.BeliefMDPGenerator.BeliefRF#BeliefRF(PODomain, RewardFunction, boolean) constructor, which will decrease the computational demands since the next states do not have to be marginalized over.

Field Summary

Fields

Modifier and Type	Field and Description
protected PODomain	podomain The source POMDP domain
protected RewardFunction	pomdpRF The source POMDP reward function to turn into a belief MDP reward function
protected boolean	srcRFIsNextStateIndependent A boolean flag indicating whether the POMDP reward function is independent of the next state transition.

Constructor Summary

Constructors

Constructor and Description
BeliefMDPGenerator.BeliefRF(PODomain podomain, RewardFunction pomdpRF) Initializes.
BeliefMDPGenerator.BeliefRF(PODomain podomain, RewardFunction pomdpRF, boolean srcRFIsNextStateIndependent) Initializes.

Method Summary

Methods

Modifier and Type	Method and Description
double	reward(State s, GroundedAction a, State sprime) Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
protected double	saOnlyReward(State s, GroundedAction a) Returns the belief MDP reward when the POMDP reward function is independent from the next state transition.
protected double	sasReward(State s, GroundedAction a) Returns the belief MDP reward when the POMDP reward function is dependent on the next state transition.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

podomain

protected PODomain podomain

The source POMDP domain

pomdpRF

protected RewardFunction pomdpRF

The source POMDP reward function to turn into a belief MDP reward function

srcRFIsNextStateIndependent

protected boolean srcRFIsNextStateIndependent

A boolean flag indicating whether the POMDP reward function is independent of the next state transition.

Constructor Detail

BeliefMDPGenerator.BeliefRF

public BeliefMDPGenerator.BeliefRF(PODomain podomain,
                           RewardFunction pomdpRF)

Initializes. Class will take the safe (but more computationally demanding) assumption that the POMDP reward function depends on the next state transition.

Parameters:

podomain - the source POMDP domain

pomdpRF - the source POMDP reward function

BeliefMDPGenerator.BeliefRF

public BeliefMDPGenerator.BeliefRF(PODomain podomain,
                           RewardFunction pomdpRF,
                           boolean srcRFIsNextStateIndependent)

Initializes.

Parameters:

podomain - the source POMDP domain

pomdpRF - the source POMDP reward function

srcRFIsNextStateIndependent - a boolean flag indicating whether the POMDP reward function is independent of the next state transition.

Method Detail

reward

public double reward(State s,
            GroundedAction a,
            State sprime)

Description copied from interface: RewardFunction

Returns the reward received when action a is executed in state s and the agent transitions to state sprime.

Specified by:

reward in interface RewardFunction

Parameters:

s - the state in which the action was executed

a - the action executed

sprime - the state to which the agent transitioned

Returns:

the reward received when action a is executed in state s and the agent transitions to state sprime.

saOnlyReward

protected double saOnlyReward(State s,
                  GroundedAction a)

Returns the belief MDP reward when the POMDP reward function is independent from the next state transition.

Parameters:

s - the input belief state

a - the action selected.

Returns:

the belief MDP reward

sasReward

protected double sasReward(State s,
               GroundedAction a)

Returns the belief MDP reward when the POMDP reward function is dependent on the next state transition. Requires marginalizing over the possible next states.

Parameters:

s - the input belief state

a - the action selected.

Returns:

the belief MDP reward