burlap.behavior.singleagent.options

Class LocalSubgoalRF

java.lang.Object

burlap.behavior.singleagent.options.LocalSubgoalRF

All Implemented Interfaces:

RewardFunction

public class LocalSubgoalRF
extends java.lang.Object
implements RewardFunction

It is typical for options to be defined for following policies to subgoals and it is often useful to use a planning or learning algorithm to define these policies, in which case a subgoal reward function for the option would need to be specified. This reward function defines a set of states in which an option is applicable and the subgoal states of the option. The subgoal state and applicable states are specified using StateConditionTest objects. By default, a subgoal reward of 0 is returned for transitions to the subgoal states, the most negative value for transitions to states in which the option is not applicable and -1 for any other transitions. All of these values can be changed.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected StateConditionTest	applicableStateTest Defines the set of states in which the option is applicable
protected double	defaultReward Defines the reward returned for transitions to applicable states, but not subgoal states; default -1
protected double	failReward Defines the reward returned for transitions to states in which the option is not applicable; default -Double.MAX_VALUE
protected double	subgoalReward Defines the reward returned for transitions to subgoal states; default 0.
protected StateConditionTest	subgoalStateTest Defines he set of subgoal states for the option

Constructor Summary

Constructors

Constructor and Description
LocalSubgoalRF(StateConditionTest subgoalStateTest) Initializes with a given set of subgoal states.
LocalSubgoalRF(StateConditionTest subgoalStateTest, double defaultReward, double subgoalReward) Initializes with a given set of subgoal states, a default reward and a subgoal reward.
LocalSubgoalRF(StateConditionTest applicableStateTest, StateConditionTest subgoalStateTest) Initializes with a set of states in which an option is applicable and which the agent should not enter and a set of subgoal states
LocalSubgoalRF(StateConditionTest applicableStateTest, StateConditionTest subgoalStateTest, double defaultReward, double failReward, double subgoalReward) Initializes

Method Summary

Methods

Modifier and Type	Method and Description
double	reward(State s, GroundedAction a, State sprime) Returns the reward received when action a is executed in state s and the agent transitions to state sprime.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

applicableStateTest

protected StateConditionTest applicableStateTest

Defines the set of states in which the option is applicable

subgoalStateTest

protected StateConditionTest subgoalStateTest

Defines he set of subgoal states for the option

subgoalReward

protected double subgoalReward

Defines the reward returned for transitions to subgoal states; default 0.

defaultReward

protected double defaultReward

Defines the reward returned for transitions to applicable states, but not subgoal states; default -1

failReward

protected double failReward

Defines the reward returned for transitions to states in which the option is not applicable; default -Double.MAX_VALUE

Constructor Detail

LocalSubgoalRF

public LocalSubgoalRF(StateConditionTest subgoalStateTest)

Initializes with a given set of subgoal states. The set of applicable states is assumed to be every state.

Parameters:

subgoalStateTest - the subgoal states

LocalSubgoalRF

public LocalSubgoalRF(StateConditionTest subgoalStateTest,
              double defaultReward,
              double subgoalReward)

Initializes with a given set of subgoal states, a default reward and a subgoal reward. The set of applicable states for the option is assumed to be every state.

Parameters:

subgoalStateTest - the subgoal states

defaultReward - the default reward

subgoalReward - the reward returned for transitioning to subgoal states

LocalSubgoalRF

public LocalSubgoalRF(StateConditionTest applicableStateTest,
              StateConditionTest subgoalStateTest)

Initializes with a set of states in which an option is applicable and which the agent should not enter and a set of subgoal states

Parameters:

applicableStateTest - the applicable states. Transitioning to a non-applicable state causes a reward of failReward.

subgoalStateTest - the subgoal states

LocalSubgoalRF

public LocalSubgoalRF(StateConditionTest applicableStateTest,
              StateConditionTest subgoalStateTest,
              double defaultReward,
              double failReward,
              double subgoalReward)

Initializes

Parameters:

applicableStateTest - Defines the set of states in which the option is applicable. Transitioning to a non-applicable state causes a reward of failReward.

subgoalStateTest - the subgoal states

defaultReward - the default reward

failReward - the reward for transitioning to a non-subgoal non-applicable state

subgoalReward - the reward returned for transitioning to subgoal states

Method Detail

reward

public double reward(State s,
            GroundedAction a,
            State sprime)

Description copied from interface: RewardFunction

Returns the reward received when action a is executed in state s and the agent transitions to state sprime.

Specified by:

reward in interface RewardFunction

Parameters:

s - the state in which the action was executed

a - the action executed

sprime - the state to which the agent transitioned

Returns:

the reward received when action a is executed in state s and the agent transitions to state sprime.