burlap.behavior.singleagent.options.support

Class EnvironmentOptionOutcome

java.lang.Object

burlap.oomdp.singleagent.environment.EnvironmentOutcome

burlap.behavior.singleagent.options.support.EnvironmentOptionOutcome

public class EnvironmentOptionOutcome
extends EnvironmentOutcome

An EnvironmentOutcome class for reporting the effects of applying an Option in a given Environment. This class extends the standard EnvironmentOutcome to include the discount to apply to the value of time steps following the application of an Option and the number of steps taken by the Option in the Environment. The discount is therefore the gamma^t, where gamma is the MDP discount factor and t is the number of time steps taken by the option. The saved reward value (EnvironmentOutcome.r) for this object will also represent the cumulative discounted reward.

Author:

James MacGlashan.

Field Summary

Fields

Modifier and Type	Field and Description
double	discount The discount factor to apply to the value of time steps immediately following the application of an Option.
int	numSteps The number of time steps for which the option was executed.

Fields inherited from class burlap.oomdp.singleagent.environment.EnvironmentOutcome

a, o, op, r, terminated

Constructor Summary

Constructors

Constructor and Description
EnvironmentOptionOutcome(State s, GroundedAction a, State sp, double r, boolean terminated, double discountFactor, int numSteps) Initializes.

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

discount

public double discount

The discount factor to apply to the value of time steps immediately following the application of an Option. Specifically, this value is gamma^t where gamma is the discount factor of the MDP and t is the number of time steps taken by the option.

numSteps

public int numSteps

The number of time steps for which the option was executed.

Constructor Detail

EnvironmentOptionOutcome

public EnvironmentOptionOutcome(State s,
                        GroundedAction a,
                        State sp,
                        double r,
                        boolean terminated,
                        double discountFactor,
                        int numSteps)

Initializes. Note that discount of this object will be set to discountFactor^numSteps, since discountFactor is the discount factor of the MDP and discount represents the amount values in the time step following the option application should be discounted.

Parameters:

s - The previous state of the environment when the action was taken.

a - The action taken in the environment

sp - The next state to which the environment transitioned

r - The reward received

terminated - Whether the next state to which the environment transitioned is a terminal state (true if so, false otherwise)

discountFactor - The discount factor of the MDP.

numSteps - The number of time steps for which the option was executed.