burlap.behavior.singleagent.options

Class EnvironmentOptionOutcome

java.lang.Object

burlap.mdp.singleagent.environment.EnvironmentOutcome

burlap.behavior.singleagent.options.EnvironmentOptionOutcome

public class EnvironmentOptionOutcome
extends EnvironmentOutcome

An EnvironmentOutcome class for reporting the effects of applying an Option in a given Environment. This class extends the standard EnvironmentOutcome to include the discount to apply to the value of time steps following the application of an Option and the number of steps taken by the Option in the Environment. The discount is therefore the gamma^t, where gamma is the MDP discount factor and t is the number of time steps taken by the option. The saved reward value (EnvironmentOutcome.r) for this object will also represent the cumulative discounted reward.

Author:

James MacGlashan.

Field Summary

Fields

Modifier and Type	Field and Description
double	discount The discount factor to apply to the value of time steps immediately following the application of an Option.
Episode	episode The executed episode from this execution

Fields inherited from class burlap.mdp.singleagent.environment.EnvironmentOutcome

a, o, op, r, terminated

Constructor Summary

Constructors

Constructor and Description
EnvironmentOptionOutcome(State s, Action a, State sp, double r, boolean terminated, double discountFactor, Episode episode) Initializes.

Method Summary

Modifier and Type	Method and Description
int	numSteps()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

discount

public double discount

The discount factor to apply to the value of time steps immediately following the application of an Option. Specifically, this value is gamma^t where gamma is the discount factor of the MDP and t is the number of time steps taken by the option.

episode

public Episode episode

The executed episode from this execution

Constructor Detail

EnvironmentOptionOutcome

public EnvironmentOptionOutcome(State s,
                                Action a,
                                State sp,
                                double r,
                                boolean terminated,
                                double discountFactor,
                                Episode episode)

Initializes. Note that discount of this object will be set to discountFactor^numSteps, since discountFactor is the discount factor of the MDP and discount represents the amount values in the time step following the option application should be discounted.

Parameters:

s - The previous state of the environment when the action was taken.

a - The action taken in the environment

sp - The next state to which the environment transitioned

r - The reward received

terminated - Whether the next state to which the environment transitioned is a terminal state (true if so, false otherwise)

discountFactor - The discount factor of the MDP.

episode - the episode of execution

Method Detail

numSteps

public int numSteps()