burlap.behavior.singleagent.options

Interface Option

All Superinterfaces:

Action

All Known Implementing Classes:

MacroAction, SubgoalOption

public interface Option
extends Action

An interface for Options [1] that extends the Action interface. Requires additional methods for defining the option, initiation set, termination conditions, its policy, whether the option is Markov, and giving it control in an environment.

The policy methods policy(State, Episode), policyDistribution(State, Episode) and the termination conditions method probabilityOfTermination(State, Episode) take as input a history (provided as an Episode object) so that Non Markov options can be supported. If the option is Markov, these history parameters can be null.

the control(Environment, double) method can generally be implemented using the control(Environment, double) method, but you can also implement it your own way if desired.

1. Sutton, Richard S., Doina Precup, and Satinder Singh. "Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning." Artificial intelligence 112.1 (1999): 181-211.

Author:

James MacGlashan

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	Option.Helper

Method Summary

Modifier and Type	Method and Description
EnvironmentOptionOutcome	control(Environment env, double discount)
boolean	inInitiationSet(State s) Returns true if the input state is in the initiation set of the Option
boolean	markov()
Action	policy(State s, Episode history)
java.util.List<ActionProb>	policyDistribution(State s, Episode history)
double	probabilityOfTermination(State s, Episode history)

Methods inherited from interface burlap.mdp.core.action.Action

actionName, copy

Method Detail

inInitiationSet

boolean inInitiationSet(State s)

Returns true if the input state is in the initiation set of the Option

Parameters:

s - the State to test.

Returns:

true if the state is in the initiation set; false if it is not

policy

Action policy(State s,
              Episode history)

policyDistribution

java.util.List<ActionProb> policyDistribution(State s,
                                              Episode history)

probabilityOfTermination

double probabilityOfTermination(State s,
                                Episode history)

control

EnvironmentOptionOutcome control(Environment env,
                                 double discount)

markov

boolean markov()