burlap.behavior.policy

Class EpsilonGreedy

java.lang.Object

burlap.behavior.policy.Policy

burlap.behavior.policy.EpsilonGreedy

All Implemented Interfaces:

SolverDerivedPolicy

public class EpsilonGreedy
extends Policy
implements SolverDerivedPolicy

This class defines a an epsilon-greedy policy over Q-values and requires a QComputable valueFunction to be specified. With probability epsilon the policy will return a random action (with uniform distribution over all possible action). With probability 1 - epsilon the policy will return the greedy action. If multiple actions tie for the highest Q-value, then one of the tied actions is randomly selected.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from class burlap.behavior.policy.Policy

Policy.ActionProb, Policy.GroundedAnnotatedAction, Policy.PolicyUndefinedException

Field Summary

Fields

Modifier and Type	Field and Description
protected double	epsilon
protected QFunction	qplanner
protected java.util.Random	rand

Fields inherited from class burlap.behavior.policy.Policy

annotateOptionDecomposition, evaluateDecomposesOptions

Constructor Summary

Constructors

Constructor and Description
EpsilonGreedy(double epsilon) Initializes with the value of epsilon, where epsilon is the probability of taking a random action.
EpsilonGreedy(QFunction planner, double epsilon) Initializes with the QComputablePlanner to use and the value of epsilon to use, where epsilon is the probability of taking a random action.

Method Summary

Methods

Modifier and Type	Method and Description
AbstractGroundedAction	getAction(State s) This method will return an action sampled by the policy for the given state.
java.util.List<Policy.ActionProb>	getActionDistributionForState(State s) This method will return action probability distribution defined by the policy.
double	getEpsilon() Returns the epsilon value, where epsilon is the probability of taking a random action.
boolean	isDefinedFor(State s) Specifies whether this policy is defined for the input state.
boolean	isStochastic() Indicates whether the policy is stochastic or deterministic.
void	setEpsilon(double epsilon) Sets the epsilon value, where epsilon is the probability of taking a random action.
void	setSolver(MDPSolverInterface solver) Sets the valueFunction whose results affect this policy.

Methods inherited from class burlap.behavior.policy.Policy

evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateMethodsShouldAnnotateOptionDecomposition, evaluateMethodsShouldDecomposeOption, followAndRecordPolicy, followAndRecordPolicy, getDeterministicPolicy, getProbOfAction, getProbOfActionGivenDistribution, getProbOfActionGivenDistribution, sampleFromActionDistribution

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

qplanner

protected QFunction qplanner

epsilon

protected double epsilon

rand

protected java.util.Random rand

Constructor Detail

EpsilonGreedy

public EpsilonGreedy(double epsilon)

Initializes with the value of epsilon, where epsilon is the probability of taking a random action.

Parameters:

epsilon - the probability of taking a random action.

EpsilonGreedy

public EpsilonGreedy(QFunction planner,
             double epsilon)

Initializes with the QComputablePlanner to use and the value of epsilon to use, where epsilon is the probability of taking a random action.

Parameters:

planner - the QComputablePlanner to use

epsilon - the probability of taking a random action.

Method Detail

getEpsilon

public double getEpsilon()

Returns the epsilon value, where epsilon is the probability of taking a random action.

Returns:

the epsilon value

setEpsilon

public void setEpsilon(double epsilon)

Sets the epsilon value, where epsilon is the probability of taking a random action.

Parameters:

epsilon - the probability of taking a random action.

setSolver

public void setSolver(MDPSolverInterface solver)

Description copied from interface: SolverDerivedPolicy

Sets the valueFunction whose results affect this policy.

Specified by:

setSolver in interface SolverDerivedPolicy

Parameters:

solver - the solver from which this policy is derived

getAction

public AbstractGroundedAction getAction(State s)

Description copied from class: Policy

This method will return an action sampled by the policy for the given state. If the defined policy is stochastic, then multiple calls to this method for the same state may return different actions. The sampling should be with respect to defined action distribution that is returned by getActionDistributionForState

Specified by:

getAction in class Policy

Parameters:

s - the state for which an action should be returned

Returns:

a sample action from the action distribution; null if the policy is undefined for s

getActionDistributionForState

public java.util.List<Policy.ActionProb> getActionDistributionForState(State s)

Description copied from class: Policy

This method will return action probability distribution defined by the policy. The action distribution is represented by a list of ActionProb objects, each which specifies a grounded action and a probability of that grounded action being taken. The returned list does not have to include actions with probability 0.

Specified by:

getActionDistributionForState in class Policy

Parameters:

s - the state for which an action distribution should be returned

Returns:

a list of possible actions taken by the policy and their probability.

isStochastic

public boolean isStochastic()

Description copied from class: Policy

Indicates whether the policy is stochastic or deterministic.

Specified by:

isStochastic in class Policy

Returns:

true when the policy is stochastic; false when it is deterministic.

isDefinedFor

public boolean isDefinedFor(State s)

Description copied from class: Policy

Specifies whether this policy is defined for the input state.

Specified by:

isDefinedFor in class Policy

Parameters:

s - the input state to test for whether this policy is defined

Returns:

true if this policy is defined for State s, false otherwise.