burlap.behavior.stochasticgames.madynamicprogramming.policies

Class EMinMaxPolicy

java.lang.Object

burlap.behavior.policy.Policy

burlap.behavior.stochasticgames.JointPolicy

burlap.behavior.stochasticgames.madynamicprogramming.MAQSourcePolicy

burlap.behavior.stochasticgames.madynamicprogramming.policies.EMinMaxPolicy

public class EMinMaxPolicy
extends MAQSourcePolicy

Class for following a minmax joint policy. Given some target agent, a minmax joint policy is computed over the space of joint action Q-values. A fraction epsilong of the time though, a random joint action is selected. If the input Q-values are not zero-sum, then they are forced to be from the perspective of the target agent.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from class burlap.behavior.policy.Policy

Policy.ActionProb, Policy.GroundedAnnotatedAction, Policy.PolicyUndefinedException

Field Summary

Fields

Modifier and Type	Field and Description
protected double	epsilon The epsilon parameter specifying how often random joint actions are returned
protected MultiAgentQSourceProvider	qSourceProvider The multi-agent q-source provider
protected java.lang.String	targetAgentQName The target agent who is maximizing action selection

Fields inherited from class burlap.behavior.stochasticgames.JointPolicy

agentsInJointPolicy, agentsSyncrhonizedSoFar, lastSyncedState, lastSynchronizedJointAction

Fields inherited from class burlap.behavior.policy.Policy

annotateOptionDecomposition, evaluateDecomposesOptions

Constructor Summary

Constructors

Constructor and Description
EMinMaxPolicy(double epsilon) Initializes for a given epsilon value; the fraction of the time a random joint action is selected
EMinMaxPolicy(MultiAgentQLearning actingAgent, double epsilon) Initializes for a given Q-learning agent and epsilon value.

Method Summary

Methods

Modifier and Type	Method and Description
JointPolicy	copy() Creates a copy of this joint policy and returns it.
AbstractGroundedAction	getAction(State s) This method will return an action sampled by the policy for the given state.
java.util.List<Policy.ActionProb>	getActionDistributionForState(State s) This method will return action probability distribution defined by the policy.
boolean	isDefinedFor(State s) Specifies whether this policy is defined for the input state.
boolean	isStochastic() Indicates whether the policy is stochastic or deterministic.
void	setQSourceProvider(MultiAgentQSourceProvider provider) Sets the MultiAgentQSourceProvider that will be used to define this object's joint policy.
void	setTargetAgent(java.lang.String agentName) Sets the target privledged agent from which this joint policy is defined.

Methods inherited from class burlap.behavior.stochasticgames.JointPolicy

getAgentsInJointPolicy, getAgentSynchronizedActionSelection, getAllJointActions, setAgentsInJointPolicy, setAgentsInJointPolicy, setAgentsInJointPolicyFromWorld

Methods inherited from class burlap.behavior.policy.Policy

evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateMethodsShouldAnnotateOptionDecomposition, evaluateMethodsShouldDecomposeOption, followAndRecordPolicy, followAndRecordPolicy, getDeterministicPolicy, getProbOfAction, getProbOfActionGivenDistribution, getProbOfActionGivenDistribution, sampleFromActionDistribution

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

qSourceProvider

protected MultiAgentQSourceProvider qSourceProvider

The multi-agent q-source provider

epsilon

protected double epsilon

The epsilon parameter specifying how often random joint actions are returned

targetAgentQName

protected java.lang.String targetAgentQName

The target agent who is maximizing action selection

Constructor Detail

EMinMaxPolicy

public EMinMaxPolicy(double epsilon)

Initializes for a given epsilon value; the fraction of the time a random joint action is selected

Parameters:

epsilon - the espilon parameter

EMinMaxPolicy

public EMinMaxPolicy(MultiAgentQLearning actingAgent,
             double epsilon)

Initializes for a given Q-learning agent and epsilon value. The Q-learning agent is set as the Q-source and they are set as the target agent. Epsilon is the fraction of the time a random joint action is selected.

Parameters:

actingAgent - the Q-learning agent

epsilon - the epsilon parameter

Method Detail

setTargetAgent

public void setTargetAgent(java.lang.String agentName)

Description copied from class: JointPolicy

Sets the target privledged agent from which this joint policy is defined.

Specified by:

setTargetAgent in class JointPolicy

Parameters:

agentName - the name of the target agent.

copy

public JointPolicy copy()

Description copied from class: JointPolicy

Creates a copy of this joint policy and returns it. This is useful when generating different agents using the same kind of policy, but have different target agents evaluating it.

Specified by:

copy in class JointPolicy

Returns:

a copy of this joint policy.

getAction

public AbstractGroundedAction getAction(State s)

Description copied from class: Policy

This method will return an action sampled by the policy for the given state. If the defined policy is stochastic, then multiple calls to this method for the same state may return different actions. The sampling should be with respect to defined action distribution that is returned by getActionDistributionForState

Specified by:

getAction in class Policy

Parameters:

s - the state for which an action should be returned

Returns:

a sample action from the action distribution; null if the policy is undefined for s

getActionDistributionForState

public java.util.List<Policy.ActionProb> getActionDistributionForState(State s)

Description copied from class: Policy

This method will return action probability distribution defined by the policy. The action distribution is represented by a list of ActionProb objects, each which specifies a grounded action and a probability of that grounded action being taken. The returned list does not have to include actions with probability 0.

Specified by:

getActionDistributionForState in class Policy

Parameters:

s - the state for which an action distribution should be returned

Returns:

a list of possible actions taken by the policy and their probability.

isStochastic

public boolean isStochastic()

Description copied from class: Policy

Indicates whether the policy is stochastic or deterministic.

Specified by:

isStochastic in class Policy

Returns:

true when the policy is stochastic; false when it is deterministic.

isDefinedFor

public boolean isDefinedFor(State s)

Description copied from class: Policy

Specifies whether this policy is defined for the input state.

Specified by:

isDefinedFor in class Policy

Parameters:

s - the input state to test for whether this policy is defined

Returns:

true if this policy is defined for State s, false otherwise.

setQSourceProvider

public void setQSourceProvider(MultiAgentQSourceProvider provider)

Description copied from class: MAQSourcePolicy

Sets the MultiAgentQSourceProvider that will be used to define this object's joint policy.

Specified by:

setQSourceProvider in class MAQSourcePolicy

Parameters:

provider - the MultiAgentQSourceProvider that will be used to define this object's joint policy.