burlap.behavior.stochasticgames

Class PolicyFromJointPolicy

java.lang.Object

burlap.behavior.policy.Policy

burlap.behavior.stochasticgames.PolicyFromJointPolicy

public class PolicyFromJointPolicy
extends Policy

This class defines a single agent's policy that is derived from a joint policy. It takes as input a source joint policy and the agent name whose actions in the returned joint actions are being followed. The action distribution for the agent is determined by marginalizing over other agent's actions in the joint policy as if they were selcted indepently. Note that this assumption may not always be correct, depending on the joint policy.

When the agent name for this policy is set, it automatically calls the JointPolicy.setTargetAgent(String) method of the source joint policy with the same agent name.

Action selection from the underlying joint policy may also be synchronized with multiple agents who are following the same underlying joint policy object. This has the effect of the joint policy choosing a joint action for each time step and causing each agent to follow that selected joint action. See the JointPolicy class documentation for more information on how this works. By default, synchronization will not be used.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from class burlap.behavior.policy.Policy

Policy.ActionProb, Policy.GroundedAnnotatedAction, Policy.PolicyUndefinedException

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.String	actingAgentName The acting agent's name whose actions from the joint policy will be returned.
protected JointPolicy	jointPolicy The underlying joint policy from which actions are selected.
protected boolean	synchronizeJointActionSelectionAmongAgents

Fields inherited from class burlap.behavior.policy.Policy

annotateOptionDecomposition, evaluateDecomposesOptions

Constructor Summary

Constructors

Constructor and Description
PolicyFromJointPolicy(JointPolicy jointPolicy) Initializes with the underlying joint polciy
PolicyFromJointPolicy(JointPolicy jointPolicy, boolean synchronizeJointActionSelectionAmongAgents) Initializes with the underlying joint polciy and whether actions should be synchronized with other agents following the same underlying joint policy.
PolicyFromJointPolicy(java.lang.String actingAgentName, JointPolicy jointPolicy) Initializes with the acting agent name whose actions from the underlying joint policy will be returned.
PolicyFromJointPolicy(java.lang.String actingAgentName, JointPolicy jointPolicy, boolean synchronizeJointActionSelectionAmongAgents) Initializes with the acting agent name whose actions from the underlying joint policy will be returned and whether actions should be synchronized with other agents following the same underlying joint policy.

Method Summary

Methods

Modifier and Type	Method and Description
PolicyFromJointPolicy	copy() Returns a copy of this policy, which entails of first making a copy of the joint policy.
java.lang.String	getActingAgentName() Returns the acting agent's name
AbstractGroundedAction	getAction(State s) This method will return an action sampled by the policy for the given state.
java.util.List<Policy.ActionProb>	getActionDistributionForState(State s) This method will return action probability distribution defined by the policy.
JointPolicy	getJointPolicy() Returns the underlying joint policy
boolean	isDefinedFor(State s) Specifies whether this policy is defined for the input state.
boolean	isStochastic() Indicates whether the policy is stochastic or deterministic.
void	setActingAgentName(java.lang.String agentName) Sets the acting agents name
void	setJointPolicy(JointPolicy jointPolicy) Sets the underlying joint policy
void	setSynchronizeJointActionSelectionAmongAgents(boolean synchronizeJointActionSelectionAmongAgents) Sets whether actions selection of this agent's policy should be synchronized with the action selection of other agents following the same underlying joint policy.

Methods inherited from class burlap.behavior.policy.Policy

evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateMethodsShouldAnnotateOptionDecomposition, evaluateMethodsShouldDecomposeOption, followAndRecordPolicy, followAndRecordPolicy, getDeterministicPolicy, getProbOfAction, getProbOfActionGivenDistribution, getProbOfActionGivenDistribution, sampleFromActionDistribution

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

jointPolicy

protected JointPolicy jointPolicy

The underlying joint policy from which actions are selected.

actingAgentName

protected java.lang.String actingAgentName

The acting agent's name whose actions from the joint policy will be returned.

synchronizeJointActionSelectionAmongAgents

protected boolean synchronizeJointActionSelectionAmongAgents

Constructor Detail

PolicyFromJointPolicy

public PolicyFromJointPolicy(JointPolicy jointPolicy)

Initializes with the underlying joint polciy

Parameters:

jointPolicy - the underlying joint polciy

PolicyFromJointPolicy

public PolicyFromJointPolicy(JointPolicy jointPolicy,
                     boolean synchronizeJointActionSelectionAmongAgents)

Initializes with the underlying joint polciy and whether actions should be synchronized with other agents following the same underlying joint policy.

Parameters:

jointPolicy - the underlying joint polciy

synchronizeJointActionSelectionAmongAgents - whether actions should be synchronized with other agents following the same underlying joint policy.

PolicyFromJointPolicy

public PolicyFromJointPolicy(java.lang.String actingAgentName,
                     JointPolicy jointPolicy)

Initializes with the acting agent name whose actions from the underlying joint policy will be returned.

Parameters:

actingAgentName - the acting agent name

jointPolicy - the underlying joint polciy

PolicyFromJointPolicy

public PolicyFromJointPolicy(java.lang.String actingAgentName,
                     JointPolicy jointPolicy,
                     boolean synchronizeJointActionSelectionAmongAgents)

Initializes with the acting agent name whose actions from the underlying joint policy will be returned and whether actions should be synchronized with other agents following the same underlying joint policy.

Parameters:

actingAgentName - the acting agent name

jointPolicy - the underlying joint polciy

synchronizeJointActionSelectionAmongAgents - whether actions should be synchronized with other agents following the same underlying joint policy.

Method Detail

setJointPolicy

public void setJointPolicy(JointPolicy jointPolicy)

Sets the underlying joint policy

Parameters:

jointPolicy - the underlying joint polciy

getJointPolicy

public JointPolicy getJointPolicy()

Returns the underlying joint policy

Returns:

the underlying joint policy

setActingAgentName

public void setActingAgentName(java.lang.String agentName)

Sets the acting agents name

Parameters:

agentName - the acting agent's name

setSynchronizeJointActionSelectionAmongAgents

public void setSynchronizeJointActionSelectionAmongAgents(boolean synchronizeJointActionSelectionAmongAgents)

Sets whether actions selection of this agent's policy should be synchronized with the action selection of other agents following the same underlying joint policy.

Parameters:

synchronizeJointActionSelectionAmongAgents - whether agent actions should be synchronized or not.

getActingAgentName

public java.lang.String getActingAgentName()

Returns the acting agent's name

Returns:

the acting agent's name

getAction

public AbstractGroundedAction getAction(State s)

Description copied from class: Policy

This method will return an action sampled by the policy for the given state. If the defined policy is stochastic, then multiple calls to this method for the same state may return different actions. The sampling should be with respect to defined action distribution that is returned by getActionDistributionForState

Specified by:

getAction in class Policy

Parameters:

s - the state for which an action should be returned

Returns:

a sample action from the action distribution; null if the policy is undefined for s

getActionDistributionForState

public java.util.List<Policy.ActionProb> getActionDistributionForState(State s)

Description copied from class: Policy

This method will return action probability distribution defined by the policy. The action distribution is represented by a list of ActionProb objects, each which specifies a grounded action and a probability of that grounded action being taken. The returned list does not have to include actions with probability 0.

Specified by:

getActionDistributionForState in class Policy

Parameters:

s - the state for which an action distribution should be returned

Returns:

a list of possible actions taken by the policy and their probability.

isStochastic

public boolean isStochastic()

Description copied from class: Policy

Indicates whether the policy is stochastic or deterministic.

Specified by:

isStochastic in class Policy

Returns:

true when the policy is stochastic; false when it is deterministic.

isDefinedFor

public boolean isDefinedFor(State s)

Description copied from class: Policy

Specifies whether this policy is defined for the input state.

Specified by:

isDefinedFor in class Policy

Parameters:

s - the input state to test for whether this policy is defined

Returns:

true if this policy is defined for State s, false otherwise.

copy

public PolicyFromJointPolicy copy()

Returns a copy of this policy, which entails of first making a copy of the joint policy.

Returns:

a copy of this policy.