burlap.behavior.stochasticgames

Class PolicyFromJointPolicy

java.lang.Object

burlap.behavior.stochasticgames.PolicyFromJointPolicy

All Implemented Interfaces:

EnumerablePolicy, Policy

public class PolicyFromJointPolicy
extends java.lang.Object
implements EnumerablePolicy

This class defines a single agent's policy that is derived from a joint policy. It takes as input a source joint policy and the agent name whose actions in the returned joint actions are being followed. The action distribution for the agent is determined by marginalizing over other agent's actions in the joint policy as if they were selcted indepently. Note that this assumption may not always be correct, depending on the joint policy.

When the agent name for this policy is set, it automatically calls the JointPolicy.setTargetAgent(int) method of the source joint policy with the same agent name.

Action selection from the underlying joint policy may also be synchronized with multiple agents who are following the same underlying joint policy object. This has the effect of the joint policy choosing a joint action for each time step and causing each agent to follow that selected joint action. See the JointPolicy class documentation for more information on how this works. By default, synchronization will not be used.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected int	actingAgent
protected JointPolicy	jointPolicy The underlying joint policy from which actions are selected.
protected boolean	synchronizeJointActionSelectionAmongAgents

Constructor Summary

Constructors

Constructor and Description
PolicyFromJointPolicy(int actingAgent, JointPolicy jointPolicy) Initializes with the acting agent name whose actions from the underlying joint policy will be returned.
PolicyFromJointPolicy(int actingAgent, JointPolicy jointPolicy, boolean synchronizeJointActionSelectionAmongAgents) Initializes with the acting agent name whose actions from the underlying joint policy will be returned and whether actions should be synchronized with other agents following the same underlying joint policy.
PolicyFromJointPolicy(JointPolicy jointPolicy) Initializes with the underlying joint polciy
PolicyFromJointPolicy(JointPolicy jointPolicy, boolean synchronizeJointActionSelectionAmongAgents) Initializes with the underlying joint polciy and whether actions should be synchronized with other agents following the same underlying joint policy.

Method Summary

Modifier and Type	Method and Description
Action	action(State s) This method will return an action sampled by the policy for the given state.
double	actionProb(State s, Action a) Returns the probability/probability density that the given action will be taken in the given state.
PolicyFromJointPolicy	copy() Returns a copy of this policy, which entails of first making a copy of the joint policy.
boolean	definedFor(State s) Specifies whether this policy is defined for the input state.
int	getActingAgent() Returns the acting agent
JointPolicy	getJointPolicy() Returns the underlying joint policy
java.util.List<ActionProb>	policyDistribution(State s) This method will return action probability distribution defined by the policy.
void	setActingAgent(int agentNum) Sets the acting agents name
void	setJointPolicy(JointPolicy jointPolicy) Sets the underlying joint policy
void	setSynchronizeJointActionSelectionAmongAgents(boolean synchronizeJointActionSelectionAmongAgents) Sets whether actions selection of this agent's policy should be synchronized with the action selection of other agents following the same underlying joint policy.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

jointPolicy

protected JointPolicy jointPolicy

The underlying joint policy from which actions are selected.

actingAgent

protected int actingAgent

synchronizeJointActionSelectionAmongAgents

protected boolean synchronizeJointActionSelectionAmongAgents

Constructor Detail

PolicyFromJointPolicy

public PolicyFromJointPolicy(JointPolicy jointPolicy)

Initializes with the underlying joint polciy

Parameters:

jointPolicy - the underlying joint polciy

PolicyFromJointPolicy

public PolicyFromJointPolicy(JointPolicy jointPolicy,
                             boolean synchronizeJointActionSelectionAmongAgents)

Initializes with the underlying joint polciy and whether actions should be synchronized with other agents following the same underlying joint policy.

Parameters:

jointPolicy - the underlying joint polciy

synchronizeJointActionSelectionAmongAgents - whether actions should be synchronized with other agents following the same underlying joint policy.

PolicyFromJointPolicy

public PolicyFromJointPolicy(int actingAgent,
                             JointPolicy jointPolicy)

Initializes with the acting agent name whose actions from the underlying joint policy will be returned.

Parameters:

actingAgent - the acting agent name

jointPolicy - the underlying joint polciy

PolicyFromJointPolicy

public PolicyFromJointPolicy(int actingAgent,
                             JointPolicy jointPolicy,
                             boolean synchronizeJointActionSelectionAmongAgents)

Initializes with the acting agent name whose actions from the underlying joint policy will be returned and whether actions should be synchronized with other agents following the same underlying joint policy.

Parameters:

actingAgent - the acting agent name

jointPolicy - the underlying joint polciy

synchronizeJointActionSelectionAmongAgents - whether actions should be synchronized with other agents following the same underlying joint policy.

Method Detail

setJointPolicy

public void setJointPolicy(JointPolicy jointPolicy)

Sets the underlying joint policy

Parameters:

jointPolicy - the underlying joint polciy

getJointPolicy

public JointPolicy getJointPolicy()

Returns the underlying joint policy

Returns:

the underlying joint policy

setActingAgent

public void setActingAgent(int agentNum)

Sets the acting agents name

Parameters:

agentNum - the acting agent

setSynchronizeJointActionSelectionAmongAgents

public void setSynchronizeJointActionSelectionAmongAgents(boolean synchronizeJointActionSelectionAmongAgents)

Sets whether actions selection of this agent's policy should be synchronized with the action selection of other agents following the same underlying joint policy.

Parameters:

synchronizeJointActionSelectionAmongAgents - whether agent actions should be synchronized or not.

getActingAgent

public int getActingAgent()

Returns the acting agent

Returns:

the acting agent

action

public Action action(State s)

Description copied from interface: Policy

This method will return an action sampled by the policy for the given state. If the defined policy is stochastic, then multiple calls to this method for the same state may return different actions. The sampling should be with respect to defined action distribution that is returned by getActionDistributionForState

Specified by:

action in interface Policy

Parameters:

s - the state for which an action should be returned

Returns:

a sample action from the action distribution; null if the policy is undefined for s

actionProb

public double actionProb(State s,
                         Action a)

Description copied from interface: Policy

Returns the probability/probability density that the given action will be taken in the given state.

Specified by:

actionProb in interface Policy

Parameters:

s - the state of interest

a - the action that may be taken in the state

Returns:

the probability/probability density

policyDistribution

public java.util.List<ActionProb> policyDistribution(State s)

Description copied from interface: EnumerablePolicy

This method will return action probability distribution defined by the policy. The action distribution is represented by a list of ActionProb objects, each which specifies a grounded action and a probability of that grounded action being taken. The returned list does not have to include actions with probability 0.

Specified by:

policyDistribution in interface EnumerablePolicy

Parameters:

s - the state for which an action distribution should be returned

Returns:

a list of possible actions taken by the policy and their probability.

definedFor

public boolean definedFor(State s)

Description copied from interface: Policy

Specifies whether this policy is defined for the input state.

Specified by:

definedFor in interface Policy

Parameters:

s - the input state to test for whether this policy is defined

Returns:

true if this policy is defined for State s, false otherwise.

copy

public PolicyFromJointPolicy copy()

Returns a copy of this policy, which entails of first making a copy of the joint policy.

Returns:

a copy of this policy.