burlap.behavior.stochasticgames

Class JointPolicy

java.lang.Object

burlap.behavior.policy.Policy

burlap.behavior.stochasticgames.JointPolicy

Direct Known Subclasses:

MAQSourcePolicy

public abstract class JointPolicy
extends Policy

An abstract Policy object for defining a stochastic games joint policy; that is, a policy over joint actions taken by all agents in the world. The primary extension of this object is a set of agent definitions which define the set of possible joint actions in any state. An agent definition consists of an agent name and their SGAgentType, the latter of which specifies the set of individual actions that they can take and which object class represents their state.

It is not uncommon for some joint policies to be defined from a privledged agent's position. This class also contains an abstract method for setting that target privledge agent: setTargetAgent(String). If the joint policy is agent agnostic, then this method does not need to do anything.

This class can also be used to synchonize the action selection of multiple agents according to the same sampled joint action. This is achieved by using the getAgentSynchronizedActionSelection(String, State) method, which returns the single action for each agent (of the specified name) from the same sampled joint action until all agents defined in the policy have queried the method for their action selection.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from class burlap.behavior.policy.Policy

Policy.ActionProb, Policy.GroundedAnnotatedAction, Policy.PolicyUndefinedException

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Map<java.lang.String,SGAgentType>	agentsInJointPolicy The agent definitions that define the set of possible joint actions in each state.
protected java.util.Set<java.lang.String>	agentsSyncrhonizedSoFar The agents whose actiosn have been syncrhonized so far
protected State	lastSyncedState The last state in which synchronized actions were queried.
protected JointAction	lastSynchronizedJointAction The last synchronized joint action that was selected

Fields inherited from class burlap.behavior.policy.Policy

annotateOptionDecomposition, evaluateDecomposesOptions

Constructor Summary

Constructors

Constructor and Description
JointPolicy()

Method Summary

Methods

Modifier and Type	Method and Description
abstract JointPolicy	copy() Creates a copy of this joint policy and returns it.
java.util.Map<java.lang.String,SGAgentType>	getAgentsInJointPolicy() Returns a map specifying the agents who contribute actions to this joint policy.
AbstractGroundedAction	getAgentSynchronizedActionSelection(java.lang.String agentName, State s) This method returns the action for a single agent by a synchonrized sampling of this joint policy, which enables multiple agents to query this policy object and act according to the same selected joint actions from it.
java.util.List<JointAction>	getAllJointActions(State s) Returns all possible joint actions that can be taken in state s for the set of agents defined to be used in this joint policy.
void	setAgentsInJointPolicy(java.util.List<SGAgent> agents) Sets the agent definitions by querying the agent names and SGAgentType objects from a list of agents.
void	setAgentsInJointPolicy(java.util.Map<java.lang.String,SGAgentType> agentsInJointPolicy) Sets the agent definitions that define the set of possible joint actions in each state.
void	setAgentsInJointPolicyFromWorld(World w) Sets teh agent definitions by querying the agents that exist in a World object.
abstract void	setTargetAgent(java.lang.String agentName) Sets the target privledged agent from which this joint policy is defined.

Methods inherited from class burlap.behavior.policy.Policy

evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateBehavior, evaluateMethodsShouldAnnotateOptionDecomposition, evaluateMethodsShouldDecomposeOption, followAndRecordPolicy, followAndRecordPolicy, getAction, getActionDistributionForState, getDeterministicPolicy, getProbOfAction, getProbOfActionGivenDistribution, getProbOfActionGivenDistribution, isDefinedFor, isStochastic, sampleFromActionDistribution

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

agentsInJointPolicy

protected java.util.Map<java.lang.String,SGAgentType> agentsInJointPolicy

The agent definitions that define the set of possible joint actions in each state.

lastSynchronizedJointAction

protected JointAction lastSynchronizedJointAction

The last synchronized joint action that was selected

agentsSyncrhonizedSoFar

protected java.util.Set<java.lang.String> agentsSyncrhonizedSoFar

The agents whose actiosn have been syncrhonized so far

lastSyncedState

protected State lastSyncedState

The last state in which synchronized actions were queried.

Constructor Detail

JointPolicy

public JointPolicy()

Method Detail

setAgentsInJointPolicy

public void setAgentsInJointPolicy(java.util.Map<java.lang.String,SGAgentType> agentsInJointPolicy)

Sets the agent definitions that define the set of possible joint actions in each state.

Parameters:

agentsInJointPolicy - the agent definitions that define the set of possible joint actions in each state.

setAgentsInJointPolicy

public void setAgentsInJointPolicy(java.util.List<SGAgent> agents)

Sets the agent definitions by querying the agent names and SGAgentType objects from a list of agents.

Parameters:

agents - the set of agents that will be involved in a joint aciton.

setAgentsInJointPolicyFromWorld

public void setAgentsInJointPolicyFromWorld(World w)

Sets teh agent definitions by querying the agents that exist in a World object.

Parameters:

w - the World object that contains the agents that will define the set of possible joint acitons.

getAllJointActions

public java.util.List<JointAction> getAllJointActions(State s)

Returns all possible joint actions that can be taken in state s for the set of agents defined to be used in this joint policy.

Parameters:

s - the state in which all joint actions should be returned.

Returns:

the set of all possible JointAction objects.

getAgentsInJointPolicy

public java.util.Map<java.lang.String,SGAgentType> getAgentsInJointPolicy()

Returns a map specifying the agents who contribute actions to this joint policy. The map goes from agent names to their agent type.

Returns:

a map specifying the agents who contribute actions to this joint policy

getAgentSynchronizedActionSelection

public AbstractGroundedAction getAgentSynchronizedActionSelection(java.lang.String agentName,
                                                         State s)

This method returns the action for a single agent by a synchonrized sampling of this joint policy, which enables multiple agents to query this policy object and act according to the same selected joint actions from it. This is useful when decisions are made from a "referee" who selects the joint action that dictates the behavior of each agent. The synchonization is implemented by selecting a joint action. Each time an agent queries for their action, it is drawn from the previously sampled joint action. A new joint action is only selected after each agent defined in this objects agentsInJointPolicy member has queried this method for their action or until an action for a different state is queried (that is, *either* condition will cause the joint action to be resampled).

Parameters:

agentName - the agent name whose action in this joint policy is being queried

s - the state in which the action is to be selected.

Returns:

the single agent action to be taken according to the synchonrized joint action that was selected.

setTargetAgent

public abstract void setTargetAgent(java.lang.String agentName)

Sets the target privledged agent from which this joint policy is defined.

Parameters:

agentName - the name of the target agent.

copy

public abstract JointPolicy copy()

Creates a copy of this joint policy and returns it. This is useful when generating different agents using the same kind of policy, but have different target agents evaluating it.

Returns:

a copy of this joint policy.