burlap.behavior.stochasticgames.agents.maql

Class MAQLFactory

java.lang.Object

burlap.behavior.stochasticgames.agents.maql.MAQLFactory

All Implemented Interfaces:

AgentFactory

Direct Known Subclasses:

MAQLFactory.CoCoQLearningFactory, MAQLFactory.MAMaxQLearningFactory

public class MAQLFactory
extends java.lang.Object
implements AgentFactory

This class provides a factory for MultiAgentQLearning agents. Subclasses for specific kinds of multi-agent Q-learning are also included. The policy given to this factory is always copied when generating a new agent to ensure that multiple agents generated from the same factory have unique policies tailored to their perspective.

Author:

James MacGlashan

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MAQLFactory.CoCoQLearningFactory Factory for generating CoCo-Q agents.
static class	MAQLFactory.MAMaxQLearningFactory Factory for generating Max multiagent Q-learning agents.

Field Summary

Fields

Modifier and Type	Field and Description
protected SGBackupOperator	backupOperator
protected double	discount
protected SGDomain	domain
protected HashableStateFactory	hashingFactory
protected PolicyFromJointPolicy	learningPolicy
protected LearningRate	learningRate
protected QFunction	qInit
protected boolean	queryOtherAgentsQSource

Constructor Summary

Constructors

Constructor and Description
MAQLFactory() Empty constructor.
MAQLFactory(SGDomain d, double discount, double learningRate, HashableStateFactory hashFactory, double qInit, SGBackupOperator backupOperator, boolean queryOtherAgentsForTheirQValues) Initializes.
MAQLFactory(SGDomain d, double discount, LearningRate learningRate, HashableStateFactory hashFactory, QFunction qInit, SGBackupOperator backupOperator, boolean queryOtherAgentsForTheirQValues, PolicyFromJointPolicy learningPolicy) Initializes.

Method Summary

Modifier and Type	Method and Description
SGAgent	generateAgent(java.lang.String agentName, SGAgentType type) Generates a new SGAgent
void	init(SGDomain d, double discount, LearningRate learningRate, HashableStateFactory hashFactory, QFunction qInit, SGBackupOperator backupOperator, boolean queryOtherAgentsForTheirQValues, PolicyFromJointPolicy learningPolicy) Initializes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

domain

protected SGDomain domain

discount

protected double discount

learningRate

protected LearningRate learningRate

qInit

protected QFunction qInit

hashingFactory

protected HashableStateFactory hashingFactory

backupOperator

protected SGBackupOperator backupOperator

learningPolicy

protected PolicyFromJointPolicy learningPolicy

queryOtherAgentsQSource

protected boolean queryOtherAgentsQSource

Constructor Detail

MAQLFactory

public MAQLFactory()

Empty constructor. All parameters will need to be set with the init(SGDomain, double, LearningRate, burlap.statehashing.HashableStateFactory, burlap.behavior.valuefunction.QFunction, SGBackupOperator, boolean, PolicyFromJointPolicy) function after construction.

MAQLFactory

public MAQLFactory(SGDomain d,
                   double discount,
                   double learningRate,
                   HashableStateFactory hashFactory,
                   double qInit,
                   SGBackupOperator backupOperator,
                   boolean queryOtherAgentsForTheirQValues)

Initializes. The policy will be defaulted to a epsilon-greedy max welfare policy.

Parameters:

d - the domain in which to perform learing

discount - the discount factor

learningRate - the constant learning rate

hashFactory - the hashing factory used to index states and Q-values

qInit - the default Q-value to which all initial Q-values will be initialized

backupOperator - the backup operator to use that defines the solution concept being learned

queryOtherAgentsForTheirQValues - it true, then the agent uses the Q-values for other agents that are stored by them; if false then the agent stores a Q-value for each other agent in the world.

MAQLFactory

public MAQLFactory(SGDomain d,
                   double discount,
                   LearningRate learningRate,
                   HashableStateFactory hashFactory,
                   QFunction qInit,
                   SGBackupOperator backupOperator,
                   boolean queryOtherAgentsForTheirQValues,
                   PolicyFromJointPolicy learningPolicy)

Initializes. The policy will be defaulted to a epsilon-greey max wellfare policy.

Parameters:

d - the domain in which to perform learing

discount - the discount factor

learningRate - the learning rate function

hashFactory - the hashing factory used to index states and Q-values

qInit - the Q-value initialization function

backupOperator - the backup operator to use that defines the solution concept being learned

queryOtherAgentsForTheirQValues - it true, then the agent uses the Q-values for other agents that are stored by them; if false then the agent stores a Q-value for each other agent in the world.

learningPolicy - the learningPolicy to follow

Method Detail

init

public void init(SGDomain d,
                 double discount,
                 LearningRate learningRate,
                 HashableStateFactory hashFactory,
                 QFunction qInit,
                 SGBackupOperator backupOperator,
                 boolean queryOtherAgentsForTheirQValues,
                 PolicyFromJointPolicy learningPolicy)

Initializes. The policy will be defaulted to a epsilon-greey max wellfare policy.

Parameters:

d - the domain in which to perform learing

discount - the discount factor

learningRate - the learning rate function

hashFactory - the hashing factory used to index states and Q-values

qInit - the Q-value initialization function

backupOperator - the backup operator to use that defines the solution concept being learned

queryOtherAgentsForTheirQValues - it true, then the agent uses the Q-values for other agents that are stored by them; if false then the agent stores a Q-value for each other agent in the world.

learningPolicy - the learningPolicy to follow

generateAgent

public SGAgent generateAgent(java.lang.String agentName,
                             SGAgentType type)

Description copied from interface: AgentFactory

Generates a new SGAgent

Specified by:

generateAgent in interface AgentFactory

Parameters:

agentName - the name for the agent

type - the SGAgentType for the agent

Returns:

a new SGAgent