burlap.behavior.stochasticgames.madynamicprogramming

Class MADynamicProgramming.BackupBasedQSource

java.lang.Object

burlap.behavior.stochasticgames.madynamicprogramming.MADynamicProgramming.BackupBasedQSource

All Implemented Interfaces:

QSourceForSingleAgent

Enclosing class:

MADynamicProgramming

protected class MADynamicProgramming.BackupBasedQSource
extends java.lang.Object
implements QSourceForSingleAgent

A QSourceForSingleAgent implementation which stores a value function for an agent and produces Joint action Q-values by marginalizing over the transition dynamics the reward and discounted next state value.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.stochasticgames.madynamicprogramming.QSourceForSingleAgent

QSourceForSingleAgent.HashBackedQSource

Field Summary

Fields

Modifier and Type	Field and Description
protected int	agentNum The agent for which this value function is assigned.
protected java.util.Map<HashableState,java.lang.Double>	valueFunction The tabular value function

Constructor Summary

Constructors

Constructor and Description
BackupBasedQSource(int agentNum) Initializes a value function for the agent of the given name.

Method Summary

Modifier and Type	Method and Description
JAQValue	getQValueFor(State s, JointAction ja) Returns a Q-value (represented with a JAQValue object) stored for the given state and joint action.
double	getValue(HashableState sh) Returns the stored state value for hashed state sh, or creates an entry with an initial value corresponding to the MADynamicProgramming instance's value function initialization object and returns that value if the the quried state has never previously been indexed.
void	setValue(HashableState sh, double v) Sets the value of the state in this objects value function map.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

agentNum

protected int agentNum

The agent for which this value function is assigned.

valueFunction

protected java.util.Map<HashableState,java.lang.Double> valueFunction

The tabular value function

Constructor Detail

BackupBasedQSource

public BackupBasedQSource(int agentNum)

Initializes a value function for the agent of the given name.

Parameters:

agentNum - the name of the agent for which the value function corresponds.

Method Detail

getQValueFor

public JAQValue getQValueFor(State s,
                             JointAction ja)

Description copied from interface: QSourceForSingleAgent

Returns a Q-value (represented with a JAQValue object) stored for the given state and joint action. Modification to the returned object will directly modify the stored value that will be returned in subsequent calls to this method.

Specified by:

getQValueFor in interface QSourceForSingleAgent

Parameters:

s - the Q-value's associated state

ja - the Q-value's associated joint aciton

Returns:

a JAQValue for the given state and joint action.

getValue

public double getValue(HashableState sh)

Returns the stored state value for hashed state sh, or creates an entry with an initial value corresponding to the MADynamicProgramming instance's value function initialization object and returns that value if the the quried state has never previously been indexed.

Parameters:

sh - the state for which the value is to be returned.

Returns:

the value of the state.

setValue

public void setValue(HashableState sh,
                     double v)

Sets the value of the state in this objects value function map.

Parameters:

sh - the hashed state for which the value is to be set

v - the value to set the state to.