burlap.behavior.valuefunction

Class QFunction.QFunctionHelper

java.lang.Object

burlap.behavior.valuefunction.QFunction.QFunctionHelper

Enclosing interface:

QFunction

public static class QFunction.QFunctionHelper
extends java.lang.Object

A class of helper static methods that may be commonly used by code that uses a QFunction instance. In particular, methods for computing the value function of a state, given the Q-values (the max Q-value or policy weighted value).

Constructor Summary

Constructors

Constructor and Description
QFunction.QFunctionHelper()

Method Summary

Methods

Modifier and Type	Method and Description
static double	getOptimalValue(QFunction qSource, State s) Returns the optimal state value function for a state given a QFunction.
static double	getOptimalValue(QFunction qSource, State s, TerminalFunction tf) Returns the optimal state value for a state given a QFunction.
static double	getPolicyValue(QFunction qSource, State s, Policy p) Returns the state value under a given policy for a state and QFunction.
static double	getPolicyValue(QFunction qSource, State s, Policy p, TerminalFunction tf) Returns the state value under a given policy for a state and QFunction.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

QFunction.QFunctionHelper

public QFunction.QFunctionHelper()

Method Detail

getOptimalValue

public static double getOptimalValue(QFunction qSource,
                     State s)

Returns the optimal state value function for a state given a QFunction. The optimal value is the max Q-value. If no actions are permissible in the input state, then zero is returned.

Parameters:

qSource - the QFunction capable of producing Q-values.

s - the query State for which the value should be returned.

Returns:

the max Q-value for all possible Q-values in the state.

getOptimalValue

public static double getOptimalValue(QFunction qSource,
                     State s,
                     TerminalFunction tf)

Returns the optimal state value for a state given a QFunction. The optimal value is the max Q-value. If no actions are permissible in the input state or the input state is a terminal state, then zero is returned.

Parameters:

qSource - the QFunction capable of producing Q-values.

s - the query State for which the value should be returned.

tf - a terminal function.

Returns:

the max Q-value for all possible Q-values in the state or zero if there are not permissible actions or if the state is a terminal state.

getPolicyValue

public static double getPolicyValue(QFunction qSource,
                    State s,
                    Policy p)

Returns the state value under a given policy for a state and QFunction. The value is the expected Q-value under the input policy action distribution. If no actions are permissible in the input state, then zero is returned.

Parameters:

qSource - the QFunction capable of producing Q-values.

s - the query State for which the value should be returned.

p - the policy defining the action distribution.

Returns:

the expected Q-value under the input policy action distribution

getPolicyValue

public static double getPolicyValue(QFunction qSource,
                    State s,
                    Policy p,
                    TerminalFunction tf)

Returns the state value under a given policy for a state and QFunction. The value is the expected Q-value under the input policy action distribution. If no actions are permissible in the input state, then zero is returned.

Parameters:

qSource - the QFunction capable of producing Q-values.

s - the query State for which the value should be returned.

p - the policy defining the action distribution.

tf - a terminal function.

Returns:

the expected Q-value under the input policy action distribution or zero if there are not permissible actions or if the state is a terminal state.