burlap.behavior.functionapproximation.sparse

Class LinearVFA

java.lang.Object

burlap.behavior.functionapproximation.sparse.LinearVFA

All Implemented Interfaces:

DifferentiableStateActionValue, DifferentiableStateValue, ParametricFunction, ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

public class LinearVFA
extends java.lang.Object
implements DifferentiableStateValue, DifferentiableStateActionValue

This class is used for general purpose linear VFA. It only needs to be provided a SparseStateFeatures object that will be used to store retrieve state features. For every feature returned by the feature functions, this class will automatically create a weight associated with it. The returned approximated value for any state is the linear combination of state features and weights.

This class can also be used for state-action value function approximation, which it will perform by creating state-action features as a cross product of the state features and action set. You should only ever using this class for state-values or state-action values, not both.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction

ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<StateFeature>	currentFeatures
protected FunctionGradient	currentGradient
protected double	currentValue
protected double	defaultWeight A default weight for the functions
protected Action	lastAction
protected State	lastState
protected SparseStateFeatures	sparseStateFeatures The state features
protected SparseCrossProductFeatures	stateActionFeatures The State-action features based on the cross product of state features and actions
protected java.util.Map<java.lang.Integer,java.lang.Double>	weights A map from feature identifiers to function weights

Constructor Summary

Constructors

Constructor and Description
LinearVFA(SparseStateFeatures sparseStateFeatures) Initializes with a feature database; the default weight value will be zero
LinearVFA(SparseStateFeatures sparseStateFeatures, double defaultWeight) Initializes

Method Summary

Modifier and Type	Method and Description
LinearVFA	copy() Returns a copy of this ParametricFunction.
double	evaluate(State s) Sets the input of this function to the given State and returns the value of it.
double	evaluate(State s, Action a) Sets the input of this function to the given State and Action and returns the value of it.
double	getParameter(int i) Returns the value of the ith parameter value
protected double	getWeight(int weightId)
FunctionGradient	gradient(State s) Returns the gradient of this function
FunctionGradient	gradient(State s, Action a) Returns the gradient of this function.
int	numParameters() Returns the number of parameters defining this function.
void	resetParameters() Resets the parameters of this function to default values.
void	setParameter(int i, double p) Sets the value of the ith parameter to given value

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sparseStateFeatures

protected SparseStateFeatures sparseStateFeatures

The state features

stateActionFeatures

protected SparseCrossProductFeatures stateActionFeatures

The State-action features based on the cross product of state features and actions

weights

protected java.util.Map<java.lang.Integer,java.lang.Double> weights

A map from feature identifiers to function weights

defaultWeight

protected double defaultWeight

A default weight for the functions

currentFeatures

protected java.util.List<StateFeature> currentFeatures

currentValue

protected double currentValue

currentGradient

protected FunctionGradient currentGradient

lastState

protected State lastState

lastAction

protected Action lastAction

Constructor Detail

LinearVFA

public LinearVFA(SparseStateFeatures sparseStateFeatures)

Initializes with a feature database; the default weight value will be zero

Parameters:

sparseStateFeatures - the feature database to use

LinearVFA

public LinearVFA(SparseStateFeatures sparseStateFeatures,
                 double defaultWeight)

Initializes

Parameters:

sparseStateFeatures - the feature database to use

defaultWeight - the default feature weight to initialize feature weights to

Method Detail

evaluate

public double evaluate(State s,
                       Action a)

Description copied from interface: ParametricFunction.ParametricStateActionFunction

Sets the input of this function to the given State and Action and returns the value of it.

Specified by:

evaluate in interface ParametricFunction.ParametricStateActionFunction

Parameters:

s - the input State

a - the input action

Returns:

the value of this function evaluated on the State and Action

evaluate

public double evaluate(State s)

Description copied from interface: ParametricFunction.ParametricStateFunction

Sets the input of this function to the given State and returns the value of it.

Specified by:

evaluate in interface ParametricFunction.ParametricStateFunction

Parameters:

s - the State to input to the function

Returns:

the value of this function evaluated on the input State

gradient

public FunctionGradient gradient(State s)

Description copied from interface: DifferentiableStateValue

Returns the gradient of this function

Specified by:

gradient in interface DifferentiableStateValue

Parameters:

s - the input state

Returns:

the gradient

gradient

public FunctionGradient gradient(State s,
                                 Action a)

Description copied from interface: DifferentiableStateActionValue

Returns the gradient of this function.

Specified by:

gradient in interface DifferentiableStateActionValue

Parameters:

s - the input State

a - the input Action

Returns:

the FunctionGradient of this function at the input

numParameters

public int numParameters()

Description copied from interface: ParametricFunction

Returns the number of parameters defining this function. Note that some implementations my have a dynamic number of parameters that grows or shrinks over time. Consult the documentation for the specific implementation for more information.

Specified by:

numParameters in interface ParametricFunction

Returns:

the number of parameters defining this function.

getParameter

public double getParameter(int i)

Description copied from interface: ParametricFunction

Returns the value of the ith parameter value

Specified by:

getParameter in interface ParametricFunction

Parameters:

i - the parameter index

Returns:

the double value of the ith parameter

setParameter

public void setParameter(int i,
                         double p)

Description copied from interface: ParametricFunction

Sets the value of the ith parameter to given value

Specified by:

setParameter in interface ParametricFunction

Parameters:

i - the index of the parameter to set

p - the parameter value to which it should be set

getWeight

protected double getWeight(int weightId)

resetParameters

public void resetParameters()

Description copied from interface: ParametricFunction

Resets the parameters of this function to default values.

Specified by:

resetParameters in interface ParametricFunction

copy

public LinearVFA copy()

Description copied from interface: ParametricFunction

Returns a copy of this ParametricFunction.

Specified by:

copy in interface ParametricFunction

Returns:

a copy of this ParametricFunction.