burlap.behavior.singleagent.vfa.common

Class LinearVFA

java.lang.Object

burlap.behavior.singleagent.vfa.common.LinearVFA

All Implemented Interfaces:

ValueFunctionApproximation

public class LinearVFA
extends java.lang.Object
implements ValueFunctionApproximation

This class is used for general purpose linear VFA. It only needs to be provided a FeatureDatabase object that will be used to store retrieve state features. For every feature returned by the feature database, this class will automatically create a weight associated with it. The returned approximated value for any state is the linear combination of state features and weights.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected double	defaultWeight A default weight for the functions
protected FeatureDatabase	featureDatabase A feature database for which a unique function weight will be associated
protected java.util.Map<java.lang.Integer,FunctionWeight>	weights A map from feature identifiers to function weights

Constructor Summary

Constructors

Constructor and Description
LinearVFA(FeatureDatabase featureDatabase) Initializes with a feature database; the default weight value will be zero
LinearVFA(FeatureDatabase featureDatabase, double defaultWeight) Initializes

Method Summary

Methods

Modifier and Type	Method and Description
protected ApproximationResult	getApproximationResultFrom(java.util.List<StateFeature> features) Computes the linear function over the given features and the stored feature weights.
FunctionWeight	getFunctionWeight(int featureId) Returns the FunctionWeight for the given function's feature id.
java.util.List<ActionApproximationResult>	getStateActionValues(State s, java.util.List<GroundedAction> gas) Returns a state-value (e.g., Q-value) approximation for the query state.
ApproximationResult	getStateValue(State s) Returns a state value approximation for the query state.
WeightGradient	getWeightGradient(ApproximationResult approximationResult) Returns the function weight gradient of the given approximation result.
int	numFeatures() Returns the number of features used in this approximator.
void	resetWeights() Resets the weights as is learning had never been performed.
void	setWeight(int featureId, double w) Sets the weight for a features

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

featureDatabase

protected FeatureDatabase featureDatabase

A feature database for which a unique function weight will be associated

weights

protected java.util.Map<java.lang.Integer,FunctionWeight> weights

A map from feature identifiers to function weights

defaultWeight

protected double defaultWeight

A default weight for the functions

Constructor Detail

LinearVFA

public LinearVFA(FeatureDatabase featureDatabase)

Initializes with a feature database; the default weight value will be zero

Parameters:

featureDatabase - the feature database to use

LinearVFA

public LinearVFA(FeatureDatabase featureDatabase,
         double defaultWeight)

Initializes

Parameters:

featureDatabase - the feature database to use

defaultWeight - the default feature weight to initialize feature weights to

Method Detail

getStateValue

public ApproximationResult getStateValue(State s)

Description copied from interface: ValueFunctionApproximation

Returns a state value approximation for the query state.

Specified by:

getStateValue in interface ValueFunctionApproximation

Parameters:

s - the query state whose state value should be approximated

Returns:

a state value approximation for the query state.

getStateActionValues

public java.util.List<ActionApproximationResult> getStateActionValues(State s,
                                                             java.util.List<GroundedAction> gas)

Description copied from interface: ValueFunctionApproximation

Returns a state-value (e.g., Q-value) approximation for the query state.

Specified by:

getStateActionValues in interface ValueFunctionApproximation

Parameters:

s - the query state of the state-action pair to be approximated

gas - the query action of the state-action pair to be approximted

Returns:

a state-value approximation for the query state.

getWeightGradient

public WeightGradient getWeightGradient(ApproximationResult approximationResult)

Description copied from interface: ValueFunctionApproximation

Returns the function weight gradient of the given approximation result.

Specified by:

getWeightGradient in interface ValueFunctionApproximation

Parameters:

approximationResult - the approximation result whose weight gradient should be returned

Returns:

the function weight gradient of the given approximation result.

getApproximationResultFrom

protected ApproximationResult getApproximationResultFrom(java.util.List<StateFeature> features)

Computes the linear function over the given features and the stored feature weights.

Parameters:

features - List of the StateFeature obejcts defining the state features of this approximator.

Returns:

the linear function over the given features and the stored feature weights stored in a ApproximationResult.

resetWeights

public void resetWeights()

Description copied from interface: ValueFunctionApproximation

Resets the weights as is learning had never been performed.

Specified by:

resetWeights in interface ValueFunctionApproximation

setWeight

public void setWeight(int featureId,
             double w)

Description copied from interface: ValueFunctionApproximation

Sets the weight for a features

Specified by:

setWeight in interface ValueFunctionApproximation

Parameters:

featureId - the feature id whose weight should be set

w - the weight value to use

numFeatures

public int numFeatures()

Description copied from interface: ValueFunctionApproximation

Returns the number of features used in this approximator. Note: if features are dynamically added with experience, this number may change with subsequent calls.

Specified by:

numFeatures in interface ValueFunctionApproximation

Returns:

the number of features used in this approximator.

getFunctionWeight

public FunctionWeight getFunctionWeight(int featureId)

Description copied from interface: ValueFunctionApproximation

Returns the FunctionWeight for the given function's feature id.

Specified by:

getFunctionWeight in interface ValueFunctionApproximation

Parameters:

featureId - the id of function's feature whose weight is returned.

Returns:

the FunctionWeight for the given function's feature id.