LinearFVVFA

java.lang.Object
- burlap.behavior.singleagent.vfa.common.LinearFVVFA

All Implemented Interfaces:

ValueFunctionApproximation
```
public class LinearFVVFA
extends java.lang.Object
implements ValueFunctionApproximation
```
This class can be used to perform linear value function approximation, either for a states or state-actions (Q-values). It takes as input a StateToFeatureVectorGenerator which defines the state features on which linear function approximation is performed. In the case of Q-value function approximation, the state features are replicated for each action with all other action's associated state features set to zero, thereby allowing for unique predictions for each action.

Objects of this class are set to use either state value function approximation or Q-value function approximation depending on whether the method getStateValue(burlap.oomdp.core.State) or getStateActionValues(burlap.oomdp.core.State, java.util.List) is called first. If the former, then it performs state value function approximation; if the latter then Q-value function approximation. Once it has been set for either state or Q-value function approximation, it cannot be used for the other and will throw a runtime exception if it queried for the other kind of function.

Author:

James MacGlashan.

Field Summary

Fields
Modifier and Type	Field and Description
`protected java.util.Map<GroundedAction,java.lang.Integer>`	`actionOffset` A feature index offset for each action when using Q-value function approximation.
`protected double`	`defaultWeight` A default weight value for the functions weights.
`protected StateToFeatureVectorGenerator`	`fvGen` The state feature vector generator used for linear value function approximation.
`protected FunctionWeight[]`	`stateActionWeights` The function weights when performing Q-value function approximation.
`protected FunctionWeight[]`	`stateWeights` The function weights when performing state value function approximation.

Constructor Summary

Constructors
Constructor and Description

LinearFVVFA(StateToFeatureVectorGenerator fvGen, double defaultWeightValue)
Initializes.

Constructors
Constructor and Description
`LinearFVVFA(StateToFeatureVectorGenerator fvGen, double defaultWeightValue)` Initializes.

Method Summary

Methods
Modifier and Type	Method and Description
`protected void`	`expandStateActionWeights(int num)` Expands the state-action function weight vector by a fixed sized and initializes their value to the default weight value set for this object.
`double`	`getDefaultWeight()`
`FunctionWeight`	`getFunctionWeight(int featureId)` Returns the FunctionWeight for the given function's feature id.
`StateToFeatureVectorGenerator`	`getFvGen()`
`java.util.List<ActionApproximationResult>`	`getStateActionValues(State s, java.util.List<GroundedAction> gas)` Returns a state-value (e.g., Q-value) approximation for the query state.
`ApproximationResult`	`getStateValue(State s)` Returns a state value approximation for the query state.
`WeightGradient`	`getWeightGradient(ApproximationResult approximationResult)` Returns the function weight gradient of the given approximation result.
`int`	`numFeatures()` Returns the number of features used in this approximator.
`void`	`resetWeights()` Resets the weights as is learning had never been performed.
`void`	`setWeight(int featureId, double w)` Sets the weight for a features

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - fvGen
```
protected StateToFeatureVectorGenerator fvGen
```
    The state feature vector generator used for linear value function approximation.
  - actionOffset
```
protected java.util.Map<GroundedAction,java.lang.Integer> actionOffset
```
    A feature index offset for each action when using Q-value function approximation.
  - stateWeights
```
protected FunctionWeight[] stateWeights
```
    The function weights when performing state value function approximation.
  - stateActionWeights
```
protected FunctionWeight[] stateActionWeights
```
    The function weights when performing Q-value function approximation.
  - defaultWeight
```
protected double defaultWeight
```
    A default weight value for the functions weights.
- Constructor Detail
  - LinearFVVFA
```
public LinearFVVFA(StateToFeatureVectorGenerator fvGen,
           double defaultWeightValue)
```
    Initializes. This object will be set to perform either state value function approximation or Q-value function approximation once a call to either getStateValue(burlap.oomdp.core.State) or getStateActionValues(burlap.oomdp.core.State, java.util.List) is queried. If the former method is called, first, then this object will be tasked with state value function approximation. If the latter method is called first, then this object will be tasked with state-action value function approximation.
    
    Parameters:
    fvGen - The state feature vector generator that produces the features used for either linear state value function approximation or Q-value function approximation.
    defaultWeightValue - The default weight value of all function weights.
- Method Detail
  - getFvGen
```
public StateToFeatureVectorGenerator getFvGen()
```
  - getDefaultWeight
```
public double getDefaultWeight()
```
  - getStateValue
```
public ApproximationResult getStateValue(State s)
```
    Description copied from interface: ValueFunctionApproximation
    
    Returns a state value approximation for the query state.
    
    Specified by:
    
    getStateValue in interface ValueFunctionApproximation
    
    Parameters:
    s - the query state whose state value should be approximated
    
    Returns:
    a state value approximation for the query state.
  - getStateActionValues
```
public java.util.List<ActionApproximationResult> getStateActionValues(State s,
                                                             java.util.List<GroundedAction> gas)
```
    Description copied from interface: ValueFunctionApproximation
    
    Returns a state-value (e.g., Q-value) approximation for the query state.
    
    Specified by:
    
    getStateActionValues in interface ValueFunctionApproximation
    
    Parameters:
    s - the query state of the state-action pair to be approximated
    gas - the query action of the state-action pair to be approximted
    
    Returns:
    a state-value approximation for the query state.
  - expandStateActionWeights
```
protected void expandStateActionWeights(int num)
```
    Expands the state-action function weight vector by a fixed sized and initializes their value to the default weight value set for this object.
    
    Parameters:
    num - the number of function weights to add to the state-action function weight vector
  - getWeightGradient
```
public WeightGradient getWeightGradient(ApproximationResult approximationResult)
```
    Description copied from interface: ValueFunctionApproximation
    
    Returns the function weight gradient of the given approximation result.
    
    Specified by:
    
    getWeightGradient in interface ValueFunctionApproximation
    
    Parameters:
    approximationResult - the approximation result whose weight gradient should be returned
    
    Returns:
    the function weight gradient of the given approximation result.
  - resetWeights
```
public void resetWeights()
```
    Description copied from interface: ValueFunctionApproximation
    
    Resets the weights as is learning had never been performed.
    
    Specified by:
    
    resetWeights in interface ValueFunctionApproximation
  - setWeight
```
public void setWeight(int featureId,
             double w)
```
    Description copied from interface: ValueFunctionApproximation
    
    Sets the weight for a features
    
    Specified by:
    
    setWeight in interface ValueFunctionApproximation
    
    Parameters:
    featureId - the feature id whose weight should be set
    w - the weight value to use
  - getFunctionWeight
```
public FunctionWeight getFunctionWeight(int featureId)
```
    Description copied from interface: ValueFunctionApproximation
    
    Returns the FunctionWeight for the given function's feature id.
    
    Specified by:
    
    getFunctionWeight in interface ValueFunctionApproximation
    
    Parameters:
    featureId - the id of function's feature whose weight is returned.
    
    Returns:
    the FunctionWeight for the given function's feature id.
  - numFeatures
```
public int numFeatures()
```
    Description copied from interface: ValueFunctionApproximation
    
    Returns the number of features used in this approximator. Note: if features are dynamically added with experience, this number may change with subsequent calls.
    
    Specified by:
    
    numFeatures in interface ValueFunctionApproximation
    
    Returns:
    the number of features used in this approximator.

Class LinearFVVFA

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

fvGen

actionOffset

stateWeights

stateActionWeights

defaultWeight

Constructor Detail

LinearFVVFA

Method Detail

getFvGen

getDefaultWeight

getStateValue

getStateActionValues

expandStateActionWeights

getWeightGradient

resetWeights

setWeight

getFunctionWeight

numFeatures