DenseLinearVFA

java.lang.Object
- burlap.behavior.functionapproximation.dense.DenseLinearVFA

All Implemented Interfaces:

DifferentiableStateActionValue, DifferentiableStateValue, ParametricFunction, ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction
```
public class DenseLinearVFA
extends java.lang.Object
implements DifferentiableStateValue, DifferentiableStateActionValue
```
This class can be used to perform linear value function approximation, either for a states or state-actions (Q-values). It takes as input a DenseStateFeatures which defines the state features on which linear function approximation is performed. In the case of Q-value function approximation, the state features are replicated for each action with all other action's associated state features set to zero, thereby allowing for unique predictions for each action.
This class can be used for either state-value functions or state-action-value functions, but only one of them. Which one is used is determined implicitly by whether the first function input is set with the evaluate(State) method or the evaluate(State, Action) method.

Author:

James MacGlashan.

Nested Class Summary
- Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction
  ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

Field Summary

Fields
Modifier and Type	Field and Description
`protected java.util.Map<Action,java.lang.Integer>`	`actionOffset` A feature index offset for each action when using Q-value function approximation.
`protected int`	`currentActionOffset`
`protected FunctionGradient`	`currentGradient`
`protected double[]`	`currentStateFeatures`
`protected double`	`currentValue`
`protected double`	`defaultWeight` A default weight value for the functions weights.
`protected State`	`lastState`
`protected double[]`	`stateActionWeights` The function weights when performing Q-value function approximation.
`protected DenseStateFeatures`	`stateFeatures` The state feature vector generator used for linear value function approximation.
`protected double[]`	`stateWeights` The function weights when performing state value function approximation.

Constructor Summary

Constructors
Constructor and Description

DenseLinearVFA(DenseStateFeatures stateFeatures, double defaultWeightValue)
Initializes.

Constructors
Constructor and Description
`DenseLinearVFA(DenseStateFeatures stateFeatures, double defaultWeightValue)` Initializes.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`DenseLinearVFA`	`copy()` Returns a copy of this `ParametricFunction`.
`double`	`evaluate(State s)` Sets the input of this function to the given `State` and returns the value of it.
`double`	`evaluate(State s, Action a)` Sets the input of this function to the given `State` and `Action` and returns the value of it.
`protected void`	`expandStateActionWeights(int num)` Expands the state-action function weight vector by a fixed sized and initializes their value to the default weight value set for this object.
`java.util.Map<Action,java.lang.Integer>`	`getActionOffset()` Returns the `Map` of feature index offsets into the full feature vector for each action
`int`	`getActionOffset(Action a)`
`double`	`getDefaultWeight()`
`double`	`getParameter(int i)` Returns the value of the ith parameter value
`DenseStateFeatures`	`getStateFeatures()`
`FunctionGradient`	`gradient(State s)` Returns the gradient of this function
`FunctionGradient`	`gradient(State s, Action a)` Returns the gradient of this function.
`void`	`initializeStateActionWeightVector(int size, double v)` Resets the state-action function weight array to a new array of the given sized and default value.
`void`	`initializeStateWeightVector(int size, double v)` Resets the state function weight array to a new array of the given sized and default value.
`int`	`numParameters()` Returns the number of parameters defining this function.
`void`	`resetParameters()` Resets the parameters of this function to default values.
`void`	`setActionOffset(Action a, int offset)` Sets the `Map` of feature index offset into the full feature vector for the given action
`void`	`setActionOffset(java.util.Map<Action,java.lang.Integer> actionOffset)` Sets the `Map` of feature index offsets into the full feature vector for each action
`void`	`setParameter(int i, double p)` Sets the value of the ith parameter to given value

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - stateFeatures
```
protected DenseStateFeatures stateFeatures
```
    The state feature vector generator used for linear value function approximation.
  - actionOffset
```
protected java.util.Map<Action,java.lang.Integer> actionOffset
```
    A feature index offset for each action when using Q-value function approximation.
  - stateWeights
```
protected double[] stateWeights
```
    The function weights when performing state value function approximation.
  - stateActionWeights
```
protected double[] stateActionWeights
```
    The function weights when performing Q-value function approximation.
  - defaultWeight
```
protected double defaultWeight
```
    A default weight value for the functions weights.
  - currentStateFeatures
```
protected double[] currentStateFeatures
```
  - currentActionOffset
```
protected int currentActionOffset
```
  - currentValue
```
protected double currentValue
```
  - currentGradient
```
protected FunctionGradient currentGradient
```
  - lastState
```
protected State lastState
```
- Constructor Detail
  - DenseLinearVFA
```
public DenseLinearVFA(DenseStateFeatures stateFeatures,
                      double defaultWeightValue)
```
    Initializes. This object will be set to perform either state value function approximation or state-action function approximation once a call to either evaluate(State) or evaluate(State, Action) is made. If the former method is called first, then this object will be tasked with state value function approximation. If the latter method is called first, then this object will be tasked with state-action value function approximation.
    
    Parameters:
    
    stateFeatures - The state feature vector generator that produces the features used for either linear state value function approximation or state-action-value function approximation.
    
    defaultWeightValue - The default weight value of all function weights.
- Method Detail
  - evaluate
```
public double evaluate(State s,
                       Action a)
```
    Description copied from interface: ParametricFunction.ParametricStateActionFunction
    
    Sets the input of this function to the given State and Action and returns the value of it.
    
    Specified by:
    
    evaluate in interface ParametricFunction.ParametricStateActionFunction
    
    Parameters:
    
    s - the input State
    
    a - the input action
    
    Returns:
    
    the value of this function evaluated on the State and Action
  - evaluate
```
public double evaluate(State s)
```
    Description copied from interface: ParametricFunction.ParametricStateFunction
    
    Sets the input of this function to the given State and returns the value of it.
    
    Specified by:
    
    evaluate in interface ParametricFunction.ParametricStateFunction
    
    Parameters:
    
    s - the State to input to the function
    
    Returns:
    
    the value of this function evaluated on the input State
  - gradient
```
public FunctionGradient gradient(State s)
```
    Description copied from interface: DifferentiableStateValue
    
    Returns the gradient of this function
    
    Specified by:
    
    gradient in interface DifferentiableStateValue
    
    Parameters:
    
    s - the input state
    
    Returns:
    
    the gradient
  - gradient
```
public FunctionGradient gradient(State s,
                                 Action a)
```
    Description copied from interface: DifferentiableStateActionValue
    
    Returns the gradient of this function.
    
    Specified by:
    
    gradient in interface DifferentiableStateActionValue
    
    Parameters:
    
    s - the input State
    
    a - the input Action
    
    Returns:
    
    the FunctionGradient of this function at the input
  - numParameters
```
public int numParameters()
```
    Description copied from interface: ParametricFunction
    
    Returns the number of parameters defining this function. Note that some implementations my have a dynamic number of parameters that grows or shrinks over time. Consult the documentation for the specific implementation for more information.
    
    Specified by:
    
    numParameters in interface ParametricFunction
    
    Returns:
    
    the number of parameters defining this function.
  - getParameter
```
public double getParameter(int i)
```
    Description copied from interface: ParametricFunction
    
    Returns the value of the ith parameter value
    
    Specified by:
    
    getParameter in interface ParametricFunction
    
    Parameters:
    
    i - the parameter index
    
    Returns:
    
    the double value of the ith parameter
  - setParameter
```
public void setParameter(int i,
                         double p)
```
    Description copied from interface: ParametricFunction
    
    Sets the value of the ith parameter to given value
    
    Specified by:
    
    setParameter in interface ParametricFunction
    
    Parameters:
    
    i - the index of the parameter to set
    
    p - the parameter value to which it should be set
  - resetParameters
```
public void resetParameters()
```
    Description copied from interface: ParametricFunction
    
    Resets the parameters of this function to default values.
    
    Specified by:
    
    resetParameters in interface ParametricFunction
  - getActionOffset
```
public int getActionOffset(Action a)
```
  - expandStateActionWeights
```
protected void expandStateActionWeights(int num)
```
    Expands the state-action function weight vector by a fixed sized and initializes their value to the default weight value set for this object.
    
    Parameters:
    
    num - the number of function weights to add to the state-action function weight vector
  - getStateFeatures
```
public DenseStateFeatures getStateFeatures()
```
  - getDefaultWeight
```
public double getDefaultWeight()
```
  - initializeStateWeightVector
```
public void initializeStateWeightVector(int size,
                                        double v)
```
    Resets the state function weight array to a new array of the given sized and default value.
    
    Parameters:
    
    size - the dimensionality of the weights
    
    v - the default value to which the weights will be set
  - initializeStateActionWeightVector
```
public void initializeStateActionWeightVector(int size,
                                              double v)
```
    Resets the state-action function weight array to a new array of the given sized and default value.
    
    Parameters:
    
    size - the dimensionality of the weights
    
    v - the default value to which the weights will be set
  - getActionOffset
```
public java.util.Map<Action,java.lang.Integer> getActionOffset()
```
    Returns the Map of feature index offsets into the full feature vector for each action
    
    Returns:
    
    the Map of feature index offsets into the full feature vector for each action
  - setActionOffset
```
public void setActionOffset(java.util.Map<Action,java.lang.Integer> actionOffset)
```
    Sets the Map of feature index offsets into the full feature vector for each action
    
    Parameters:
    
    actionOffset - the Map of feature index offsets into the full feature vector for each action
  - setActionOffset
```
public void setActionOffset(Action a,
                            int offset)
```
    Sets the Map of feature index offset into the full feature vector for the given action
    
    Parameters:
    
    a - the action whose feature vector index is to be set
    
    offset - the feature index offset for the action
  - copy
```
public DenseLinearVFA copy()
```
    Description copied from interface: ParametricFunction
    
    Returns a copy of this ParametricFunction.
    
    Specified by:
    
    copy in interface ParametricFunction
    
    Returns:
    
    a copy of this ParametricFunction.

Class DenseLinearVFA

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

stateFeatures

actionOffset

stateWeights

stateActionWeights

defaultWeight

currentStateFeatures

currentActionOffset

currentValue

currentGradient

lastState

Constructor Detail

DenseLinearVFA

Method Detail

evaluate

evaluate

gradient

gradient

numParameters

getParameter

setParameter

resetParameters

getActionOffset

expandStateActionWeights

getStateFeatures

getDefaultWeight

initializeStateWeightVector

initializeStateActionWeightVector

getActionOffset

setActionOffset

setActionOffset

copy