burlap.behavior.functionapproximation.dense.fourier

## Class FourierBasis

• java.lang.Object
• burlap.behavior.functionapproximation.dense.fourier.FourierBasis
• All Implemented Interfaces:
DenseStateFeatures

```public class FourierBasis
extends java.lang.Object
implements DenseStateFeatures```
An implementation of Fourier Basis functions . This class expects a normalized state variable/feature vector of input states, if it is not normalized, behavior is not well defined. Therefore, you may want to consider using the `NormalizedVariableFeatures`. The higher order the basis functions, the higher the VFA resolution is. Typically, order n will produce (n+1)^d state basis functions (and a copy for each action), where d is the number of state variables. Since this grows quickly, a way to manage the complexity is to simplify the number of coefficient vectors. That is, each basis function is a function of the dot product of the input state variable vector and a coefficient vector {0...n}^d and normally all possible coefficient vectors (and their corresponding basis functions) for order n are produced. However, this class can be told to limit the permitted coefficient vectors to those that have no more than k non-zero entries in the coefficient vector. When k = 1, all features are treated as indepdent resulting in n*d basis functions.

When using a learning algorithm like `GradientDescentSarsaLam` with Fourier basis functions, it is typically a good idea to use the `FourierBasisLearningRateWrapper`, which scales the normal learning rate by the inverse of the norm of a basis function's coefficient vector.

1. G.D. Konidaris, S. Osentoski and P.S. Thomas. Value Function Approximation in Reinforcement Learning using the Fourier Basis. In Proceedings of the Twenty-Fifth Conference on Artificial Intelligence, pages 380-385, August 2011.

Author:
James MacGlashan
• ### Field Summary

Fields
Modifier and Type Field and Description
`protected java.util.Map<Action,java.lang.Integer>` `actionFeatureMultiplier`
A map for returning a multiplier to the number of state features for each action.
`protected java.util.List<short[]>` `coefficientVectors`
The coefficient vectors used
`protected DenseStateFeatures` `inputFeatures`
The input features over which fourier basis functions will be generated
`protected int` `maxNonZeroCoefficients`
The maximum number of non-zero coefficient entries permitted in a coefficient vector
`protected int` `nextActionMultiplier`
The next action Fourier basis function size multiplier to use for the next newly seen action.
`protected int` `numStateVariables`
The number of state variables on which the produced basis functions operate
`protected int` `order`
The order of the Fourier basis functions.
• ### Constructor Summary

Constructors
Constructor and Description
```FourierBasis(DenseStateFeatures inputFeatures, int order)```
Initializes.
```FourierBasis(DenseStateFeatures inputFeatures, int order, int maxNonZeroCoefficients)```
Initializes.
• ### Method Summary

All Methods
Modifier and Type Method and Description
`double` ```basisValue(double[] input, int basisFunction)```
Returns the basis function value for the given state input for the given basis function index.
`double` `coefficientNorm(int i)`
Returns the norm of the coefficient vector for the given basis function index.
`FourierBasis` `copy()`
Returns a copy of this `DenseStateFeatures`
`double[]` `features(State s)`
Returns a feature vector represented as a double array for a given input state.
`protected void` `generateCoefficientVectors()`
Generates all coefficient vectors given the number of state variables and the maximum number of non-zero coefficient element entries.
`protected void` ```generateCoefficientVectorsHelper(int index, short[] vector, int numNonZeroEntries)```
Recursive cofficient generator helper method.
`DenseLinearVFA` `generateVFA(double defaultWeightValue)`
Creates and returns a linear VFA object over this Fourier basis feature database.
`short[]` `getCoefficientVector(int i)`
Returns the coefficient vector for the given basis function index.
`void` `setCoefficientVectors(java.util.List<short[]> coefficientVectors)`
Forces the set of coefficient vectors (and thereby Fourier basis functions) used.
• ### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`
• ### Field Detail

• #### numStateVariables

`protected int numStateVariables`
The number of state variables on which the produced basis functions operate
• #### inputFeatures

`protected DenseStateFeatures inputFeatures`
The input features over which fourier basis functions will be generated
• #### coefficientVectors

`protected java.util.List<short[]> coefficientVectors`
The coefficient vectors used
• #### maxNonZeroCoefficients

`protected int maxNonZeroCoefficients`
The maximum number of non-zero coefficient entries permitted in a coefficient vector
• #### order

`protected int order`
The order of the Fourier basis functions.
• #### actionFeatureMultiplier

`protected java.util.Map<Action,java.lang.Integer> actionFeatureMultiplier`
A map for returning a multiplier to the number of state features for each action. Effectively this ensures a unique feature ID for each Fourier basis function for each action.
• #### nextActionMultiplier

`protected int nextActionMultiplier`
The next action Fourier basis function size multiplier to use for the next newly seen action.
• ### Constructor Detail

• #### FourierBasis

```public FourierBasis(DenseStateFeatures inputFeatures,
int order)```
Initializes. The coefficient vectors used by this Fourier Basis function will be generated lazily when the first features for an input state/state-action pair are queried. The maximum number of non-zero coefficient entries in a coefficient vector will be set to the maixmum (the state variable dimensionality).
Parameters:
`inputFeatures` - the state feature vector generator that turns OO-MDP `State` objects into double arrays.
`order` - the Fourier basis order
• #### FourierBasis

```public FourierBasis(DenseStateFeatures inputFeatures,
int order,
int maxNonZeroCoefficients)```
Initializes. The coefficient vectors used by this Fourier Basis function will be generated lazily when the first features for an input state are queried. Setting maxNonZeroCoefficents to one results in treating each state variable as independent and thereby producing order*d basis functions (for each action) where d is the number of state variables. Larger values of maxNonZeroCoefficents will result in more variable dependency combinations.
Parameters:
`inputFeatures` - The input features over which fourier basis functions will be generated
`order` - the fourier basis order
`maxNonZeroCoefficients` - the maximum number of entries in coefficient vectors that can have non-zero values.
• ### Method Detail

• #### setCoefficientVectors

`public void setCoefficientVectors(java.util.List<short[]> coefficientVectors)`
Forces the set of coefficient vectors (and thereby Fourier basis functions) used. Use this method only if you want to fine tune the basis functions used.
Parameters:
`coefficientVectors` - the coefficient vectors used to produce the Fourier basis functions.
• #### basisValue

```public double basisValue(double[] input,
int basisFunction)```
Returns the basis function value for the given state input for the given basis function index. The basisFunction index may be greater than the number of coefficient vectors because it may refer to an state-action feature's basis function (and there is a copy of each basis function for each action). The basis function used will be the one associated with the coefficient vector at index basisFunction % m, where m is the number of this object's coefficient vectors.
Parameters:
`input` - the state variables
`basisFunction` - the basis function index
Returns:
the value of the basis function for the given input state variables
• #### features

`public double[] features(State s)`
Description copied from interface: `DenseStateFeatures`
Returns a feature vector represented as a double array for a given input state.
Specified by:
`features` in interface `DenseStateFeatures`
Parameters:
`s` - the input state to turn into a feature vector.
Returns:
the feature vector represented as a double array.
• #### getCoefficientVector

`public short[] getCoefficientVector(int i)`
Returns the coefficient vector for the given basis function index. The basisFunction index may be greater than the number of coefficient vectors because it may refer to an state-action feature's basis function (and there is a copy of each basis function for each action). The coefficient vector returned is at the index basisFunction % m, where m is the number of this object's coefficient vectors.
Parameters:
`i` - the basis function index
Returns:
the coefficient vector for the given basis function
• #### coefficientNorm

`public double coefficientNorm(int i)`
Returns the norm of the coefficient vector for the given basis function index. The basisFunction index may be greater than the number of coefficient vectors because it may refer to an state-action feature's basis function (and there is a copy of each basis function for each action). The coefficient vector returned is at the index basisFunction % m, where m is the number of this object's coefficient vectors.
Parameters:
`i` - the basis function index
Returns:
the norm of the coefficient vector for the given basis function
• #### generateVFA

`public DenseLinearVFA generateVFA(double defaultWeightValue)`
Creates and returns a linear VFA object over this Fourier basis feature database.
Parameters:
`defaultWeightValue` - the default feature weight value to use for all features
Returns:
a linear VFA object over this Fourier basis feature database.
• #### generateCoefficientVectors

`protected void generateCoefficientVectors()`
Generates all coefficient vectors given the number of state variables and the maximum number of non-zero coefficient element entries.
• #### generateCoefficientVectorsHelper

```protected void generateCoefficientVectorsHelper(int index,
short[] vector,
int numNonZeroEntries)```
Recursive cofficient generator helper method. Once a permitted coefficient vector is fully generated, it is copied and added to this object's list of coefficient vectors.
Parameters:
`index` - the index into the coefficient vector that needs to have its values filled in.
`vector` - the coefficient vector generated thus far
`numNonZeroEntries` - the number of non-zero coefficient vector entires currently in the vector.
• #### copy

`public FourierBasis copy()`
Description copied from interface: `DenseStateFeatures`
Returns a copy of this `DenseStateFeatures`
Specified by:
`copy` in interface `DenseStateFeatures`
Returns:
a copy of this `DenseStateFeatures`