burlap.behavior.functionapproximation.sparse.tilecoding

Class TileCodingFeatures

java.lang.Object

burlap.behavior.functionapproximation.sparse.tilecoding.TileCodingFeatures

All Implemented Interfaces:

SparseStateFeatures

public class TileCodingFeatures
extends java.lang.Object
implements SparseStateFeatures

A feature database using CMACs [1] AKA Tiling Coding for states that are first converted into a feature vector. Because States are converted into a feature vector before tiling them.

Different tilings can be created over different dimensions of the converted state feature vector and different tiling widths for each dimension can be specified. Each tiling over the same dimensions can either be randomly jittered from each other or uniformly distributed across the space, which is specified using the TilingArrangement enumerator.

To specify the tiling used, use the addTilingsForAllDimensionsWithWidths(double[], int, TilingArrangement) or addTilingsForDimensionsAndWidths(boolean[], double[], int, TilingArrangement) method.

1. Albus, James S. "A theory of cerebellar function." Mathematical Biosciences 10.1 (1971): 25-61

Author:

James MacGlashan

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	TileCodingFeatures.ActionFeatureID A class for associating a Action with a feature id.

Field Summary

Fields

Modifier and Type	Field and Description
protected DenseStateFeatures	featureVectorGenerator The generator that turns OO-MDP state objects into state feature vectors.
protected int	nextStateFeatureId The identifier to use for the next state feature.
protected java.util.Random	rand A random object for jittering the tile alignments.

Constructor Summary

Constructors

Constructor and Description
TileCodingFeatures(DenseStateFeatures featureVectorGenerator) Initializes specifying the kind of state feature vector generator to use for turning OO-MDP states into feature vectors.

Method Summary

Modifier and Type	Method and Description
void	addTilingsForAllDimensionsWithWidths(double[] widths, int nTilings, TilingArrangement tileArrangement) Adss a number of tilings where each tile is dependent on *all* the dimensions of a state feature vector.
void	addTilingsForDimensionsAndWidths(boolean[] dimensionMask, double[] widths, int nTilings, TilingArrangement tileArrangement) Adss a number of tilings where each tile is dependent on the dimensions that are labeled as "true" in the dimensionMask parameter.
TileCodingFeatures	copy() Returns a deep copy of this feature database.
java.util.List<StateFeature>	features(State s) Returns non-zero state features for a given state.
LinearVFA	generateVFA(double defaultWeightValue) After all the tiling specifications have been set, this method can be called to produce a linear VFA object.
protected java.util.List<TileCodingFeatures.ActionFeatureID>	getOrGenerateActionFeatureList(java.util.Map<Tiling.FVTile,java.util.List<TileCodingFeatures.ActionFeatureID>> tileFeatureMap, Tiling.FVTile tile) Returns or creates stores and returns the list of action feature ids in the given map for the given tile.
protected int	getOrGenerateFeature(java.util.Map<Tiling.FVTile,java.lang.Integer> tileFeatureMap, Tiling.FVTile tile) Returns the stored feature id or creates, stores and returns one.
protected TileCodingFeatures.ActionFeatureID	matchingActionFeature(java.util.List<TileCodingFeatures.ActionFeatureID> actionFeatures, Action forAction) Returns the TileCodingFeatures.ActionFeatureID with an equivalent Action in the given list or null if there is none.
int	numFeatures() Returns the total number of features
protected double[]	produceRandomOffset(boolean[] dimensionMask, double[] widths) Creates and returns a random tiling offset for the given widths and required dimensions.
protected double[]	produceUniformTilingsOffset(boolean[] dimensionMask, double[] widths, int ithTiling, int nTilings) Creates and returns an offset that is uniformly spaced from other tilings.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

featureVectorGenerator

protected DenseStateFeatures featureVectorGenerator

The generator that turns OO-MDP state objects into state feature vectors.

rand

protected java.util.Random rand

A random object for jittering the tile alignments.

nextStateFeatureId

protected int nextStateFeatureId

The identifier to use for the next state feature.

Constructor Detail

TileCodingFeatures

public TileCodingFeatures(DenseStateFeatures featureVectorGenerator)

Initializes specifying the kind of state feature vector generator to use for turning OO-MDP states into feature vectors. The resulting feature vectors are what is tiled by this class.

Parameters:

featureVectorGenerator - the OO-MDP state to feature vector generator to use

Method Detail

copy

public TileCodingFeatures copy()

Description copied from interface: SparseStateFeatures

Returns a deep copy of this feature database. If the feature database is dynamic, then changes to one will not affect the other.

Specified by:

copy in interface SparseStateFeatures

Returns:

a deep copy of this feature database

addTilingsForDimensionsAndWidths

public void addTilingsForDimensionsAndWidths(boolean[] dimensionMask,
                                             double[] widths,
                                             int nTilings,
                                             TilingArrangement tileArrangement)

Adss a number of tilings where each tile is dependent on the dimensions that are labeled as "true" in the dimensionMask parameter. The widths parameter specifies the width of each tile along that given dimension. If tileArrangement is set to TilingArrangement.UNIFORM then each of the nTilings created with will be uniformly spaced across the width of each dimension. If it is set to TilingArrangement.RANDOM_JITTER then each tiling will be offset by a random amount.

Parameters:

dimensionMask - each true entry in this boolen array is a dimension over which the tiling will be defined.

widths - the width of tiles along each dimension. This value should be non-zero for each dimension unless the tiling doesn't depend on that dimension.

nTilings - the number of tilings over the specified dimensions to create

tileArrangement - whether the created tiles are uniformally spaced or randomly spaced.

addTilingsForAllDimensionsWithWidths

public void addTilingsForAllDimensionsWithWidths(double[] widths,
                                                 int nTilings,
                                                 TilingArrangement tileArrangement)

Adss a number of tilings where each tile is dependent on *all* the dimensions of a state feature vector. The widths parameter specifies the width of each tile along that given dimension. If tileArrangement is set to TilingArrangement.UNIFORM then each of the nTilings created with will be uniformly spaced across the width of each dimension. If it is set to TilingArrangement.RANDOM_JITTER then each tiling will be offset by a random amount.

Parameters:

widths - the width of tiles along each dimension. This value should be non-zero for each dimension .

nTilings - the number of tilings over the specified dimensions to create.

tileArrangement - whether the created tiles are uniformally spaced or randomly spaced.

features

public java.util.List<StateFeature> features(State s)

Description copied from interface: SparseStateFeatures

Returns non-zero state features for a given state.

Specified by:

features in interface SparseStateFeatures

Parameters:

s - the state for which features should be returned

Returns:

the features for state s

numFeatures

public int numFeatures()

Description copied from interface: SparseStateFeatures

Returns the total number of features

Specified by:

numFeatures in interface SparseStateFeatures

Returns:

the total number of features

getOrGenerateFeature

protected int getOrGenerateFeature(java.util.Map<Tiling.FVTile,java.lang.Integer> tileFeatureMap,
                                   Tiling.FVTile tile)

Returns the stored feature id or creates, stores and returns one. If a feature id is created, then the nextStateFeatureId data member of this object is incremented.

Parameters:

tileFeatureMap - the map from tiles to feature ids

tile - the tile for which a feature id is returned.

Returns:

the feature id for the tile.

getOrGenerateActionFeatureList

protected java.util.List<TileCodingFeatures.ActionFeatureID> getOrGenerateActionFeatureList(java.util.Map<Tiling.FVTile,java.util.List<TileCodingFeatures.ActionFeatureID>> tileFeatureMap,
                                                                                            Tiling.FVTile tile)

Returns or creates stores and returns the list of action feature ids in the given map for the given tile. If a new list is created, it will be empty and will need to action features added to it.

Parameters:

tileFeatureMap - the map from tiles for a state to the list of action features for that state

tile - the tile for which the list of action features is returned.

Returns:

the list of action features.

generateVFA

public LinearVFA generateVFA(double defaultWeightValue)

After all the tiling specifications have been set, this method can be called to produce a linear VFA object.

Parameters:

defaultWeightValue - the default value weights for the CMAC features will use.

Returns:

a linear ValueFunctionApproximation object that uses this feature database

produceRandomOffset

protected double[] produceRandomOffset(boolean[] dimensionMask,
                                       double[] widths)

Creates and returns a random tiling offset for the given widths and required dimensions.

Parameters:

dimensionMask - each true entry is a dimension on which the tiling depends

widths - the width of each dimension

Returns:

a random tiling offset.

produceUniformTilingsOffset

protected double[] produceUniformTilingsOffset(boolean[] dimensionMask,
                                               double[] widths,
                                               int ithTiling,
                                               int nTilings)

Creates and returns an offset that is uniformly spaced from other tilings.

Parameters:

dimensionMask - each true entry is a dimension on which the tiling depends

widths - widths the width of each dimension

ithTiling - which tiling of the nTilings for which this offset is to be generated

nTilings - the total number of tilings that will be uniformaly spaced

Returns:

an offset that is uniformly spaced from other tilings.

matchingActionFeature

protected TileCodingFeatures.ActionFeatureID matchingActionFeature(java.util.List<TileCodingFeatures.ActionFeatureID> actionFeatures,
                                                                   Action forAction)

Returns the TileCodingFeatures.ActionFeatureID with an equivalent Action in the given list or null if there is none.

Parameters:

actionFeatures - the list of TileCodingFeatures.ActionFeatureID objects to search.

forAction - the Action for which a match is to be found.

Returns:

the TileCodingFeatures.ActionFeatureID with an equivalent Action in the given list or null if there is none.