public class CMACFeatureDatabase extends java.lang.Object implements FeatureDatabase
FVCMACFeatureDatabase
implementation instead, which is slightly more efficient but object identifier dependent.
This CMAC can be set to define
multiple tilings, each specified for different sets attributes or different object classes. The multiple tilings
can be uniformly offset from each other, or randomly jittered to different alignments. The set of state
(or state-action) features stored are dynamically created as new states are introduced. That is, Tiling
objects procedurally determine which tile a state is in, but the associated state feature id for each state
feature is determined as needed.
Note that different tilings in a CMAC can be defined over different attributes and object classes. For instance
a CMAC can consist of two one dimensional tilings that are over different attibutes, such as one that tiles an
x position while another tiles the y position. Alternatively, a tiling may be multidimensional with different tilings
being defined over different setns of attributes. Having different tilings defined over different sets of attributes
enables the VFA to capture value function independence over different attributes.
1. Albus, James S. "A theory of cerebellar function." Mathematical Biosciences 10.1 (1971): 25-61Modifier and Type | Class and Description |
---|---|
static class |
CMACFeatureDatabase.TilingArrangement
Enum for specifying whether tilings should have their tile alignments should be chossen so that they
are randomly jittered from each other, or if each subsequent tiling should be offset by a uniform amount.
|
Modifier and Type | Field and Description |
---|---|
protected java.util.List<java.util.Map<Tiling.StateTile,burlap.behavior.singleagent.vfa.cmac.CMACFeatureDatabase.StoredFeaturesForTiling>> |
actionTilings
For each tiling, a map from state tiles to
CMACFeatureDatabase.StoredFeaturesForTiling objects, which contain
distinct state features for each action. |
protected CMACFeatureDatabase.TilingArrangement |
arrangement
Whether each tiling should have its offset alignment be randomly jittered or uniformly spaced.
|
protected State |
lastState |
protected int |
nextActionFeatureId
The identifier to use for the next state-action pair feature
|
protected int |
nextStateFeatureId
The identifier to use for the next state feature.
|
protected int |
nTilings
The number of tilings
|
protected java.util.Random |
rand
A random object for jittering the tile alignments.
|
protected java.util.List<java.util.Map<Tiling.StateTile,java.lang.Integer>> |
stateTilings
For each tiling, a map from state tiles to an integer representing their feature identifier
|
protected java.util.List<Tiling.StateTile> |
tilesForLastState |
protected java.util.List<Tiling> |
tilings
The set of tilings for producing state features
|
Constructor and Description |
---|
CMACFeatureDatabase(int nTilings,
CMACFeatureDatabase.TilingArrangement arrangement)
Initializes with a set of
nTilings and sets
the offset arrangement for subsequent tilings to be determined according to arrangement . |
Modifier and Type | Method and Description |
---|---|
void |
addSpecificaitonForTiling(int i,
java.lang.String className,
Attribute attribute,
double windowSize)
Causes the
i th tiling in this CMAC to be defined over the given attribute for the given OO-MDP class. |
void |
addSpecificationForAllTilings(java.lang.String className,
Attribute attribute,
double windowSize)
Causes all tilings in this CMAC to be defined over the given attribute for the given OO-MDP class.
|
CMACFeatureDatabase |
copy()
Returns a deep copy of this feature database.
|
void |
freezeDatabaseState(boolean toggle)
Enable or disable the database from generating new features.
|
LinearVFA |
generateVFA(double defaultWeightValue)
After all the tiling specifications have been set, this method can be called to produce a linear
VFA object.
|
protected java.util.List<ActionFeaturesQuery> |
getActionFeatureSetsFromCacheStateTiles(State s,
java.util.List<GroundedAction> actions) |
java.util.List<ActionFeaturesQuery> |
getActionFeaturesSets(State s,
java.util.List<GroundedAction> actions)
Returns non-zero action features for all of provided actions in state s.
|
java.util.List<StateFeature> |
getStateFeatures(State s)
Returns non-zero state features for a given state.
|
int |
numberOfFeatures()
Returns the number of features this database tracks.
|
protected int nTilings
protected CMACFeatureDatabase.TilingArrangement arrangement
protected java.util.Random rand
protected java.util.List<Tiling> tilings
protected java.util.List<java.util.Map<Tiling.StateTile,burlap.behavior.singleagent.vfa.cmac.CMACFeatureDatabase.StoredFeaturesForTiling>> actionTilings
CMACFeatureDatabase.StoredFeaturesForTiling
objects, which contain
distinct state features for each action. This is useful when doing approximation for state-action values.protected java.util.List<java.util.Map<Tiling.StateTile,java.lang.Integer>> stateTilings
protected int nextActionFeatureId
protected int nextStateFeatureId
protected State lastState
protected java.util.List<Tiling.StateTile> tilesForLastState
public CMACFeatureDatabase(int nTilings, CMACFeatureDatabase.TilingArrangement arrangement)
nTilings
and sets
the offset arrangement for subsequent tilings to be determined according to arrangement
.
The OO-MDP object classes attributes over which the tilings will be defined are unspecified at the start
and will need to be set using other methods on this object such as addSpecificationForAllTilings(String, Attribute, double)
or addSpecificaitonForTiling(int, String, Attribute, double)
.nTilings
- the number of tilings that will be created.arrangement
- either RANDOMJITTER or UNIFORM.public void addSpecificationForAllTilings(java.lang.String className, Attribute attribute, double windowSize)
windowSize
.className
- the OO-MDP class name for which the provided attribute will be tiled.attribute
- the OO-MDP attribute that will be tiledwindowSize
- the width of tilings over the specified attribute and OO-MDP class.public void addSpecificaitonForTiling(int i, java.lang.String className, Attribute attribute, double windowSize)
i
th tiling in this CMAC to be defined over the given attribute for the given OO-MDP class. Along that
dimension, the tiling will have a width of windowSize
.className
- the OO-MDP class name for which the provided attribute will be tiled.attribute
- the OO-MDP attribute that will be tiledwindowSize
- the width of tilings over the specified attribute and OO-MDP class.public LinearVFA generateVFA(double defaultWeightValue)
defaultWeightValue
- the default value weights for the CMAC features will use.public java.util.List<StateFeature> getStateFeatures(State s)
FeatureDatabase
getStateFeatures
in interface FeatureDatabase
s
- the state for which features should be returnedpublic java.util.List<ActionFeaturesQuery> getActionFeaturesSets(State s, java.util.List<GroundedAction> actions)
FeatureDatabase
getActionFeaturesSets
in interface FeatureDatabase
s
- the state for which features should be returnedactions
- the action set for which the action features should be returnedprotected java.util.List<ActionFeaturesQuery> getActionFeatureSetsFromCacheStateTiles(State s, java.util.List<GroundedAction> actions)
public void freezeDatabaseState(boolean toggle)
FeatureDatabase
freezeDatabaseState
in interface FeatureDatabase
public int numberOfFeatures()
FeatureDatabase
numberOfFeatures
in interface FeatureDatabase
public CMACFeatureDatabase copy()
FeatureDatabase
copy
in interface FeatureDatabase