public class LinearFVVFA extends java.lang.Object implements ValueFunctionApproximation
StateToFeatureVectorGenerator
which defines
the state features on which linear function approximation is performed. In the case of Q-value
function approximation, the state features are replicated for each action with all other action's associated
state features set to zero, thereby allowing for unique predictions for each action.
getStateValue(burlap.oomdp.core.State)
or getStateActionValues(burlap.oomdp.core.State, java.util.List)
is called first. If the former, then it performs state value function approximation; if the latter then Q-value
function approximation. Once it has been set for either state or Q-value function approximation, it cannot be used
for the other and will throw a runtime exception if it queried for the other kind of function.Modifier and Type | Field and Description |
---|---|
protected java.util.Map<GroundedAction,java.lang.Integer> |
actionOffset
A feature index offset for each action when using Q-value function approximation.
|
protected double |
defaultWeight
A default weight value for the functions weights.
|
protected StateToFeatureVectorGenerator |
fvGen
The state feature vector generator used for linear value function approximation.
|
protected FunctionWeight[] |
stateActionWeights
The function weights when performing Q-value function approximation.
|
protected FunctionWeight[] |
stateWeights
The function weights when performing state value function approximation.
|
Constructor and Description |
---|
LinearFVVFA(StateToFeatureVectorGenerator fvGen,
double defaultWeightValue)
Initializes.
|
Modifier and Type | Method and Description |
---|---|
protected void |
expandStateActionWeights(int num)
Expands the state-action function weight vector by a fixed sized and initializes their value
to the default weight value set for this object.
|
double |
getDefaultWeight() |
FunctionWeight |
getFunctionWeight(int featureId)
Returns the FunctionWeight for the given function's feature id.
|
StateToFeatureVectorGenerator |
getFvGen() |
java.util.List<ActionApproximationResult> |
getStateActionValues(State s,
java.util.List<GroundedAction> gas)
Returns a state-value (e.g., Q-value) approximation for the query state.
|
ApproximationResult |
getStateValue(State s)
Returns a state value approximation for the query state.
|
WeightGradient |
getWeightGradient(ApproximationResult approximationResult)
Returns the function weight gradient of the given approximation result.
|
int |
numFeatures()
Returns the number of features used in this approximator.
|
void |
resetWeights()
Resets the weights as is learning had never been performed.
|
void |
setWeight(int featureId,
double w)
Sets the weight for a features
|
protected StateToFeatureVectorGenerator fvGen
protected java.util.Map<GroundedAction,java.lang.Integer> actionOffset
protected FunctionWeight[] stateWeights
protected FunctionWeight[] stateActionWeights
protected double defaultWeight
public LinearFVVFA(StateToFeatureVectorGenerator fvGen, double defaultWeightValue)
getStateValue(burlap.oomdp.core.State)
or
getStateActionValues(burlap.oomdp.core.State, java.util.List)
is queried. If the former method is called,
first, then this object will be tasked with state value function approximation. If the latter
method is called first, then this object will be tasked with state-action value function approximation.fvGen
- The state feature vector generator that produces the features used for either linear state value function approximation or Q-value function approximation.defaultWeightValue
- The default weight value of all function weights.public StateToFeatureVectorGenerator getFvGen()
public double getDefaultWeight()
public ApproximationResult getStateValue(State s)
ValueFunctionApproximation
getStateValue
in interface ValueFunctionApproximation
s
- the query state whose state value should be approximatedpublic java.util.List<ActionApproximationResult> getStateActionValues(State s, java.util.List<GroundedAction> gas)
ValueFunctionApproximation
getStateActionValues
in interface ValueFunctionApproximation
s
- the query state of the state-action pair to be approximatedgas
- the query action of the state-action pair to be approximtedprotected void expandStateActionWeights(int num)
num
- the number of function weights to add to the state-action function weight vectorpublic WeightGradient getWeightGradient(ApproximationResult approximationResult)
ValueFunctionApproximation
getWeightGradient
in interface ValueFunctionApproximation
approximationResult
- the approximation result whose weight gradient should be returnedpublic void resetWeights()
ValueFunctionApproximation
resetWeights
in interface ValueFunctionApproximation
public void setWeight(int featureId, double w)
ValueFunctionApproximation
setWeight
in interface ValueFunctionApproximation
featureId
- the feature id whose weight should be setw
- the weight value to usepublic FunctionWeight getFunctionWeight(int featureId)
ValueFunctionApproximation
getFunctionWeight
in interface ValueFunctionApproximation
featureId
- the id of function's feature whose weight is returned.public int numFeatures()
ValueFunctionApproximation
numFeatures
in interface ValueFunctionApproximation