LinearStateActionDifferentiableRF

java.lang.Object
- burlap.behavior.singleagent.learnfromdemo.mlirl.commonrfs.LinearStateActionDifferentiableRF

All Implemented Interfaces:

ParametricFunction, DifferentiableRF, RewardFunction
```
public class LinearStateActionDifferentiableRF
extends java.lang.Object
implements DifferentiableRF
```
A class for defining a state-action linear DifferentiableRF. The class takes as input a DenseStateFeatures and the set of possible grounded actions that can be applied in the world. The dimensionality of this reward function is equal to |A|*|f|, where A is the set of possible grounded actions, and |f| is the state feature vector dimensionality.
The reward function is defined as R(s, a, s') = w(a) * f(s), where w(a) is the set of weights (the parameters) of this reward functions associated with action a, * is the dot product operator, and f(s) is the feature vector for state s.
Note that when the gradient is a vector of size |A||f|, since the feature vector is replicated for each action, and the gradient for all entries associated with an action other than the one taken in the (s, a, s') query will have a gradient value of zero.
The set of possible grounded actions must be defined either in the LinearStateActionDifferentiableRF(DenseStateFeatures, int, Action...) constructor, or added iteratively with the addAction(burlap.mdp.core.action.Action) method.

Author:

James MacGlashan.

Nested Class Summary
- Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction
  ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

Field Summary

Fields
Modifier and Type	Field and Description
`protected java.util.Map<Action,java.lang.Integer>`	`actionMap` An ordering of grounded actions
`protected int`	`dim` The dimension of this reward function
`protected DenseStateFeatures`	`fvGen` The state feature vector generator to use
`protected int`	`numStateFeatures` The number of state features
`protected double[]`	`parameters` The parameters of this reward function

Constructor Summary

Constructors
Constructor and Description
`LinearStateActionDifferentiableRF(DenseStateFeatures stateFeatures, int numStateFeatures, Action... allPossibleActions)` Initializes.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addAction(Action ga)` Adds a possible grounded action.
`ParametricFunction`	`copy()` Returns a copy of this `ParametricFunction`.
`double`	`getParameter(int i)` Returns the value of the ith parameter value
`FunctionGradient`	`gradient(State s, Action a, State sprime)`
`int`	`numParameters()` Returns the number of parameters defining this function.
`void`	`resetParameters()` Resets the parameters of this function to default values.
`double`	`reward(State s, Action a, State sprime)` Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
`void`	`setParameter(int i, double p)` Sets the value of the ith parameter to given value
`java.lang.String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - actionMap
```
protected java.util.Map<Action,java.lang.Integer> actionMap
```
    An ordering of grounded actions
  - parameters
```
protected double[] parameters
```
    The parameters of this reward function
  - dim
```
protected int dim
```
    The dimension of this reward function
  - fvGen
```
protected DenseStateFeatures fvGen
```
    The state feature vector generator to use
  - numStateFeatures
```
protected int numStateFeatures
```
    The number of state features
- Constructor Detail
  - LinearStateActionDifferentiableRF
```
public LinearStateActionDifferentiableRF(DenseStateFeatures stateFeatures,
                                         int numStateFeatures,
                                         Action... allPossibleActions)
```
    Initializes. If not all possible grounded actions are provided, they can be defined/added later with the addAction(Action) method.
    
    Parameters:
    
    stateFeatures - the state feature vector generator
    
    numStateFeatures - the dimensionality of the state feature vector
    
    allPossibleActions - the set of possible grounded actions.
- Method Detail
  - addAction
```
public void addAction(Action ga)
```
    Adds a possible grounded action. This addition increases the dimensionality of this reward function by |f| where |f| is the dimensionality of the state feature vector.
    
    Parameters:
    
    ga - the possible grounded action to add to this reward function's definition.
  - reward
```
public double reward(State s,
                     Action a,
                     State sprime)
```
    Description copied from interface: RewardFunction
    
    Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
    
    Specified by:
    
    reward in interface RewardFunction
    
    Parameters:
    
    s - the state in which the action was executed
    
    a - the action executed
    
    sprime - the state to which the agent transitioned
    
    Returns:
    
    the reward received when action a is executed in state s and the agent transitions to state sprime.
  - gradient
```
public FunctionGradient gradient(State s,
                                 Action a,
                                 State sprime)
```
    Specified by:
    
    gradient in interface DifferentiableRF
  - numParameters
```
public int numParameters()
```
    Description copied from interface: ParametricFunction
    
    Returns the number of parameters defining this function. Note that some implementations my have a dynamic number of parameters that grows or shrinks over time. Consult the documentation for the specific implementation for more information.
    
    Specified by:
    
    numParameters in interface ParametricFunction
    
    Returns:
    
    the number of parameters defining this function.
  - getParameter
```
public double getParameter(int i)
```
    Description copied from interface: ParametricFunction
    
    Returns the value of the ith parameter value
    
    Specified by:
    
    getParameter in interface ParametricFunction
    
    Parameters:
    
    i - the parameter index
    
    Returns:
    
    the double value of the ith parameter
  - setParameter
```
public void setParameter(int i,
                         double p)
```
    Description copied from interface: ParametricFunction
    
    Sets the value of the ith parameter to given value
    
    Specified by:
    
    setParameter in interface ParametricFunction
    
    Parameters:
    
    i - the index of the parameter to set
    
    p - the parameter value to which it should be set
  - resetParameters
```
public void resetParameters()
```
    Description copied from interface: ParametricFunction
    
    Resets the parameters of this function to default values.
    
    Specified by:
    
    resetParameters in interface ParametricFunction
  - copy
```
public ParametricFunction copy()
```
    Description copied from interface: ParametricFunction
    
    Returns a copy of this ParametricFunction.
    
    Specified by:
    
    copy in interface ParametricFunction
    
    Returns:
    
    a copy of this ParametricFunction.
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object

Class LinearStateActionDifferentiableRF

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

actionMap

parameters

dim

fvGen

numStateFeatures

Constructor Detail

LinearStateActionDifferentiableRF

Method Detail

addAction

reward

gradient

numParameters

getParameter

setParameter

resetParameters

copy

toString