LinearStateDifferentiableRF

java.lang.Object
- burlap.behavior.singleagent.learnfromdemo.mlirl.commonrfs.LinearStateDifferentiableRF

All Implemented Interfaces:

ParametricFunction, DifferentiableRF, RewardFunction
```
public class LinearStateDifferentiableRF
extends java.lang.Object
implements DifferentiableRF
```
A class for defining a linear state DifferentiableRF. The features of the reward function are produced by a DenseStateFeatures. By default, the reward function is defined as: R(s, a, s') = w * f(s'), where w is the weight vector (the parameters) of this object, * is the dot product operator, and f(s') is the feature vector for state s'. Alternatively, the reward function may be defined R(s, a, s') = w * f(s), (that is, using the feature vector for the previous state) by using the LinearStateDifferentiableRF(DenseStateFeatures, int, boolean) constructor or the setFeaturesAreForNextState(boolean)} method and setting the featuresAreForNextState boolean to false.

Author:

James MacGlashan.

Nested Class Summary
- Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction
  ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

Field Summary

Fields
Modifier and Type	Field and Description
`protected int`	`dim` The dimension of this reward function
`protected boolean`	`featuresAreForNextState` Whether features are based on the next state or previous state.
`protected DenseStateFeatures`	`fvGen` The state feature vector generator.
`protected double[]`	`parameters` The parameters of this reward function

Constructor Summary

Constructors
Constructor and Description
`LinearStateDifferentiableRF(DenseStateFeatures fvGen, int dim)` Initializes.
`LinearStateDifferentiableRF(DenseStateFeatures fvGen, int dim, boolean featuresAreForNextState)` Initializes.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`ParametricFunction`	`copy()` Returns a copy of this `ParametricFunction`.
`double`	`getParameter(int i)` Returns the value of the ith parameter value
`FunctionGradient`	`gradient(State s, Action a, State sprime)`
`int`	`numParameters()` Returns the number of parameters defining this function.
`void`	`resetParameters()` Resets the parameters of this function to default values.
`double`	`reward(State s, Action a, State sprime)` Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
`void`	`setFeaturesAreForNextState(boolean featuresAreForNextState)` Sets whether features for the reward function are generated from the next state or previous state.
`void`	`setParameter(int i, double p)` Sets the value of the ith parameter to given value
`java.lang.String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - featuresAreForNextState
```
protected boolean featuresAreForNextState
```
    Whether features are based on the next state or previous state. Default is for the next state (true).
  - fvGen
```
protected DenseStateFeatures fvGen
```
    The state feature vector generator.
  - parameters
```
protected double[] parameters
```
    The parameters of this reward function
  - dim
```
protected int dim
```
    The dimension of this reward function
- Constructor Detail
  - LinearStateDifferentiableRF
```
public LinearStateDifferentiableRF(DenseStateFeatures fvGen,
                                   int dim)
```
    Initializes. The reward function will use the features for the next state.
    
    Parameters:
    
    fvGen - the state feature vector generator
    
    dim - the dimensionality of the state features that will be produced
  - LinearStateDifferentiableRF
```
public LinearStateDifferentiableRF(DenseStateFeatures fvGen,
                                   int dim,
                                   boolean featuresAreForNextState)
```
    Initializes.
    
    Parameters:
    
    fvGen - the state feature vector generator
    
    dim - the dimensionality of the state features that will be produced
    
    featuresAreForNextState - If true, then the features will be generated from the next state in the (s, a, s') transition. If false, then the previous state.
- Method Detail
  - setFeaturesAreForNextState
```
public void setFeaturesAreForNextState(boolean featuresAreForNextState)
```
    Sets whether features for the reward function are generated from the next state or previous state.
    
    Parameters:
    
    featuresAreForNextState - If true, then the features will be generated from the next state in the (s, a, s') transition. If false, then the previous state.
  - gradient
```
public FunctionGradient gradient(State s,
                                 Action a,
                                 State sprime)
```
    Specified by:
    
    gradient in interface DifferentiableRF
  - numParameters
```
public int numParameters()
```
    Description copied from interface: ParametricFunction
    
    Returns the number of parameters defining this function. Note that some implementations my have a dynamic number of parameters that grows or shrinks over time. Consult the documentation for the specific implementation for more information.
    
    Specified by:
    
    numParameters in interface ParametricFunction
    
    Returns:
    
    the number of parameters defining this function.
  - getParameter
```
public double getParameter(int i)
```
    Description copied from interface: ParametricFunction
    
    Returns the value of the ith parameter value
    
    Specified by:
    
    getParameter in interface ParametricFunction
    
    Parameters:
    
    i - the parameter index
    
    Returns:
    
    the double value of the ith parameter
  - setParameter
```
public void setParameter(int i,
                         double p)
```
    Description copied from interface: ParametricFunction
    
    Sets the value of the ith parameter to given value
    
    Specified by:
    
    setParameter in interface ParametricFunction
    
    Parameters:
    
    i - the index of the parameter to set
    
    p - the parameter value to which it should be set
  - resetParameters
```
public void resetParameters()
```
    Description copied from interface: ParametricFunction
    
    Resets the parameters of this function to default values.
    
    Specified by:
    
    resetParameters in interface ParametricFunction
  - copy
```
public ParametricFunction copy()
```
    Description copied from interface: ParametricFunction
    
    Returns a copy of this ParametricFunction.
    
    Specified by:
    
    copy in interface ParametricFunction
    
    Returns:
    
    a copy of this ParametricFunction.
  - reward
```
public double reward(State s,
                     Action a,
                     State sprime)
```
    Description copied from interface: RewardFunction
    
    Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
    
    Specified by:
    
    reward in interface RewardFunction
    
    Parameters:
    
    s - the state in which the action was executed
    
    a - the action executed
    
    sprime - the state to which the agent transitioned
    
    Returns:
    
    the reward received when action a is executed in state s and the agent transitions to state sprime.
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object

Class LinearStateDifferentiableRF

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

featuresAreForNextState

fvGen

parameters

dim

Constructor Detail

LinearStateDifferentiableRF

LinearStateDifferentiableRF

Method Detail

setFeaturesAreForNextState

gradient

numParameters

getParameter

setParameter

resetParameters

copy

reward

toString