LinearDiffRFVInit

java.lang.Object
- burlap.behavior.singleagent.learnfromdemo.mlirl.differentiableplanners.diffvinit.LinearDiffRFVInit

All Implemented Interfaces:

ParametricFunction, DifferentiableVInit, DifferentiableRF, DifferentiableValueFunction, ValueFunction, RewardFunction
```
public class LinearDiffRFVInit
extends java.lang.Object
implements DifferentiableVInit, DifferentiableRF
```
A class for creating a DifferentiableRF and a DifferentiableVInit when the reward function and value function initialization are linear functions over some set of features. The total parameter dimensionality will be the sum of the reward function feature dimension and value function initialization feature dimension.
This class is useful when learning both a reward function and the shaping values at the leaf nodes of a finite horizon valueFunction.

Author:

James MacGlashan.

Nested Class Summary
- Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction
  ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

Field Summary

Fields
Modifier and Type	Field and Description
`protected int`	`dim`
`protected double[]`	`parameters`
`protected int`	`rfDim` The dimensionality of the reward function parameters
`protected boolean`	`rfFeaturesAreForNextState` Whether features are based on the next state or previous state.
`protected DenseStateFeatures`	`rfFvGen` The state feature vector generator.
`protected int`	`vinitDim` The dimensionality of the value function initialization parameters
`protected DenseStateFeatures`	`vinitFvGen` The state feature vector generator.

Constructor Summary

Constructors
Constructor and Description
`LinearDiffRFVInit(DenseStateFeatures rfFvGen, DenseStateFeatures vinitFvGen, int rfDim, int vinitDim)` Initializes a linear reward function for a given feature vector of a given dimension and linear value function initialization for a given feature vector and set of dimensions.
`LinearDiffRFVInit(DenseStateFeatures rfFvGen, DenseStateFeatures vinitFvGen, int rfDim, int vinitDim, boolean rfFeaturesAreForNextState)` Initializes a linear reward function for a given feature vector of a given dimension and linear value function initialization for a given feature vector and set of dimensions.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`ParametricFunction`	`copy()` Returns a copy of this `ParametricFunction`.
`double`	`getParameter(int i)` Returns the value of the ith parameter value
`int`	`getRfDim()`
`DenseStateFeatures`	`getRfFvGen()`
`int`	`getVinitDim()`
`DenseStateFeatures`	`getVinitFvGen()`
`FunctionGradient`	`gradient(State s, Action a, State sp)`
`boolean`	`isRfFeaturesAreForNextState()` Returns whether the reward function state features are evaluated on the next state of the transition (s' of R(s,a,s')) or the previous state of the transition (s of R(s,a,s'))
`int`	`numParameters()` Returns the number of parameters defining this function.
`void`	`resetParameters()` Resets the parameters of this function to default values.
`double`	`reward(State s, Action a, State sprime)` Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
`void`	`setParameter(int i, double p)` Sets the value of the ith parameter to given value
`void`	`setRfDim(int rfDim)`
`void`	`setRfFeaturesAreForNextState(boolean rfFeaturesAreForNextState)`
`void`	`setRfFvGen(DenseStateFeatures rfFvGen)`
`void`	`setVinitDim(int vinitDim)`
`void`	`setVinitFvGen(DenseStateFeatures vinitFvGen)`
`double`	`value(State s)` Returns the value function evaluation of the given state.
`FunctionGradient`	`valueGradient(State s)` Returns the gradient of this value function

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - rfFeaturesAreForNextState
```
protected boolean rfFeaturesAreForNextState
```
    Whether features are based on the next state or previous state. Default is for the next state (true).
  - rfFvGen
```
protected DenseStateFeatures rfFvGen
```
    The state feature vector generator.
  - vinitFvGen
```
protected DenseStateFeatures vinitFvGen
```
    The state feature vector generator.
  - rfDim
```
protected int rfDim
```
    The dimensionality of the reward function parameters
  - vinitDim
```
protected int vinitDim
```
    The dimensionality of the value function initialization parameters
  - parameters
```
protected double[] parameters
```
  - dim
```
protected int dim
```
- Constructor Detail
  - LinearDiffRFVInit
```
public LinearDiffRFVInit(DenseStateFeatures rfFvGen,
                         DenseStateFeatures vinitFvGen,
                         int rfDim,
                         int vinitDim)
```
    Initializes a linear reward function for a given feature vector of a given dimension and linear value function initialization for a given feature vector and set of dimensions.
    
    Parameters:
    
    rfFvGen - the reward function feature vector generator
    
    vinitFvGen - the value function initialization feature vector generator
    
    rfDim - the reward function feature/parameter dimensionality
    
    vinitDim - the value function initialization feature/parameter dimensionality
  - LinearDiffRFVInit
```
public LinearDiffRFVInit(DenseStateFeatures rfFvGen,
                         DenseStateFeatures vinitFvGen,
                         int rfDim,
                         int vinitDim,
                         boolean rfFeaturesAreForNextState)
```
    Initializes a linear reward function for a given feature vector of a given dimension and linear value function initialization for a given feature vector and set of dimensions.
    
    Parameters:
    
    rfFvGen - the reward function feature vector generator
    
    vinitFvGen - the value function initialization feature vector generator
    
    rfDim - the reward function feature/parameter dimensionality
    
    vinitDim - the value function initialization feature/parameter dimensionality
    
    rfFeaturesAreForNextState - if true, the the rf features are evaluated on the next state of the transition; if false then on the previous state of the transition.
- Method Detail
  - isRfFeaturesAreForNextState
```
public boolean isRfFeaturesAreForNextState()
```
    Returns whether the reward function state features are evaluated on the next state of the transition (s' of R(s,a,s')) or the previous state of the transition (s of R(s,a,s'))
    
    Returns:
    
    True if features are evaluated on the next state; false if they are evaluated on the previous state.
  - setRfFeaturesAreForNextState
```
public void setRfFeaturesAreForNextState(boolean rfFeaturesAreForNextState)
```
  - getRfFvGen
```
public DenseStateFeatures getRfFvGen()
```
  - setRfFvGen
```
public void setRfFvGen(DenseStateFeatures rfFvGen)
```
  - getVinitFvGen
```
public DenseStateFeatures getVinitFvGen()
```
  - setVinitFvGen
```
public void setVinitFvGen(DenseStateFeatures vinitFvGen)
```
  - getRfDim
```
public int getRfDim()
```
  - setRfDim
```
public void setRfDim(int rfDim)
```
  - getVinitDim
```
public int getVinitDim()
```
  - setVinitDim
```
public void setVinitDim(int vinitDim)
```
  - gradient
```
public FunctionGradient gradient(State s,
                                 Action a,
                                 State sp)
```
    Specified by:
    
    gradient in interface DifferentiableRF
  - reward
```
public double reward(State s,
                     Action a,
                     State sprime)
```
    Description copied from interface: RewardFunction
    
    Returns the reward received when action a is executed in state s and the agent transitions to state sprime.
    
    Specified by:
    
    reward in interface RewardFunction
    
    Parameters:
    
    s - the state in which the action was executed
    
    a - the action executed
    
    sprime - the state to which the agent transitioned
    
    Returns:
    
    the reward received when action a is executed in state s and the agent transitions to state sprime.
  - valueGradient
```
public FunctionGradient valueGradient(State s)
```
    Description copied from interface: DifferentiableValueFunction
    
    Returns the gradient of this value function
    
    Specified by:
    
    valueGradient in interface DifferentiableValueFunction
    
    Parameters:
    
    s - the state on which the function is to be evaluated
    
    Returns:
    
    the gradient of this value function
  - value
```
public double value(State s)
```
    Description copied from interface: ValueFunction
    
    Returns the value function evaluation of the given state. If the value is not stored, then the default value specified by the ValueFunctionInitialization object of this class is returned.
    
    Specified by:
    
    value in interface ValueFunction
    
    Parameters:
    
    s - the state to evaluate.
    
    Returns:
    
    the value function evaluation of the given state.
  - numParameters
```
public int numParameters()
```
    Description copied from interface: ParametricFunction
    
    Returns the number of parameters defining this function. Note that some implementations my have a dynamic number of parameters that grows or shrinks over time. Consult the documentation for the specific implementation for more information.
    
    Specified by:
    
    numParameters in interface ParametricFunction
    
    Returns:
    
    the number of parameters defining this function.
  - getParameter
```
public double getParameter(int i)
```
    Description copied from interface: ParametricFunction
    
    Returns the value of the ith parameter value
    
    Specified by:
    
    getParameter in interface ParametricFunction
    
    Parameters:
    
    i - the parameter index
    
    Returns:
    
    the double value of the ith parameter
  - setParameter
```
public void setParameter(int i,
                         double p)
```
    Description copied from interface: ParametricFunction
    
    Sets the value of the ith parameter to given value
    
    Specified by:
    
    setParameter in interface ParametricFunction
    
    Parameters:
    
    i - the index of the parameter to set
    
    p - the parameter value to which it should be set
  - resetParameters
```
public void resetParameters()
```
    Description copied from interface: ParametricFunction
    
    Resets the parameters of this function to default values.
    
    Specified by:
    
    resetParameters in interface ParametricFunction
  - copy
```
public ParametricFunction copy()
```
    Description copied from interface: ParametricFunction
    
    Returns a copy of this ParametricFunction.
    
    Specified by:
    
    copy in interface ParametricFunction
    
    Returns:
    
    a copy of this ParametricFunction.

Class LinearDiffRFVInit

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.functionapproximation.ParametricFunction

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

rfFeaturesAreForNextState

rfFvGen

vinitFvGen

rfDim

vinitDim

parameters

dim

Constructor Detail

LinearDiffRFVInit

LinearDiffRFVInit

Method Detail

isRfFeaturesAreForNextState

setRfFeaturesAreForNextState

getRfFvGen

setRfFvGen

getVinitFvGen

setVinitFvGen

getRfDim

setRfDim

getVinitDim

setVinitDim

gradient

reward

valueGradient

value

numParameters

getParameter

setParameter

resetParameters

copy