burlap.behavior.singleagent.learnfromdemo.mlirl.differentiableplanners.diffvinit

Class LinearStateDiffVF

java.lang.Object

burlap.behavior.singleagent.learnfromdemo.mlirl.differentiableplanners.diffvinit.LinearStateDiffVF

All Implemented Interfaces:

DifferentiableVInit, ParametricFunction, ValueFunction, ValueFunctionInitialization

public class LinearStateDiffVF
extends java.lang.Object
implements DifferentiableVInit

A class for defining a (differentiable) linear function over state features for value function initialization. This class is useful for learning the value function initialization for leaf nodes of a finite horizon valueFunction with MLIRL.

Author:

James MacGlashan.

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.valuefunction.ValueFunctionInitialization

ValueFunctionInitialization.ConstantValueFunctionInitialization

Nested classes/interfaces inherited from interface burlap.behavior.singleagent.vfa.ParametricFunction

ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

Field Summary

Fields

Modifier and Type	Field and Description
protected int	dim
protected StateToFeatureVectorGenerator	fvgen The state feature vector generator over which the linear function operates
protected double[]	parameters

Constructor Summary

Constructors

Constructor and Description
LinearStateDiffVF(StateToFeatureVectorGenerator fvgen, int dim) Initializes with the state feature vector generator over which the linear function is defined and the dimensionality of it.

Method Summary

Methods

Modifier and Type	Method and Description
ParametricFunction	copy() Returns a copy of this ParametricFunction.
double	getParameter(int i) Returns the value of the ith parameter value
FunctionGradient	getQGradient(State s, AbstractGroundedAction ga) Returns the Q-value function gradient.
FunctionGradient	getVGradient(State s) Returns the value function gradient.
int	numParameters() Returns the number of parameters defining this function.
double	qValue(State s, AbstractGroundedAction a) Returns the initialization value of the Q-value function for a given state and action pair.
void	resetParameters() Resets the parameters of this function to default values.
void	setParameter(int i, double p) Sets the value of the ith parameter to given value
double	value(State s) Returns the value function evaluation of the given state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fvgen

protected StateToFeatureVectorGenerator fvgen

The state feature vector generator over which the linear function operates

dim

protected int dim

parameters

protected double[] parameters

Constructor Detail

LinearStateDiffVF

public LinearStateDiffVF(StateToFeatureVectorGenerator fvgen,
                 int dim)

Initializes with the state feature vector generator over which the linear function is defined and the dimensionality of it.

Parameters:

fvgen - the state feature vector generator over which the linear function is defined.

dim - the dimensionality of the feature vector/parameters

Method Detail

numParameters

public int numParameters()

Description copied from interface: ParametricFunction

Returns the number of parameters defining this function. Note that some implementations my have a dynamic number of parameters that grows or shrinks over time. Consult the documentation for the specific implementation for more information.

Specified by:

numParameters in interface ParametricFunction

Returns:

the number of parameters defining this function.

getParameter

public double getParameter(int i)

Description copied from interface: ParametricFunction

Returns the value of the ith parameter value

Specified by:

getParameter in interface ParametricFunction

Parameters:

i - the parameter index

Returns:

the double value of the ith parameter

setParameter

public void setParameter(int i,
                double p)

Description copied from interface: ParametricFunction

Sets the value of the ith parameter to given value

Specified by:

setParameter in interface ParametricFunction

Parameters:

i - the index of the parameter to set

p - the parameter value to which it should be set

resetParameters

public void resetParameters()

Description copied from interface: ParametricFunction

Resets the parameters of this function to default values.

Specified by:

resetParameters in interface ParametricFunction

copy

public ParametricFunction copy()

Description copied from interface: ParametricFunction

Returns a copy of this ParametricFunction.

Specified by:

copy in interface ParametricFunction

Returns:

a copy of this ParametricFunction.

getVGradient

public FunctionGradient getVGradient(State s)

Description copied from interface: DifferentiableVInit

Returns the value function gradient.

Specified by:

getVGradient in interface DifferentiableVInit

Parameters:

s - the state on which the value function is to be evaluated

Returns:

the value function gradient.

getQGradient

public FunctionGradient getQGradient(State s,
                            AbstractGroundedAction ga)

Description copied from interface: DifferentiableVInit

Returns the Q-value function gradient.

Specified by:

getQGradient in interface DifferentiableVInit

Parameters:

s - the state on which the Q-value is to be evaluated.

ga - the action on which the Q-value is to be evaluated.

Returns:

the Q-value function gradient

value

public double value(State s)

Description copied from interface: ValueFunction

Returns the value function evaluation of the given state. If the value is not stored, then the default value specified by the ValueFunctionInitialization object of this class is returned.

Specified by:

value in interface ValueFunction

Parameters:

s - the state to evaluate.

Returns:

the value function evaluation of the given state.

qValue

public double qValue(State s,
            AbstractGroundedAction a)

Description copied from interface: ValueFunctionInitialization

Returns the initialization value of the Q-value function for a given state and action pair.

Specified by:

qValue in interface ValueFunctionInitialization

Parameters:

s - the state for which to get the initial value of the Q-value function.

a - the action for which to get the initial value of the Q-value function.

Returns:

the initialization value of the Q-value function for a given state and action pair.