burlap.behavior.singleagent.learnfromdemo.mlirl.differentiableplanners.diffvinit

Class VanillaDiffVinit

java.lang.Object

burlap.behavior.singleagent.learnfromdemo.mlirl.differentiableplanners.diffvinit.VanillaDiffVinit

All Implemented Interfaces:

DifferentiableVInit, ParametricFunction, ValueFunction, ValueFunctionInitialization

public class VanillaDiffVinit
extends java.lang.Object
implements DifferentiableVInit

A class for the default condition when a value function initialization returns an unparameterized value for each state, but must be differentiable with respect to the reward function parameters for use with a differentiable finite horizon valueFunction.

Author:

James MacGlashan.

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.valuefunction.ValueFunctionInitialization

ValueFunctionInitialization.ConstantValueFunctionInitialization

Nested classes/interfaces inherited from interface burlap.behavior.singleagent.vfa.ParametricFunction

ParametricFunction.ParametricStateActionFunction, ParametricFunction.ParametricStateFunction

Field Summary

Fields

Modifier and Type	Field and Description
protected DifferentiableRF	rf The differentiable reward function that defines the parameter space over which this value function initialization must differentiate.
protected ValueFunctionInitialization	vinit The source value function initialization.

Constructor Summary

Constructors

Constructor and Description
VanillaDiffVinit(ValueFunctionInitialization vinit, DifferentiableRF rf) Initializes.

Method Summary

Methods

Modifier and Type	Method and Description
ParametricFunction	copy() Returns a copy of this ParametricFunction.
double	getParameter(int i) Returns the value of the ith parameter value
FunctionGradient	getQGradient(State s, AbstractGroundedAction ga) Returns the Q-value function gradient.
FunctionGradient	getVGradient(State s) Returns the value function gradient.
int	numParameters() Returns the number of parameters defining this function.
double	qValue(State s, AbstractGroundedAction a) Returns the initialization value of the Q-value function for a given state and action pair.
void	resetParameters() Resets the parameters of this function to default values.
void	setParameter(int i, double p) Sets the value of the ith parameter to given value
double	value(State s) Returns the value function evaluation of the given state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

vinit

protected ValueFunctionInitialization vinit

The source value function initialization.

rf

protected DifferentiableRF rf

The differentiable reward function that defines the parameter space over which this value function initialization must differentiate.

Constructor Detail

VanillaDiffVinit

public VanillaDiffVinit(ValueFunctionInitialization vinit,
                DifferentiableRF rf)

Initializes.

Parameters:

vinit - The vanilla unparameterized value function initialization

rf - the differentiable reward function that defines the total parameter space

Method Detail

numParameters

public int numParameters()

Description copied from interface: ParametricFunction

Returns the number of parameters defining this function. Note that some implementations my have a dynamic number of parameters that grows or shrinks over time. Consult the documentation for the specific implementation for more information.

Specified by:

numParameters in interface ParametricFunction

Returns:

the number of parameters defining this function.

getParameter

public double getParameter(int i)

Description copied from interface: ParametricFunction

Returns the value of the ith parameter value

Specified by:

getParameter in interface ParametricFunction

Parameters:

i - the parameter index

Returns:

the double value of the ith parameter

setParameter

public void setParameter(int i,
                double p)

Description copied from interface: ParametricFunction

Sets the value of the ith parameter to given value

Specified by:

setParameter in interface ParametricFunction

Parameters:

i - the index of the parameter to set

p - the parameter value to which it should be set

resetParameters

public void resetParameters()

Description copied from interface: ParametricFunction

Resets the parameters of this function to default values.

Specified by:

resetParameters in interface ParametricFunction

copy

public ParametricFunction copy()

Description copied from interface: ParametricFunction

Returns a copy of this ParametricFunction.

Specified by:

copy in interface ParametricFunction

Returns:

a copy of this ParametricFunction.

getVGradient

public FunctionGradient getVGradient(State s)

Description copied from interface: DifferentiableVInit

Returns the value function gradient.

Specified by:

getVGradient in interface DifferentiableVInit

Parameters:

s - the state on which the value function is to be evaluated

Returns:

the value function gradient.

getQGradient

public FunctionGradient getQGradient(State s,
                            AbstractGroundedAction ga)

Description copied from interface: DifferentiableVInit

Returns the Q-value function gradient.

Specified by:

getQGradient in interface DifferentiableVInit

Parameters:

s - the state on which the Q-value is to be evaluated.

ga - the action on which the Q-value is to be evaluated.

Returns:

the Q-value function gradient

value

public double value(State s)

Description copied from interface: ValueFunction

Returns the value function evaluation of the given state. If the value is not stored, then the default value specified by the ValueFunctionInitialization object of this class is returned.

Specified by:

value in interface ValueFunction

Parameters:

s - the state to evaluate.

Returns:

the value function evaluation of the given state.

qValue

public double qValue(State s,
            AbstractGroundedAction a)

Description copied from interface: ValueFunctionInitialization

Returns the initialization value of the Q-value function for a given state and action pair.

Specified by:

qValue in interface ValueFunctionInitialization

Parameters:

s - the state for which to get the initial value of the Q-value function.

a - the action for which to get the initial value of the Q-value function.

Returns:

the initialization value of the Q-value function for a given state and action pair.