burlap.behavior.singleagent.planning.stochastic.sparsesampling

Class SparseSampling.StateNode

java.lang.Object

burlap.behavior.singleagent.planning.stochastic.sparsesampling.SparseSampling.StateNode

Enclosing class:

SparseSampling

public class SparseSampling.StateNode
extends java.lang.Object

A class for state nodes. Includes the state, a value estimate, whether the node has been closed and methods for estimating the Q and V values.

Author:

James MacGlashan

Constructor Summary

Constructors

Constructor and Description
StateNode(HashableState sh, int height) Creates a node for the given hased state at the given height

Method Summary

Modifier and Type	Method and Description
java.util.List<QValue>	estimateQs() Estimates and returns the Q-values for this node.
double	estimateV() Returns the estimated Q-value if this node is closed, or estimates it and closes it otherwise.
protected double	exactQValue(Action ga) Computes the exact Q-value using full Bellman update with the actual transition dynamics.
protected double	sampledQEstimate(Action ga) Estimates the Q-value using sampling from the transition dynamics.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StateNode

public StateNode(HashableState sh,
                 int height)

Creates a node for the given hased state at the given height

Parameters:

sh - the hashed state

height - the height of the node

Method Detail

estimateQs

public java.util.List<QValue> estimateQs()

Estimates and returns the Q-values for this node. Q-values and used state samples are forgotten after this call completes.

Returns:

a List of the estiamted Q-values for each action.

sampledQEstimate

protected double sampledQEstimate(Action ga)

Estimates the Q-value using sampling from the transition dynamics. This is the standard Sparse Sampling procedure.

Parameters:

ga - the action for which the Q-value estimate is to be returned

Returns:

the Q-value estimate

exactQValue

protected double exactQValue(Action ga)

Computes the exact Q-value using full Bellman update with the actual transition dynamics. This procedure will cause Sparse Sampling to compute the exact Q-values and optimal policy for a finite horizon problem. It is recommended when the number of transitions from any given state is small tractable to compute.

Parameters:

ga - the action for which the Q-value estimate is to be returned

Returns:

the exact finite horizon Q-value

estimateV

public double estimateV()

Returns the estimated Q-value if this node is closed, or estimates it and closes it otherwise.

Returns:

the estimated Q-value for this node.