burlap.behavior.singleagent.learning.modellearning.models

Class TabularModel

java.lang.Object

burlap.behavior.singleagent.learning.modellearning.Model

burlap.behavior.singleagent.learning.modellearning.models.TabularModel

public class TabularModel
extends Model

A tabular model using frequencies to model the transition dynamics.

Author:

James MacGlashan; adapted from code provided by Takehiro Oyakawa and Chan Trau

Field Summary

Fields

Modifier and Type	Field and Description
protected HashableStateFactory	hashingFactory The hashing factory to use for indexing states
protected RewardFunction	modeledRF The modeled reward function.
protected TerminalFunction	modeledTF The modeled terminal funciton.
protected int	nConfident The number of transitions necessary to be confident in a model's prediction.
protected Domain	sourceDomain The source actual domain object for which actions will be modeled.
protected java.util.Map<HashableState,burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateNode>	stateNodes A mapping from (hashed) states to state nodes that store transition statistics
protected java.util.Set<HashableState>	terminalStates The set of states marked as terminal states.

Fields inherited from class burlap.behavior.singleagent.learning.modellearning.Model

rand

Constructor Summary

Constructors

Constructor and Description
TabularModel(Domain sourceDomain, HashableStateFactory hashingFactory, int nConfident) Initializes.

Method Summary

Methods

Modifier and Type	Method and Description
RewardFunction	getModelRF() Returns the learned reward function in this model
TerminalFunction	getModelTF() Returns the learned terminal function in this model
protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode	getOrCreateActionNode(HashableState sh, GroundedAction ga) Returns the TabularModel.StateActionNode object associated with the given hashed state and action.
protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode	getStateActionNode(HashableState sh, GroundedAction ga) Returns the TabularModel.StateActionNode object associated with the given hashed state and action.
java.util.List<TransitionProbability>	getTransitionProbabilities(State s, GroundedAction ga) Returns this model's transition probabilities for the given source state and action
java.util.List<AbstractGroundedAction>	getUnmodeledActionsForState(State s) Returns a list specifying the actions for which the transition dynamics are not yet "known."
void	resetModel() Resets the model data so that learning can begin anew.
State	sampleModelHelper(State s, GroundedAction ga) A helper method to sample this model's transition dynamics for the given state and action.
boolean	stateTransitionsAreModeled(State s) Indicates whether this model "knows" the transition dynamics from the given input state for all applicable actions.
boolean	transitionIsModeled(State s, GroundedAction ga) Indicates whether this model "knows" how the transition dynamics from the given input state and action work.
void	updateModel(State s, GroundedAction ga, State sprime, double r, boolean sprimeIsTerminal) Causes this model to be updated with a new interaction with the world.

Methods inherited from class burlap.behavior.singleagent.learning.modellearning.Model

sampleModel, sampleTransitionFromTransitionProbabilities, updateModel

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sourceDomain

protected Domain sourceDomain

The source actual domain object for which actions will be modeled.

hashingFactory

protected HashableStateFactory hashingFactory

The hashing factory to use for indexing states

stateNodes

protected java.util.Map<HashableState,burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateNode> stateNodes

A mapping from (hashed) states to state nodes that store transition statistics

terminalStates

protected java.util.Set<HashableState> terminalStates

The set of states marked as terminal states.

nConfident

protected int nConfident

The number of transitions necessary to be confident in a model's prediction.

modeledTF

protected TerminalFunction modeledTF

The modeled terminal funciton.

modeledRF

protected RewardFunction modeledRF

The modeled reward function.

Constructor Detail

TabularModel

public TabularModel(Domain sourceDomain,
            HashableStateFactory hashingFactory,
            int nConfident)

Initializes.

Parameters:

sourceDomain - the source domain whose actions will be modeled.

hashingFactory - the hashing factory to index states

nConfident - the number of observed transitions to be confidnent in the model's prediction.

Method Detail

getModelRF

public RewardFunction getModelRF()

Description copied from class: Model

Returns the learned reward function in this model

Specified by:

getModelRF in class Model

Returns:

the learned reward function in this model

getModelTF

public TerminalFunction getModelTF()

Description copied from class: Model

Returns the learned terminal function in this model

Specified by:

getModelTF in class Model

Returns:

the learned terminal function in this model

transitionIsModeled

public boolean transitionIsModeled(State s,
                          GroundedAction ga)

Description copied from class: Model

Indicates whether this model "knows" how the transition dynamics from the given input state and action work.

Specified by:

transitionIsModeled in class Model

Parameters:

s - the state that is checked

ga - the action to take in state s

Returns:

true if the transition dynamics from the input state and action are "known;" false otherwise.

stateTransitionsAreModeled

public boolean stateTransitionsAreModeled(State s)

Description copied from class: Model

Indicates whether this model "knows" the transition dynamics from the given input state for all applicable actions.

Specified by:

stateTransitionsAreModeled in class Model

Parameters:

s - the state that is checked.

Returns:

true if the transition dynamics for all actions are "known;" false otherwise.

getUnmodeledActionsForState

public java.util.List<AbstractGroundedAction> getUnmodeledActionsForState(State s)

Description copied from class: Model

Returns a list specifying the actions for which the transition dynamics are not yet "known."

Specified by:

getUnmodeledActionsForState in class Model

Parameters:

s - the state for which the un-modeled actions should be returned.

Returns:

a List of AbstractGroundedAction objects

sampleModelHelper

public State sampleModelHelper(State s,
                      GroundedAction ga)

Description copied from class: Model

A helper method to sample this model's transition dynamics for the given state and action.

Specified by:

sampleModelHelper in class Model

Parameters:

s - a copied source state which can be directly modified.

ga - the action taken in the source state

Returns:

a State which is a sampled result of taking the given action in the given state. This may be the same object as State s.

getTransitionProbabilities

public java.util.List<TransitionProbability> getTransitionProbabilities(State s,
                                                               GroundedAction ga)

Description copied from class: Model

Returns this model's transition probabilities for the given source state and action

Specified by:

getTransitionProbabilities in class Model

Parameters:

s - the source state

ga - an action taken in the source state

Returns:

the list of the possible transition probabilities; should sum to 1 and states will probability zero do *not* have to be included.

updateModel

public void updateModel(State s,
               GroundedAction ga,
               State sprime,
               double r,
               boolean sprimeIsTerminal)

Description copied from class: Model

Causes this model to be updated with a new interaction with the world.

Specified by:

updateModel in class Model

Parameters:

s - a source state

ga - an action that was taken in the source state

sprime - the state to which the agent transitioned for taking action

r - a reward that was received for taking the given action in the given source state and transitioning to the given next state

sprimeIsTerminal - whether the next state is a terminal state or not

getStateActionNode

protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode getStateActionNode(HashableState sh,
                                                                                                        GroundedAction ga)

Returns the TabularModel.StateActionNode object associated with the given hashed state and action. If there is not an associated TabularModel.StateActionNode object, then null is returned.

Parameters:

sh - the hashed state

ga - the grounded action

Returns:

the associated TabularModel.StateActionNode or null if it does not exist.

getOrCreateActionNode

protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode getOrCreateActionNode(HashableState sh,
                                                                                                           GroundedAction ga)

Returns the TabularModel.StateActionNode object associated with the given hashed state and action. If there is not an associated TabularModel.StateActionNode object, then one will be created.

Parameters:

sh - the hashed state

ga - the grounded action

Returns:

the associated TabularModel.StateActionNode

resetModel

public void resetModel()

Description copied from class: Model

Resets the model data so that learning can begin anew.

Specified by:

resetModel in class Model