burlap.behavior.singleagent.learning.modellearning.models

Class TabularModel

java.lang.Object

burlap.behavior.singleagent.learning.modellearning.models.TabularModel

All Implemented Interfaces:

KWIKModel, LearnedModel, FullModel, SampleModel

public class TabularModel
extends java.lang.Object
implements KWIKModel

A tabular model using frequencies to model the transition dynamics.

Acknowledgements: Takehiro Oyakawa and Chan Trau for code on which this was based.

Author:

James MacGlashan

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.singleagent.learning.modellearning.KWIKModel

KWIKModel.Helper

Field Summary

Fields

Modifier and Type	Field and Description
protected HashableStateFactory	hashingFactory The hashing factory to use for indexing states
protected int	nConfident The number of transitions necessary to be confident in a model's prediction.
protected SADomain	sourceDomain The source actual domain object for which actions will be modeled.
protected java.util.Map<HashableState,burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateNode>	stateNodes A mapping from (hashed) states to state nodes that store transition statistics
protected java.util.Set<HashableState>	terminalStates The set of states marked as terminal states.

Constructor Summary

Constructors

Constructor and Description
TabularModel(SADomain sourceDomain, HashableStateFactory hashingFactory, int nConfident) Initializes.

Method Summary

Modifier and Type	Method and Description
protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode	getOrCreateActionNode(HashableState sh, Action ga) Returns the TabularModel.StateActionNode object associated with the given hashed state and action.
protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode	getStateActionNode(HashableState sh, Action a) Returns the TabularModel.StateActionNode object associated with the given hashed state and action.
void	resetModel() Resets the model data so that learning can begin anew.
EnvironmentOutcome	sample(State s, Action a) Samples a transition from the transition distribution and returns it.
boolean	terminal(State s) Indicates whether a state is a terminal state (i.e., no more action occurs and zero reward received from there on out)
boolean	transitionIsModeled(State s, Action ga) Indicates whether this model "knows" how the transition dynamics from the given input state and action work.
java.util.List<TransitionProb>	transitions(State s, Action a) Returns the set of possible transitions when Action is applied in State s.
void	updateModel(EnvironmentOutcome eo) Updates this model with respect to the observed EnvironmentOutcome.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sourceDomain

protected SADomain sourceDomain

The source actual domain object for which actions will be modeled.

hashingFactory

protected HashableStateFactory hashingFactory

The hashing factory to use for indexing states

stateNodes

protected java.util.Map<HashableState,burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateNode> stateNodes

A mapping from (hashed) states to state nodes that store transition statistics

terminalStates

protected java.util.Set<HashableState> terminalStates

The set of states marked as terminal states.

nConfident

protected int nConfident

The number of transitions necessary to be confident in a model's prediction.

Constructor Detail

TabularModel

public TabularModel(SADomain sourceDomain,
                    HashableStateFactory hashingFactory,
                    int nConfident)

Initializes.

Parameters:

sourceDomain - the source domain whose actions will be modeled.

hashingFactory - the hashing factory to index states

nConfident - the number of observed transitions to be confident in the model's prediction.

Method Detail

transitionIsModeled

public boolean transitionIsModeled(State s,
                                   Action ga)

Description copied from interface: KWIKModel

Indicates whether this model "knows" how the transition dynamics from the given input state and action work.

Specified by:

transitionIsModeled in interface KWIKModel

Parameters:

s - the state that is checked

ga - the action to take in state s

Returns:

true if the transition dynamics from the input state and action are "known;" false otherwise.

transitions

public java.util.List<TransitionProb> transitions(State s,
                                                  Action a)

Description copied from interface: FullModel

Returns the set of possible transitions when Action is applied in State s. The returned list only needs to include transitions that have non-zero probability of occurring.

Specified by:

transitions in interface FullModel

Parameters:

s - the source State

a - the Action applied in the source state

Returns:

the probability distribution over possible transitions.

sample

public EnvironmentOutcome sample(State s,
                                 Action a)

Description copied from interface: SampleModel

Samples a transition from the transition distribution and returns it.

Specified by:

sample in interface SampleModel

Parameters:

s - the source state

a - the action taken in the source state

Returns:

and EnvironmentOutcome describing the sampled transition

terminal

public boolean terminal(State s)

Description copied from interface: SampleModel

Indicates whether a state is a terminal state (i.e., no more action occurs and zero reward received from there on out)

Specified by:

terminal in interface SampleModel

Parameters:

s - the input state to test

Returns:

true if the state is a terminal state, false if it is not.

updateModel

public void updateModel(EnvironmentOutcome eo)

Description copied from interface: LearnedModel

Updates this model with respect to the observed EnvironmentOutcome.

Specified by:

updateModel in interface LearnedModel

Parameters:

eo - The EnvironmentOutcome specifying the observed interaction with an Environment.

getStateActionNode

protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode getStateActionNode(HashableState sh,
                                                                                                                    Action a)

Returns the TabularModel.StateActionNode object associated with the given hashed state and action. If there is not an associated TabularModel.StateActionNode object, then null is returned.

Parameters:

sh - the hashed state

a - the action

Returns:

the associated TabularModel.StateActionNode or null if it does not exist.

getOrCreateActionNode

protected burlap.behavior.singleagent.learning.modellearning.models.TabularModel.StateActionNode getOrCreateActionNode(HashableState sh,
                                                                                                                       Action ga)

Returns the TabularModel.StateActionNode object associated with the given hashed state and action. If there is not an associated TabularModel.StateActionNode object, then one will be created.

Parameters:

sh - the hashed state

ga - the grounded action

Returns:

the associated TabularModel.StateActionNode

resetModel

public void resetModel()

Description copied from interface: LearnedModel

Resets the model data so that learning can begin anew.

Specified by:

resetModel in interface LearnedModel