burlap.behavior.singleagent.learning.modellearning.rmax

Class RMaxModel

java.lang.Object

burlap.behavior.singleagent.learning.modellearning.rmax.RMaxModel

All Implemented Interfaces:

KWIKModel, LearnedModel, FullModel, SampleModel

public class RMaxModel
extends java.lang.Object
implements KWIKModel

Author:

James MacGlashan.

Nested Class Summary

Nested classes/interfaces inherited from interface burlap.behavior.singleagent.learning.modellearning.KWIKModel

KWIKModel.Helper

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<ActionType>	actionsTypes
protected double	gamma
protected PotentialFunction	potentialFunction
protected KWIKModel	sourceModel

Constructor Summary

Constructors

Constructor and Description
RMaxModel(KWIKModel sourceModel, PotentialFunction potentialFunction, double gamma, java.util.List<ActionType> actionsTypes)

Method Summary

Modifier and Type	Method and Description
java.util.List<ActionType>	getActionsTypes()
double	getGamma()
PotentialFunction	getPotentialFunction()
KWIKModel	getSourceModel()
protected void	modifyEO(EnvironmentOutcome eo)
void	resetModel() Resets the model data so that learning can begin anew.
EnvironmentOutcome	sample(State s, Action a) Samples a transition from the transition distribution and returns it.
void	setActionsTypes(java.util.List<ActionType> actionsTypes)
void	setGamma(double gamma)
void	setPotentialFunction(PotentialFunction potentialFunction)
void	setSourceModel(KWIKModel sourceModel)
boolean	terminal(State s) Indicates whether a state is a terminal state (i.e., no more action occurs and zero reward received from there on out)
boolean	transitionIsModeled(State s, Action a) Indicates whether this model "knows" how the transition dynamics from the given input state and action work.
java.util.List<TransitionProb>	transitions(State s, Action a) Returns the set of possible transitions when Action is applied in State s.
void	updateModel(EnvironmentOutcome eo) Updates this model with respect to the observed EnvironmentOutcome.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sourceModel

protected KWIKModel sourceModel

potentialFunction

protected PotentialFunction potentialFunction

gamma

protected double gamma

actionsTypes

protected java.util.List<ActionType> actionsTypes

Constructor Detail

RMaxModel

public RMaxModel(KWIKModel sourceModel,
                 PotentialFunction potentialFunction,
                 double gamma,
                 java.util.List<ActionType> actionsTypes)

Method Detail

transitionIsModeled

public boolean transitionIsModeled(State s,
                                   Action a)

Description copied from interface: KWIKModel

Indicates whether this model "knows" how the transition dynamics from the given input state and action work.

Specified by:

transitionIsModeled in interface KWIKModel

Parameters:

s - the state that is checked

a - the action to take in state s

Returns:

true if the transition dynamics from the input state and action are "known;" false otherwise.

updateModel

public void updateModel(EnvironmentOutcome eo)

Description copied from interface: LearnedModel

Updates this model with respect to the observed EnvironmentOutcome.

Specified by:

updateModel in interface LearnedModel

Parameters:

eo - The EnvironmentOutcome specifying the observed interaction with an Environment.

resetModel

public void resetModel()

Description copied from interface: LearnedModel

Resets the model data so that learning can begin anew.

Specified by:

resetModel in interface LearnedModel

transitions

public java.util.List<TransitionProb> transitions(State s,
                                                  Action a)

Description copied from interface: FullModel

Returns the set of possible transitions when Action is applied in State s. The returned list only needs to include transitions that have non-zero probability of occurring.

Specified by:

transitions in interface FullModel

Parameters:

s - the source State

a - the Action applied in the source state

Returns:

the probability distribution over possible transitions.

sample

public EnvironmentOutcome sample(State s,
                                 Action a)

Description copied from interface: SampleModel

Samples a transition from the transition distribution and returns it.

Specified by:

sample in interface SampleModel

Parameters:

s - the source state

a - the action taken in the source state

Returns:

and EnvironmentOutcome describing the sampled transition

terminal

public boolean terminal(State s)

Description copied from interface: SampleModel

Indicates whether a state is a terminal state (i.e., no more action occurs and zero reward received from there on out)

Specified by:

terminal in interface SampleModel

Parameters:

s - the input state to test

Returns:

true if the state is a terminal state, false if it is not.

modifyEO

protected void modifyEO(EnvironmentOutcome eo)

getSourceModel

public KWIKModel getSourceModel()

setSourceModel

public void setSourceModel(KWIKModel sourceModel)

getPotentialFunction

public PotentialFunction getPotentialFunction()

setPotentialFunction

public void setPotentialFunction(PotentialFunction potentialFunction)

getGamma

public double getGamma()

setGamma

public void setGamma(double gamma)

getActionsTypes

public java.util.List<ActionType> getActionsTypes()

setActionsTypes

public void setActionsTypes(java.util.List<ActionType> actionsTypes)