burlap.mdp.singleagent.pomdp

Class BeliefAgent

java.lang.Object

burlap.mdp.singleagent.pomdp.BeliefAgent

Direct Known Subclasses:

BeliefPolicyAgent

public abstract class BeliefAgent
extends java.lang.Object

An agent that interacts with a POMDP environment. This class contains methods for acting until environment termination or some fixed number of steps and recording the results in an Episode object. These methods will automatically update this agent's BeliefState, specified by the curBelief data member, as observations are made. Before beginning, the agent's initial BeliefState will need to be specified with the setBeliefState(burlap.mdp.singleagent.pomdp.beliefstate.BeliefState) method. Different agents can be specified by subclassing and implementing the getAction(burlap.mdp.singleagent.pomdp.beliefstate.BeliefState) method.

Field Summary

Fields

Modifier and Type	Field and Description
protected BeliefState	curBelief The agent's current BeliefState
protected Environment	environment The POMDP environment.
protected PODomain	poDomain The POMDP Domain defining the environment mechanics.
protected BeliefUpdate	updater The belief update to use

Constructor Summary

Constructors

Constructor and Description
BeliefAgent(PODomain poDomain, Environment environment) Initializes.

Method Summary

Modifier and Type	Method and Description
Episode	actUntilTerminal() Causes the agent to act until the environment reaches a termination condition.
Episode	actUntilTerminalOrMaxSteps(int maxSteps) Causes the agent to act for some fixed number of steps.
abstract Action	getAction(BeliefState curBelief) Returns the action the agent should take for the input BeliefState.
BeliefUpdate	getUpdater()
void	setBeliefState(BeliefState beliefState) Sets this agent's current belief
void	setEnvironment(Environment environment) Sets the POMDP environment
void	setUpdater(BeliefUpdate updater)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

environment

protected Environment environment

The POMDP environment.

curBelief

protected BeliefState curBelief

The agent's current BeliefState

poDomain

protected PODomain poDomain

The POMDP Domain defining the environment mechanics.

updater

protected BeliefUpdate updater

The belief update to use

Constructor Detail

BeliefAgent

public BeliefAgent(PODomain poDomain,
                   Environment environment)

Initializes. By default will use a TabularBeliefUpdate, but you can change that with the setUpdater(BeliefUpdate) method.

Parameters:

poDomain - the POMDP domain defining the mechanics of the environment

environment - the environment in which the agent will be interacting.

Method Detail

setEnvironment

public void setEnvironment(Environment environment)

Sets the POMDP environment

Parameters:

environment - the POMDP environment

setBeliefState

public void setBeliefState(BeliefState beliefState)

Sets this agent's current belief

Parameters:

beliefState - the agent' current belief

getUpdater

public BeliefUpdate getUpdater()

setUpdater

public void setUpdater(BeliefUpdate updater)

actUntilTerminal

public Episode actUntilTerminal()

Causes the agent to act until the environment reaches a termination condition. The agent's belief is automatically updated by this method using the specified BeliefUpdate. The agent's action selection for the current belief state is defined by the getAction(burlap.mdp.singleagent.pomdp.beliefstate.BeliefState) method. The observation, action, and reward sequence is saved and Episode object and returned.

Returns:

and Episode that recorded the observation, action, and reward sequence.

actUntilTerminalOrMaxSteps

public Episode actUntilTerminalOrMaxSteps(int maxSteps)

Causes the agent to act for some fixed number of steps. The agent's belief is automatically updated by this method using the specified BeliefUpdate. The agent's action selection for the current belief state is defined by the getAction(burlap.mdp.singleagent.pomdp.beliefstate.BeliefState) method. The observation, action, and reward sequence is saved and Episode object and returned.

Parameters:

maxSteps - the maximum number of steps to take in the environment

Returns:

and Episode that recorded the observation, action, and reward sequence.

getAction

public abstract Action getAction(BeliefState curBelief)

Returns the action the agent should take for the input BeliefState.

Parameters:

curBelief - the BeliefState in which the agent must make a decision.

Returns:

A Action specifying the agent's decision for the input BeliefState.