burlap.behavior.stochasticgames.agents.interfacing.singleagent

Class LearningAgentToSGAgentInterface

java.lang.Object

burlap.mdp.stochasticgames.agent.SGAgentBase

burlap.behavior.stochasticgames.agents.interfacing.singleagent.LearningAgentToSGAgentInterface

All Implemented Interfaces:

Environment, SGAgent

public class LearningAgentToSGAgentInterface
extends SGAgentBase
implements Environment

A stochastic games SGAgent that takes as input a single agent LearningAgent to handle behavior. The interface from the single agent paradigm to the multi-agent paradigm is handled by this class also implementing the Environment interface. When a game starts, a new thread is launched in which the provided LearningAgent interacts with this class's Environment methods.

Author:

James MacGlashan.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	LearningAgentToSGAgentInterface.ActionReference A wrapper that maintains a reference to a Action or null.
protected static class	LearningAgentToSGAgentInterface.StateReference A wrapper that maintains a reference to a State or null.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	agentNum
protected State	currentState The current state of the world
protected boolean	curStateIsTerminal Whether the last state was a terminal state
protected double	lastReward The last reward received by this agent
protected LearningAgent	learningAgent The single agent LearningAgent that will be learning in this stochastic game as if the other players are part of the environment.
protected LearningAgentToSGAgentInterface.ActionReference	nextAction The next action selected by the single agent
protected LearningAgentToSGAgentInterface.StateReference	nextState The next state received
protected java.lang.Thread	saThread The thread that runs the single agent learning algorithm

Fields inherited from class burlap.mdp.stochasticgames.agent.SGAgentBase

agentType, domain, internalRewardFunction, world, worldAgentName

Constructor Summary

Constructors

Constructor and Description
LearningAgentToSGAgentInterface(SGDomain domain, LearningAgent learningAgent, java.lang.String agentName, SGAgentType agentType) Initializes.

Method Summary

Modifier and Type	Method and Description
Action	action(State s) This method is called by the world when it needs the agent to choose an action
State	currentObservation() Returns the current observation of the environment as a State.
EnvironmentOutcome	executeAction(Action ga) Executes the specified action in this environment
void	gameStarting(World w, int agentNum) This method is called by the world when a new game is starting.
void	gameTerminated() This method is called by the world when a game has ended.
boolean	isInTerminalState() Returns whether the environment is in a terminal state that prevents further action by the agent.
double	lastReward() Returns the last reward returned by the environment
void	observeOutcome(State s, JointAction jointAction, double[] jointReward, State sprime, boolean isTerminal) This method is called by the world when every agent in the world has taken their action.
void	resetEnvironment() Resets this environment to some initial state, if the functionality exists.

Methods inherited from class burlap.mdp.stochasticgames.agent.SGAgentBase

agentName, agentType, getInternalRewardFunction, init, init, setAgentDetails, setInternalRewardFunction

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

learningAgent

protected LearningAgent learningAgent

The single agent LearningAgent that will be learning in this stochastic game as if the other players are part of the environment.

curStateIsTerminal

protected boolean curStateIsTerminal

Whether the last state was a terminal state

lastReward

protected double lastReward

The last reward received by this agent

currentState

protected State currentState

The current state of the world

saThread

protected java.lang.Thread saThread

The thread that runs the single agent learning algorithm

nextAction

protected LearningAgentToSGAgentInterface.ActionReference nextAction

The next action selected by the single agent

nextState

protected LearningAgentToSGAgentInterface.StateReference nextState

The next state received

agentNum

protected int agentNum

Constructor Detail

LearningAgentToSGAgentInterface

public LearningAgentToSGAgentInterface(SGDomain domain,
                                       LearningAgent learningAgent,
                                       java.lang.String agentName,
                                       SGAgentType agentType)

Initializes.

Parameters:

domain - The stochastic games SGDomain in which this agent will interact.

learningAgent - the LearningAgent that will handle this SGAgent's control.

agentName - the name of the agent

agentType - the SGAgentType for the agent defining its action space

Method Detail

gameStarting

public void gameStarting(World w,
                         int agentNum)

Description copied from interface: SGAgent

This method is called by the world when a new game is starting.

Specified by:

gameStarting in interface SGAgent

Parameters:

w - the world in which the game is starting

agentNum - the agent number of the agent in the world

action

public Action action(State s)

Description copied from interface: SGAgent

This method is called by the world when it needs the agent to choose an action

Specified by:

action in interface SGAgent

Parameters:

s - the current state of the world

Returns:

the action this agent wishes to take

observeOutcome

public void observeOutcome(State s,
                           JointAction jointAction,
                           double[] jointReward,
                           State sprime,
                           boolean isTerminal)

Description copied from interface: SGAgent

This method is called by the world when every agent in the world has taken their action. It conveys the result of the joint action.

Specified by:

observeOutcome in interface SGAgent

Parameters:

s - the state in which the last action of each agent was taken

jointAction - the joint action of all agents in the world

jointReward - the joint reward of all agents in the world

sprime - the next state to which the agent transitioned

isTerminal - whether the new state is a terminal state

gameTerminated

public void gameTerminated()

Description copied from interface: SGAgent

This method is called by the world when a game has ended.

Specified by:

gameTerminated in interface SGAgent

currentObservation

public State currentObservation()

Description copied from interface: Environment

Returns the current observation of the environment as a State.

Specified by:

currentObservation in interface Environment

Returns:

the current observation of the environment as a State.

executeAction

public EnvironmentOutcome executeAction(Action ga)

Description copied from interface: Environment

Executes the specified action in this environment

Specified by:

executeAction in interface Environment

Parameters:

ga - the Action that is to be performed in this environment.

Returns:

the resulting observation and reward transition from applying the given GroundedAction in this environment.

lastReward

public double lastReward()

Description copied from interface: Environment

Returns the last reward returned by the environment

Specified by:

lastReward in interface Environment

Returns:

the last reward returned by the environment

isInTerminalState

public boolean isInTerminalState()

Description copied from interface: Environment

Returns whether the environment is in a terminal state that prevents further action by the agent.

Specified by:

isInTerminalState in interface Environment

Returns:

true if the current environment is in a terminal state; false otherwise.

resetEnvironment

public void resetEnvironment()

Description copied from interface: Environment

Resets this environment to some initial state, if the functionality exists.

Specified by:

resetEnvironment in interface Environment