burlap.domain.singleagent.graphdefined

Class GraphDefinedDomain

java.lang.Object

burlap.domain.singleagent.graphdefined.GraphDefinedDomain

All Implemented Interfaces:

DomainGenerator

public class GraphDefinedDomain
extends java.lang.Object
implements DomainGenerator

A domain generator for generating domains that are represented as graphs. The graph transitions may be stochastic. Edges are added/modified for state-action pairs with the setTransition(int, int, int, double) method and transitions can be cleared/removed with the methods clearStateTransitionsFrom(int), clearStateActionTransitions(int, int), and removeEdge(int, int, int).

A constructed graph's transition dynamics can be validated as proper (all edges from a state-action pair sum to 1) using the method isValidMDPGraph() and if they are false, a string reporting which state-actions do not sum to 1 (and their various edges) can be returned with the invalidMDPReport() method.

Modifying the transition dynamics of a graph will not affect the transition dynamics of previously generated Domain, allowing you to reuse the same generator without affected previous domains.

Author:

James MacGlashan

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GraphDefinedDomain.GraphActionType An action class for defining actions that can be taken from state nodes.
static class	GraphDefinedDomain.GraphStateModel
class	GraphDefinedDomain.NodeTransitionProbability A class for specifying transition probabilities to result node states.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	BASE_ACTION_NAME Constant for the base name of each action
protected int	maxActions The maximum number of actions available from any given state node.
protected int	numNodes The number of state nodes in the graph
protected RewardFunction	rf
protected TerminalFunction	tf
protected java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>>	transitionDynamics The state-action stochastic transition dynamics from each state node.
static java.lang.String	VAR the variable key for the single graph node values of a state

Constructor Summary

Constructors

Constructor and Description
GraphDefinedDomain() Initializes the generator.
GraphDefinedDomain(int numNodes) Initializes the generator to create a domain with the given number of state nodes in it.

Method Summary

Modifier and Type	Method and Description
void	clearStateActionTransitions(int srcNode, int action) Clears all (stochastic) edges for a given state-action pair.
void	clearStateTransitionsFrom(int srcNode) Clears all transitions from a given state node
protected java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>>	copyTransitionDynamics() Returns a deep copy of the transition dynamics
SADomain	generateDomain() Returns a newly instanced Domain object
protected GraphDefinedDomain.NodeTransitionProbability	getNodeTransitionTo(java.util.Set<GraphDefinedDomain.NodeTransitionProbability> nts, int tNode) Returns the GraphDefinedDomain.NodeTransitionProbability object in the provided set that corresponds to a transition to state tNode or null if it doesn't exist.
int	getNumNodes() Returns the number of state nodes specified in this domain
RewardFunction	getRf()
TerminalFunction	getTf()
java.lang.String	invalidMDPReport() Returns a string that lists the state-action paris that have improper transition dynamics (transitions that don't sum to 1).
static java.lang.String	invalidMDPReport(java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>> transitionDynamics) Returns a string that lists the state-action paris that have improper transition dynamics (transitions that don't sum to 1).
boolean	isValidMDPGraph() Checks whether the the probability of all state-action outcomes sum to 1 in this graph.
static boolean	isValidMDPGraph(java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>> transitionDynamics) Checks whether the the probability of all state-action outcomes sum to 1 in he provided transition dynamics.
static void	main(java.lang.String[] args)
void	removeEdge(int srcNode, int action, int tNode) Removes a given edge from the transition dynamics.
void	setRf(RewardFunction rf)
void	setTf(TerminalFunction tf)
void	setTransition(int srcNode, int action, int tNode, double p) Sets the probability p for transitioning to state node tNode after taking action number action in state node srcNode.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VAR

public static final java.lang.String VAR

the variable key for the single graph node values of a state

See Also:

Constant Field Values

BASE_ACTION_NAME

public static final java.lang.String BASE_ACTION_NAME

Constant for the base name of each action

See Also:

Constant Field Values

numNodes

protected int numNodes

The number of state nodes in the graph

maxActions

protected int maxActions

The maximum number of actions available from any given state node.

transitionDynamics

protected java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>> transitionDynamics

The state-action stochastic transition dynamics from each state node.

rf

protected RewardFunction rf

tf

protected TerminalFunction tf

Constructor Detail

GraphDefinedDomain

public GraphDefinedDomain()

Initializes the generator. States and transition dynamics will be constructed lazily with calls to the setTransition(int, int, int, double) method.

GraphDefinedDomain

public GraphDefinedDomain(int numNodes)

Initializes the generator to create a domain with the given number of state nodes in it.

Parameters:

numNodes - the number of state nodes in the domain.

Method Detail

getNumNodes

public int getNumNodes()

Returns the number of state nodes specified in this domain

Returns:

the number of state nodes specified in this domain

getRf

public RewardFunction getRf()

setRf

public void setRf(RewardFunction rf)

getTf

public TerminalFunction getTf()

setTf

public void setTf(TerminalFunction tf)

setTransition

public void setTransition(int srcNode,
                          int action,
                          int tNode,
                          double p)

Sets the probability p for transitioning to state node tNode after taking action number action in state node srcNode. Note that this method also defines from which nodes an action can be executed. If this method is never called for source node i and action j, then it will be assumed that action j cannot be executed in state node i. The the specified source node or target node is greater than this domain generator's previously specified number of nodes, then space will be added for them in the transition dynamics.

Parameters:

srcNode - the source node number from which an action will be taken

action - the action to be taken

tNode - the resulting state from taking the action in the given source node

p - the probability of this transition occurring for the given action and source node.

isValidMDPGraph

public boolean isValidMDPGraph()

Checks whether the the probability of all state-action outcomes sum to 1 in this graph. Returns true if they do, false otherwise. If this method returns false, you can use the method to get a string reporting where errors are.

Returns:

true if all probability distributions sum to 1; false otherwise

isValidMDPGraph

public static boolean isValidMDPGraph(java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>> transitionDynamics)

Checks whether the the probability of all state-action outcomes sum to 1 in he provided transition dynamics. Returns true if they do, false otherwise. If this method returns false, you can use the method to get a string reporting where errors are.

Parameters:

transitionDynamics - the graph transition dynamics to check

Returns:

true if all probability distributions sum to 1; false otherwise

invalidMDPReport

public java.lang.String invalidMDPReport()

Returns a string that lists the state-action paris that have improper transition dynamics (transitions that don't sum to 1). If there are no improper dynamics, then the returned string is empty. The output form is:

(s, a): sumProbability

(s, a)->s_1' p_1

(s, a)->s_2' p_2

...

Returns:

a string that lists the state-action paris that have improper transition dynamics or an empty string if all transitions are proper.

invalidMDPReport

public static java.lang.String invalidMDPReport(java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>> transitionDynamics)

Returns a string that lists the state-action paris that have improper transition dynamics (transitions that don't sum to 1). If there are no improper dynamics, then the returned string is empty. The output form is:

(s, a): sumProbability

(s, a)->s_1' p_1

(s, a)->s_2' p_2

...

Parameters:

transitionDynamics - the transition dynamics of the MDP/Graph

Returns:

a string that lists the state-action paris that have improper transition dynamics or an empty string if all transitions are proper.

clearStateTransitionsFrom

public void clearStateTransitionsFrom(int srcNode)

Clears all transitions from a given state node

Parameters:

srcNode - the state node from which transitions will be cleared

clearStateActionTransitions

public void clearStateActionTransitions(int srcNode,
                                        int action)

Clears all (stochastic) edges for a given state-action pair.

Parameters:

srcNode - The state node of the pair to clear

action - the action of the pair to clear

removeEdge

public void removeEdge(int srcNode,
                       int action,
                       int tNode)

Removes a given edge from the transition dynamics.

Parameters:

srcNode - the source state node of the edge

action - the action in the state node of the edge

tNode - the target state node of the dge

getNodeTransitionTo

protected GraphDefinedDomain.NodeTransitionProbability getNodeTransitionTo(java.util.Set<GraphDefinedDomain.NodeTransitionProbability> nts,
                                                                           int tNode)

Returns the GraphDefinedDomain.NodeTransitionProbability object in the provided set that corresponds to a transition to state tNode or null if it doesn't exist.

Parameters:

nts - the set of possible node transitions

tNode - the query transition node id

Returns:

a GraphDefinedDomain.NodeTransitionProbability or null if the transition to the state does not already exist.

copyTransitionDynamics

protected java.util.Map<java.lang.Integer,java.util.Map<java.lang.Integer,java.util.Set<GraphDefinedDomain.NodeTransitionProbability>>> copyTransitionDynamics()

Returns a deep copy of the transition dynamics

Returns:

a deep copy of the transition dynamics

generateDomain

public SADomain generateDomain()

Description copied from interface: DomainGenerator

Returns a newly instanced Domain object

Specified by:

generateDomain in interface DomainGenerator

Returns:

the newly instantiated Domain object.

main

public static void main(java.lang.String[] args)