SimulatedEnvironment

java.lang.Object
- burlap.mdp.singleagent.environment.SimulatedEnvironment

All Implemented Interfaces:

Environment, EnvironmentServerInterface, StateSettableEnvironment

Direct Known Subclasses:

SimulatedPOEnvironment
```
public class SimulatedEnvironment
extends java.lang.Object
implements StateSettableEnvironment, EnvironmentServerInterface
```
An Environment that simulates interactions using a SampleModel that is provided in an input domain. Initial states of the environment are defined using a StateGenerator. If no StateGenerator is specified, but an initial State is provided in a constructor, then the StateGenerator is set to a ConstantStateGenerator so that upon resetEnvironment() method calls, the initial state is the same as the original input state.
All returned environment observations are fully observable returning a copy of the true internal State of the environment. Copies of the state are returned to prevent tampering of the internal environment state.
By default, this Environment will not allow states to change when the current environment state is a terminal state (as specified by the input TerminalFunction); instead, the same current state will be returned with a reward of zero if someone attempts to interact with the environment through executeAction(Action). In this case, the environment state will have to be manually changed with resetEnvironment() or setCurStateTo(State) to a non-terminal state before actions will affect the state again. Alternatively, you can allow actions to affect the state from terminal states with the setAllowActionFromTerminalStates(boolean) method.

Author:

James MacGlashan.

Field Summary

Fields
Modifier and Type	Field and Description
`protected boolean`	`allowActionFromTerminalStates` A flag indicating whether the environment will respond to actions from a terminal state.
`protected State`	`curState` The current state of the environment
`protected double`	`lastReward` The last reward generated from this environment.
`protected SampleModel`	`model`
`protected java.util.List<EnvironmentObserver>`	`observers` The `EnvironmentObserver` objects that will be notified of `Environment` events.
`protected StateGenerator`	`stateGenerator` The state generator used to generate new states when the environment is reset with `resetEnvironment()`;
`protected boolean`	`terminated`

Constructor Summary

Constructors
Constructor and Description
`SimulatedEnvironment(SADomain domain)`
`SimulatedEnvironment(SADomain domain, State initialState)`
`SimulatedEnvironment(SADomain domain, StateGenerator stateGenerator)`
`SimulatedEnvironment(SampleModel model)`
`SimulatedEnvironment(SampleModel model, State initialState)`
`SimulatedEnvironment(SampleModel model, StateGenerator stateGenerator)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addObservers(EnvironmentObserver... observers)` Adds one or more `EnvironmentObserver`s
`void`	`clearAllObservers()` Clears all `EnvironmentObserver`s from this server.
`State`	`currentObservation()` Returns the current observation of the environment as a `State`.
`EnvironmentOutcome`	`executeAction(Action a)` Executes the specified action in this environment
`StateGenerator`	`getStateGenerator()`
`boolean`	`isInTerminalState()` Returns whether the environment is in a terminal state that prevents further action by the agent.
`double`	`lastReward()` Returns the last reward returned by the environment
`java.util.List<EnvironmentObserver>`	`observers()` Returns all `EnvironmentObserver`s registered with this server.
`void`	`removeObservers(EnvironmentObserver... observers)` Removes one or more `EnvironmentObserver`s from this server.
`void`	`resetEnvironment()` Resets this environment to some initial state, if the functionality exists.
`void`	`setAllowActionFromTerminalStates(boolean allowActionFromTerminalStates)` Sets whether the environment will respond to actions from a terminal state.
`void`	`setCurStateTo(State s)` Sets the current state of the environment to the specified state.
`void`	`setStateGenerator(StateGenerator stateGenerator)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - model
```
protected SampleModel model
```
  - stateGenerator
```
protected StateGenerator stateGenerator
```
    The state generator used to generate new states when the environment is reset with resetEnvironment();
  - curState
```
protected State curState
```
    The current state of the environment
  - lastReward
```
protected double lastReward
```
    The last reward generated from this environment.
  - terminated
```
protected boolean terminated
```
  - allowActionFromTerminalStates
```
protected boolean allowActionFromTerminalStates
```
    A flag indicating whether the environment will respond to actions from a terminal state. If false, then once a the environment transitions to a terminal state, any action attempted by the executeAction(Action) method will result in no change in state and to enable action again, the Environment state will have to be manually changed with the resetEnvironment() method or the setCurStateTo(State) method. If this value is true, then actions will be carried out according to the domain's transition dynamics.
  - observers
```
protected java.util.List<EnvironmentObserver> observers
```
    The EnvironmentObserver objects that will be notified of Environment events.
- Constructor Detail
  - SimulatedEnvironment
```
public SimulatedEnvironment(SADomain domain)
```
  - SimulatedEnvironment
```
public SimulatedEnvironment(SADomain domain,
                            State initialState)
```
  - SimulatedEnvironment
```
public SimulatedEnvironment(SADomain domain,
                            StateGenerator stateGenerator)
```
  - SimulatedEnvironment
```
public SimulatedEnvironment(SampleModel model)
```
  - SimulatedEnvironment
```
public SimulatedEnvironment(SampleModel model,
                            State initialState)
```
  - SimulatedEnvironment
```
public SimulatedEnvironment(SampleModel model,
                            StateGenerator stateGenerator)
```
- Method Detail
  - getStateGenerator
```
public StateGenerator getStateGenerator()
```
  - setStateGenerator
```
public void setStateGenerator(StateGenerator stateGenerator)
```
  - addObservers
```
public void addObservers(EnvironmentObserver... observers)
```
    Description copied from interface: EnvironmentServerInterface
    
    Adds one or more EnvironmentObservers
    
    Specified by:
    
    addObservers in interface EnvironmentServerInterface
    
    Parameters:
    
    observers - and EnvironmentObserver
  - clearAllObservers
```
public void clearAllObservers()
```
    Description copied from interface: EnvironmentServerInterface
    
    Clears all EnvironmentObservers from this server.
    
    Specified by:
    
    clearAllObservers in interface EnvironmentServerInterface
  - removeObservers
```
public void removeObservers(EnvironmentObserver... observers)
```
    Description copied from interface: EnvironmentServerInterface
    
    Removes one or more EnvironmentObservers from this server.
    
    Specified by:
    
    removeObservers in interface EnvironmentServerInterface
    
    Parameters:
    
    observers - the EnvironmentObservers to remove.
  - observers
```
public java.util.List<EnvironmentObserver> observers()
```
    Description copied from interface: EnvironmentServerInterface
    
    Returns all EnvironmentObservers registered with this server.
    
    Specified by:
    
    observers in interface EnvironmentServerInterface
    
    Returns:
    
    all EnvironmentObservers registered with this server.
  - setAllowActionFromTerminalStates
```
public void setAllowActionFromTerminalStates(boolean allowActionFromTerminalStates)
```
    Sets whether the environment will respond to actions from a terminal state. If false, then once a the environment transitions to a terminal state, any action attempted by the executeAction(Action) method will result in no change in state and to enable action again, the Environment state will have to be manually changed with the resetEnvironment() method or the setCurStateTo(State) method. If this value is true, then actions will be carried out according to the domain's transition dynamics.
    
    Parameters:
    
    allowActionFromTerminalStates - if false, then actions are not allowed from terminal states; if true, then they are allowed.
  - setCurStateTo
```
public void setCurStateTo(State s)
```
    Description copied from interface: StateSettableEnvironment
    
    Sets the current state of the environment to the specified state.
    
    Specified by:
    
    setCurStateTo in interface StateSettableEnvironment
    
    Parameters:
    
    s - the state to which this Environment will be set.
  - currentObservation
```
public State currentObservation()
```
    Description copied from interface: Environment
    
    Returns the current observation of the environment as a State.
    
    Specified by:
    
    currentObservation in interface Environment
    
    Returns:
    
    the current observation of the environment as a State.
  - executeAction
```
public EnvironmentOutcome executeAction(Action a)
```
    Description copied from interface: Environment
    
    Executes the specified action in this environment
    
    Specified by:
    
    executeAction in interface Environment
    
    Parameters:
    
    a - the Action that is to be performed in this environment.
    
    Returns:
    
    the resulting observation and reward transition from applying the given GroundedAction in this environment.
  - lastReward
```
public double lastReward()
```
    Description copied from interface: Environment
    
    Returns the last reward returned by the environment
    
    Specified by:
    
    lastReward in interface Environment
    
    Returns:
    
    the last reward returned by the environment
  - isInTerminalState
```
public boolean isInTerminalState()
```
    Description copied from interface: Environment
    
    Returns whether the environment is in a terminal state that prevents further action by the agent.
    
    Specified by:
    
    isInTerminalState in interface Environment
    
    Returns:
    
    true if the current environment is in a terminal state; false otherwise.
  - resetEnvironment
```
public void resetEnvironment()
```
    Description copied from interface: Environment
    
    Resets this environment to some initial state, if the functionality exists.
    
    Specified by:
    
    resetEnvironment in interface Environment

Class SimulatedEnvironment

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

model

stateGenerator

curState

lastReward

terminated

allowActionFromTerminalStates

observers

Constructor Detail

SimulatedEnvironment

SimulatedEnvironment

SimulatedEnvironment

SimulatedEnvironment

SimulatedEnvironment

SimulatedEnvironment

Method Detail

getStateGenerator

setStateGenerator

addObservers

clearAllObservers

removeObservers

observers

setAllowActionFromTerminalStates

setCurStateTo

currentObservation

executeAction

lastReward

isInTerminalState

resetEnvironment