burlap.behavior.singleagent.learning.lspi

Class SARSCollector

java.lang.Object

burlap.behavior.singleagent.learning.lspi.SARSCollector

Direct Known Subclasses:

SARSCollector.UniformRandomSARSCollector

public abstract class SARSCollector
extends java.lang.Object

This object is used to collected SARSData (state-action-reard-state tuples) that can then be used by algorithms like LSPI for learning.

Author:

James MacGlashan

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SARSCollector.UniformRandomSARSCollector Collects SARS data from source states generated by a StateGenerator by choosing actions uniformly at random.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<Action>	actions The actions used for collecting data.

Constructor Summary

Constructors

Constructor and Description
SARSCollector(Domain domain) Initializes the collector's action set using the actions that are part of the domain.
SARSCollector(java.util.List<Action> actions) Initializes this collector's action set to use for collecting data.

Method Summary

Methods

Modifier and Type	Method and Description
abstract SARSData	collectDataFrom(Environment env, int maxSteps, SARSData intoDataset) Collects data from an Environment's current state until either the maximum number of steps is taken or a terminal state is reached.
abstract SARSData	collectDataFrom(State s, RewardFunction rf, int maxSteps, TerminalFunction tf, SARSData intoDataset) Collects data from an initial state until either a terminal state is reached or until the maximum number of steps is taken.
SARSData	collectNInstances(Environment env, int nSamples, int maxEpisodeSteps, SARSData intoDataset) Collects nSamples of SARS tuples from an Environment and returns it in a SARSData object.
SARSData	collectNInstances(StateGenerator sg, RewardFunction rf, int nSamples, int maxEpisodeSteps, TerminalFunction tf, SARSData intoDataset) Collects nSamples of SARS tuples and returns it in a SARSData object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

actions

protected java.util.List<Action> actions

The actions used for collecting data.

Constructor Detail

SARSCollector

public SARSCollector(Domain domain)

Initializes the collector's action set using the actions that are part of the domain.

Parameters:

domain - the domain containing the actions to use

SARSCollector

public SARSCollector(java.util.List<Action> actions)

Initializes this collector's action set to use for collecting data.

Parameters:

actions - the action set to use for collecting data.

Method Detail

collectDataFrom

public abstract SARSData collectDataFrom(State s,
                       RewardFunction rf,
                       int maxSteps,
                       TerminalFunction tf,
                       SARSData intoDataset)

Collects data from an initial state until either a terminal state is reached or until the maximum number of steps is taken. Data is stored into the dataset intoDataset and returned. If intoDataset is null, then it is first created.

Parameters:

s - the initial state from which data should be collected.

rf - the reward function from which rewards are tracks.

maxSteps - the maximum number of steps that can be taken.

tf - the terminal function that ends data collection.

intoDataset - the dataset into which data will be stored. If null, a dataset is created.

Returns:

the resulting dataset with the newly collected data.

collectDataFrom

public abstract SARSData collectDataFrom(Environment env,
                       int maxSteps,
                       SARSData intoDataset)

Collects data from an Environment's current state until either the maximum number of steps is taken or a terminal state is reached. Data is stored into the dataset intoDataset and returned. If intoDataset is null, then it is first created.

Parameters:

env - the Environment from which data will be collected.

maxSteps - the maximum number of steps to take in the environment.

intoDataset - the dataset into which data will be stored. If null, a dataset is created.

Returns:

the resulting dataset with the newly collected data.

collectNInstances

public SARSData collectNInstances(StateGenerator sg,
                         RewardFunction rf,
                         int nSamples,
                         int maxEpisodeSteps,
                         TerminalFunction tf,
                         SARSData intoDataset)

Collects nSamples of SARS tuples and returns it in a SARSData object.

Parameters:

sg - a state generator for finding initial state from which data can be collected.

rf - the reward function that defines the reward received.

nSamples - the number of SARS samples to collect.

maxEpisodeSteps - the maximum number of steps that can be taken when rolling out from a state generated by StateGenerator sg, before a new rollout is started.

tf - the terminal function that caused a rollout to stop and a new state to be generated from StateGenerator sg.

intoDataset - the dataset into which the results will be collected. If null, a new dataset is created.

Returns:

the intoDataset object, which is created if it is input as null.

collectNInstances

public SARSData collectNInstances(Environment env,
                         int nSamples,
                         int maxEpisodeSteps,
                         SARSData intoDataset)

Collects nSamples of SARS tuples from an Environment and returns it in a SARSData object. Each sequence of samples is no longer than maxEpisodeSteps and samples are collected using this object's collectDataFrom(burlap.oomdp.singleagent.environment.Environment, int, SARSData) method. After each call to collectDataFrom(burlap.oomdp.singleagent.environment.Environment, int, SARSData), the provided Environment is sent the Environment.resetEnvironment() message.

Parameters:

env - The Environment from which samples should be collected.

nSamples - The number of samples to generate.

maxEpisodeSteps - the maximum number of steps to take from any initial state of the Environment.

intoDataset - the dataset into which the results will be collected. If null, a new dataset is created.

Returns:

the intoDataset object, which is created if it is input as null.