burlap.behavior.singleagent.auxiliary.performance

Class LearningAlgorithmExperimenter

java.lang.Object

burlap.behavior.singleagent.auxiliary.performance.LearningAlgorithmExperimenter

public class LearningAlgorithmExperimenter
extends java.lang.Object

This class is used to simplify the comparison of different learning algorithms. It takes as input a test Environment in which to perform the experiments, a number of trials, the length of the trials, and an array of learning agent factories used to generated agent instances and compare their performance. The Environment may optionally implement the ExperimentalEnvironment interface which will let this class to tell the Environment whenever experiments with a new agent class (defined by an LearningAgentFactory is begun). The length of the trials by default is assumed to be in episodes, but it may also be changed to indicate length in total number of steps using the toggleTrialLengthInterpretation(boolean) method.

Performance results are displayed in plots using the PerformancePlotter class, but visualization may also be disabled with the toggleVisualPlots(boolean) method. Results may be saved to csv files after the experiment is complete.

The purpose of the experimenter is to test an agent for a specified number of trials. At the beginning of each trial, a new agent is generated using the designated LearningAgentFactory and is used for the specified trial length. After all trials are complete for an agent, the next agent is tested. Note that immediately before an agent is generated from an agent factory, the performance plotter is temporarily frozen from collecting data until the new agent is returned. This allows agent factories to perform offline learning before returning a new agent in the same domain without affecting the experimenter results.

By default the cumulative reward per step will be plotted and if more than one trial is specified, the both the most recent trail and the trial average plot will be shown. If only one trial is specified, then only the most recent trial plot will be shown. To control the kinds of plots displayed use the setUpPlottingConfiguration(int, int, int, int, TrialMode, PerformanceMetric...) method.

Author:

James MacGlashan

Field Summary

Fields

Modifier and Type	Field and Description
protected LearningAgentFactory[]	agentFactories The array of agent factories for the agents to be compared.
protected boolean	completedExperiment Whether the experimenter has completed.
int	debugCode The debug code used for debug printing.
protected boolean	displayPlots Whether the performance should be visually plotted (by default they will)
protected EnvironmentServer	environmentSever The EnvironmentServer that wraps the test Environment and tells a PerformancePlotter about the individual interactions.
protected int	nTrials The number of trials that each agent is evaluated
protected double	plotCISignificance The signficance value for the confidence interval in the plots.
protected int	plotRefresh The delay in milliseconds between autmatic refreshes of the plots
protected PerformancePlotter	plotter The PerformancePlotter used to collect and plot results
protected Environment	testEnvironment The test Environment in which experiments will be performed.
protected int	trialLength The length of each trial
protected boolean	trialLengthIsInEpisodes Whether the trial length specifies a number of episodes (which is the default) or the total number of steps

Constructor Summary

Constructors

Constructor and Description
LearningAlgorithmExperimenter(Environment testEnvironment, int nTrials, int trialLength, LearningAgentFactory... agentFactories) Initializes.

Method Summary

Modifier and Type	Method and Description
protected void	runEpisodeBoundTrial(LearningAgentFactory agentFactory) Runs a trial for an agent generated by the given factory when interpreting trial length as a number of episodes.
protected void	runStepBoundTrial(LearningAgentFactory agentFactory) Runs a trial for an agent generated by the given factor when interpreting trial length as a number of total steps.
void	setPlotCISignificance(double significance) Sets the significance used for confidence intervals.
void	setPlotRefreshDelay(int delayInMS) Sets the delay in milliseconds between automatic plot refreshes
void	setUpPlottingConfiguration(int chartWidth, int chartHeight, int columns, int maxWindowHeight, TrialMode trialMode, PerformanceMetric... metrics) Setsup the plotting confiruation.
void	startExperiment() Starts the experiment and runs all trails for all agents.
void	toggleTrialLengthInterpretation(boolean lengthRepresentsEpisodes) Changes whether the trial length provided in the constructor is interpreted as the number of episodes or total number of steps.
void	toggleVisualPlots(boolean shouldPlotResults) Toggles whether plots should be displayed or not.
void	writeEpisodeDataToCSV(java.lang.String filePath) Writes an step-wise data to a csv file.
void	writeStepAndEpisodeDataToCSV(java.lang.String pathAndBaseNameToUse) Writes the step-wise and episode-wise data to CSV files.
void	writeStepDataToCSV(java.lang.String filePath) Writes an episode-wise data to a csv file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

testEnvironment

protected Environment testEnvironment

The test Environment in which experiments will be performed.

environmentSever

protected EnvironmentServer environmentSever

The EnvironmentServer that wraps the test Environment and tells a PerformancePlotter about the individual interactions.

agentFactories

protected LearningAgentFactory[] agentFactories

The array of agent factories for the agents to be compared.

nTrials

protected int nTrials

The number of trials that each agent is evaluated

trialLength

protected int trialLength

The length of each trial

trialLengthIsInEpisodes

protected boolean trialLengthIsInEpisodes

Whether the trial length specifies a number of episodes (which is the default) or the total number of steps

plotter

protected PerformancePlotter plotter

The PerformancePlotter used to collect and plot results

displayPlots

protected boolean displayPlots

Whether the performance should be visually plotted (by default they will)

plotRefresh

protected int plotRefresh

The delay in milliseconds between autmatic refreshes of the plots

plotCISignificance

protected double plotCISignificance

The signficance value for the confidence interval in the plots. The default is 0.05 which correspodns to a 95% CI

completedExperiment

protected boolean completedExperiment

Whether the experimenter has completed.

debugCode

public int debugCode

The debug code used for debug printing. This experimenter will print with the debugger the number of trials completed for each agent.

Constructor Detail

LearningAlgorithmExperimenter

public LearningAlgorithmExperimenter(Environment testEnvironment,
                                     int nTrials,
                                     int trialLength,
                                     LearningAgentFactory... agentFactories)

Initializes. The trialLength will be interpreted as the number of episodes, but it can be reinterpreted as a total number of steps per trial using the toggleTrialLengthInterpretation(boolean).

Parameters:

testEnvironment - the test Environment in which experiments will be performed.

nTrials - the number of trials

trialLength - the length of the trials (by default in episodes, but can be intereted as maximum step length)

agentFactories - factories to generate the agents to be tested.

Method Detail

setUpPlottingConfiguration

public void setUpPlottingConfiguration(int chartWidth,
                                       int chartHeight,
                                       int columns,
                                       int maxWindowHeight,
                                       TrialMode trialMode,
                                       PerformanceMetric... metrics)

Setsup the plotting confiruation.

Parameters:

chartWidth - the width of each chart/plot

chartHeight - the height of each chart//plot

columns - the number of columns of the plots displayed. Plots are filled in columns first, then move down the next row.

maxWindowHeight - the maximum window height allowed before a scroll view is used.

trialMode - which plots to use; most recent trial, average over all trials, or both. If both, the most recent plot will be inserted into the window first, then the average.

metrics - the metrics that should be plotted. The metrics will appear in the window in the order that they are specified (columns first)

setPlotRefreshDelay

public void setPlotRefreshDelay(int delayInMS)

Sets the delay in milliseconds between automatic plot refreshes

Parameters:

delayInMS - the delay in milliseconds

setPlotCISignificance

public void setPlotCISignificance(double significance)

Sets the significance used for confidence intervals. The default is 0.05 which corresponds to a 95% CI.

Parameters:

significance - the significance for confidence intervals to use

toggleVisualPlots

public void toggleVisualPlots(boolean shouldPlotResults)

Toggles whether plots should be displayed or not.

Parameters:

shouldPlotResults - if true, then plots will be displayed; if false plots will not be displayed.

toggleTrialLengthInterpretation

public void toggleTrialLengthInterpretation(boolean lengthRepresentsEpisodes)

Changes whether the trial length provided in the constructor is interpreted as the number of episodes or total number of steps.

Parameters:

lengthRepresentsEpisodes - if true, interpret length as number of episodes; if false interprete as total number of steps.

startExperiment

public void startExperiment()

Starts the experiment and runs all trails for all agents.

writeStepAndEpisodeDataToCSV

public void writeStepAndEpisodeDataToCSV(java.lang.String pathAndBaseNameToUse)

Writes the step-wise and episode-wise data to CSV files. The episode-wise data will be saved to the file <pathAndBaseNameToUse>Episodes.csv. The step-wise data will. If the experimenter as not been run, then nothing will be saved and a warning message will be printed to indicate as such. be saved to the file <pathAndBaseNameToUse>Steps.csv

Parameters:

pathAndBaseNameToUse - the base path and file name for the episode-wise and step-wise csv files.

writeStepDataToCSV

public void writeStepDataToCSV(java.lang.String filePath)

Writes an episode-wise data to a csv file. If the file path does not include the .csv extension, it will automatically be added. If the experimenter as not been run, then nothing will be saved and a warrning message will be printed to indicate as such.

Parameters:

filePath - the path to the csv file to write to.

writeEpisodeDataToCSV

public void writeEpisodeDataToCSV(java.lang.String filePath)

Writes an step-wise data to a csv file. If the file path does not include the .csv extension, it will automatically be added. If the experimenter as not been run, then nothing will be saved and a warrning message will be printed to indicate as such.

Parameters:

filePath - the path to the csv file to write to.

runEpisodeBoundTrial

protected void runEpisodeBoundTrial(LearningAgentFactory agentFactory)

Runs a trial for an agent generated by the given factory when interpreting trial length as a number of episodes.

Parameters:

agentFactory - the agent factory used to generate the agent to test.

runStepBoundTrial

protected void runStepBoundTrial(LearningAgentFactory agentFactory)

Runs a trial for an agent generated by the given factor when interpreting trial length as a number of total steps.

Parameters:

agentFactory - the agent factory used to generate the agent to test.