MultiAgentPerformancePlotter

java.lang.Object
- java.awt.Component
- - java.awt.Container
  - - java.awt.Window
    - - java.awt.Frame
      - javax.swing.JFrame
        
        burlap.behavior.stochasticgames.auxiliary.performance.MultiAgentPerformancePlotter

All Implemented Interfaces:

WorldObserver, java.awt.image.ImageObserver, java.awt.MenuContainer, java.io.Serializable, javax.accessibility.Accessible, javax.swing.RootPaneContainer, javax.swing.WindowConstants
```
public class MultiAgentPerformancePlotter
extends javax.swing.JFrame
implements WorldObserver
```
This class is a world observer used for recording and plotting the performance of the agents in the world. There are six possible performance metrics that can be used to plot performance, as specified in the PerformanceMetric enumerator. A plot showing the most recent "trial" of the agents can be displayed, the average of the metric over all trials with confidence intervals, or both may be displayed; which plots are shown is specfied by the TrialMode enumerator. Any subset of these metrics may be displayed in any order specified by the user and plots are displayed in a matrix format with a maximum number of columns that are filled out first. If the number of plots would cause a window height larger than a maximimum specified, then the plots are placed in a scroll view.
The way this class should be used is first the constructor should be called, then the startGUI() method. At the start of each trial, the startNewTrial() method should be called (although it is unncessary to call this method if it is the *first* trial since that will automatically be created.) A world object should then be created with this object attached to it. The world should then be run for as many episodes as desired until the next trial when things repeat. After all trials are complete, the startNewTrial() method should be called.
To ensure proper use of this class, it is highly recommended that the MultiAgentExperimenter class is used, because it will perform all the necessary steps for you.
When testing is done, you may optionally request all data to be printed to CSV files. One CSV file will produce the step-wise performance metric (cumulaitve reward by step) for all agents and trials. Another will produce all the episode-wise performance metric data. This data can be produced regardless of which metrics you requested to be plotted.
Note that the plots that are created have a number of interactive options. Try right-clicking on them to see the list of things you can modfiy in the GUI.

Author:

James MacGlashan

See Also:

Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`protected class`	`MultiAgentPerformancePlotter.AgentDatasets` A datastructure for maintain the plot series data in the current agent
`protected class`	`MultiAgentPerformancePlotter.DatasetsAndTrials` A class for storing the tiral data and series datasets for a given agent.
`protected class`	`MultiAgentPerformancePlotter.MutableBoolean` A class for a mutable boolean
`protected class`	`MultiAgentPerformancePlotter.Trial` A datastructure for maintaining all the metric stats for a single trial.

Nested classes/interfaces inherited from class javax.swing.JFrame
javax.swing.JFrame.AccessibleJFrame

Nested classes/interfaces inherited from class java.awt.Frame
java.awt.Frame.AccessibleAWTFrame

Nested classes/interfaces inherited from class java.awt.Window
java.awt.Window.AccessibleAWTWindow, java.awt.Window.Type

Nested classes/interfaces inherited from class java.awt.Container
java.awt.Container.AccessibleAWTContainer

Nested classes/interfaces inherited from class java.awt.Component
java.awt.Component.AccessibleAWTComponent, java.awt.Component.BaselineResizeBehavior, java.awt.Component.BltBufferStrategy, java.awt.Component.FlipBufferStrategy

Field Summary

Fields
Modifier and Type	Field and Description
`protected java.util.Map<java.lang.String,MultiAgentPerformancePlotter.DatasetsAndTrials>`	`agentWiseData` Datastructure for maintaining data for each agent playing in the game.
`protected org.jfree.data.xy.XYSeriesCollection`	`colAER` All agent plot series for the the most recetent trial's average reward per episode
`protected org.jfree.data.xy.YIntervalSeriesCollection`	`colAERAvg` All agent plot series for the average of all trial's average reward per episode
`protected org.jfree.data.xy.XYSeriesCollection`	`colCER` All agent plot series for the the most recetent trial's cumulative reward per episode
`protected org.jfree.data.xy.YIntervalSeriesCollection`	`colCERAvg` All agent plot series for the average of all trial's cumulative reward per episode
`protected org.jfree.data.xy.XYSeriesCollection`	`colCSE` All agent plot series for the the most recetent trial's cumulative step per episode
`protected org.jfree.data.xy.YIntervalSeriesCollection`	`colCSEAvg` All agent plot series for the average of all trial's cumulative steps per episode
`protected org.jfree.data.xy.XYSeriesCollection`	`colCSR` All agent plot series for the the most recetent trial's cumulative reward per step
`protected org.jfree.data.xy.YIntervalSeriesCollection`	`colCSRAvg` All agent plot series for the average of all trial's cumulative reward per step
`protected boolean`	`collectData` Whether the data from action observations received should be recoreded or not.
`protected org.jfree.data.xy.XYSeriesCollection`	`colMER` All agent plot series for the the most recetent trial's median reward per episode
`protected org.jfree.data.xy.YIntervalSeriesCollection`	`colMERAvg` All agent plot series for the average of all trial's median reward per episode
`protected org.jfree.data.xy.XYSeriesCollection`	`colSE` All agent plot series for the most recetent trial's steps per episode
`protected org.jfree.data.xy.YIntervalSeriesCollection`	`colSEAvg` All agent plot series for the average of all trial's steps per episode
`protected int`	`curEpisode` the current episode that was recorded
`protected int`	`curTimeStep` the current time step that was recorded
`int`	`debugCode` The debug code used for debug printing.
`protected int`	`delay` the delay in milliseconds between which the charts are updated automatically
`protected boolean`	`freshStart` Indicates whether this observer has observed any outcomes yet.
`protected int`	`lastEpisode` The last episode at which the plot's series data was updated
`protected int`	`lastTimeStepUpdate` The last time step at which the plots' series data was updated
`protected java.util.Set<PerformanceMetric>`	`metricsSet` A set specifying the performance metrics that will be plotted
`protected boolean`	`needsClearing` Whether the current plots need their series data cleared for a new trial
`protected double`	`significance` the significance level used for confidence intervals.
`protected TerminalFunction`	`tf` Terminal funciton for determining when episodes have ended.
`protected TrialMode`	`trialMode` specifies whether the most recent trial, average of all trials, or both plots will be displayed
`protected MultiAgentPerformancePlotter.MutableBoolean`	`trialUpdateComplete` Synchronization object to ensure proper threaded plot updating
`protected World`	`world`

Fields inherited from class javax.swing.JFrame
accessibleContext, EXIT_ON_CLOSE, rootPane, rootPaneCheckingEnabled

Fields inherited from class java.awt.Frame
CROSSHAIR_CURSOR, DEFAULT_CURSOR, E_RESIZE_CURSOR, HAND_CURSOR, ICONIFIED, MAXIMIZED_BOTH, MAXIMIZED_HORIZ, MAXIMIZED_VERT, MOVE_CURSOR, N_RESIZE_CURSOR, NE_RESIZE_CURSOR, NORMAL, NW_RESIZE_CURSOR, S_RESIZE_CURSOR, SE_RESIZE_CURSOR, SW_RESIZE_CURSOR, TEXT_CURSOR, W_RESIZE_CURSOR, WAIT_CURSOR

Fields inherited from class java.awt.Component
BOTTOM_ALIGNMENT, CENTER_ALIGNMENT, LEFT_ALIGNMENT, RIGHT_ALIGNMENT, TOP_ALIGNMENT

Fields inherited from interface javax.swing.WindowConstants
DISPOSE_ON_CLOSE, DO_NOTHING_ON_CLOSE, HIDE_ON_CLOSE

Fields inherited from interface java.awt.image.ImageObserver
ABORT, ALLBITS, ERROR, FRAMEBITS, HEIGHT, PROPERTIES, SOMEBITS, WIDTH

Constructor Summary

Constructors
Constructor and Description
`MultiAgentPerformancePlotter(TerminalFunction tf, int chartWidth, int chartHeight, int columns, int maxWindowHeight, TrialMode trialMode, PerformanceMetric... metrics)` Initializes

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected static void`	`accumulate(java.util.List<java.lang.Double> list, double v)` Computes the sum of the last entry in list and the value v and adds it to the end of list.
`protected org.jfree.chart.renderer.xy.DeviationRenderer`	`createDeviationRenderer()` Creates a DeviationRenderer to use for the trial average plots
`void`	`endAllTrials()` Specifies that all trials are complete and that the average trial results and error bars should be plotted.
`protected void`	`endAllTrialsForAgent(java.lang.String agentName)` Ends all the trials, plotting the average trial data for the agent with the given name
`protected void`	`endAllTrialsHelper()` The end all trial methods helper called at the end of a swing update.
`protected void`	`endTrial()` Ends the current trial data and updates the plots accordingly.
`void`	`gameEnding(State s)` This method is called whenever a game in a world ends.
`void`	`gameStarting(State s)` This method is called whenever a new game in a world is starting.
`static double[]`	`getCI(org.apache.commons.math3.stat.descriptive.DescriptiveStatistics stats, double significanceLevel)` Returns the confidence interval for the specified significance level
`protected void`	`insertChart(java.awt.Container plotContainer, java.awt.GridBagConstraints c, int columns, int chartWidth, int chartHeight, java.lang.String title, java.lang.String xlab, java.lang.String ylab, org.jfree.data.xy.XYSeriesCollection mostRecentCollection, org.jfree.data.xy.YIntervalSeriesCollection averageCollection)` Adds the most recent trial (if enabled) chart and trial average (if enabled) chart into the provided container.
`protected void`	`launchThread()` Launches the automatic plot refresh thread.
`protected int[]`	`minStepAndEpisodes(java.util.List<MultiAgentPerformancePlotter.Trial> trials)` Returns the minimum steps and episodes across all trials
`void`	`observe(State s, JointAction ja, double[] reward, State sp)` This method is called whenever an interaction in the world occurs.
`void`	`setRefreshDelay(int delayInMS)` sets the delay in milliseconds between automatic refreshes of the plots
`void`	`setSignificanceForCI(double signifcance)` Sets the significance used for confidence intervals.
`void`	`setWorld(World world)`
`void`	`startGUI()` Launches the GUI and automatic refresh thread.
`void`	`startNewTrial()` Initializes the datastructures for a new trial.
`void`	`toggleDataCollection(boolean collectData)` Toggle whether performance data collected from the action observation is recorded or not
`protected void`	`updateAERSeris(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)` Updates the average reward by episode series.
`protected void`	`updateCERSeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)` Updates the cumulative reward by episode series.
`protected void`	`updateCSESeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)` Updates the cumulative steps by episode series.
`protected void`	`updateCSRSeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)` Updates the cumulative reward by step series.
`protected void`	`updateGBConstraint(java.awt.GridBagConstraints c, int maxCol)` Increments the x-y position of a constraint to the next position.
`protected void`	`updateMERSeris(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)` Updates the median reward by episode series.
`protected void`	`updateMostRecentSeriesHelper()` Updates the series data for the most recent trial plots.
`protected void`	`updateSESeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)` Updates the steps by episode series.
`protected void`	`updateTimeSeries()` Updates all the most recent trial time series with the latest data
`void`	`writeEpisodeDataToCSV(java.lang.String filePath)` Writes the episode-wise data to a csv file.
`void`	`writeStepAndEpisodeDataToCSV(java.lang.String pathAndBaseNameToUse)` Writes the step-wise and episode-wise data to CSV files.
`void`	`writeStepDataToCSV(java.lang.String filePath)` Writes the step-wise data to a csv file.

Methods inherited from class javax.swing.JFrame
addImpl, createRootPane, frameInit, getAccessibleContext, getContentPane, getDefaultCloseOperation, getGlassPane, getGraphics, getJMenuBar, getLayeredPane, getRootPane, getTransferHandler, isDefaultLookAndFeelDecorated, isRootPaneCheckingEnabled, paramString, processWindowEvent, remove, repaint, setContentPane, setDefaultCloseOperation, setDefaultLookAndFeelDecorated, setGlassPane, setIconImage, setJMenuBar, setLayeredPane, setLayout, setRootPane, setRootPaneCheckingEnabled, setTransferHandler, update

Methods inherited from class java.awt.Frame
addNotify, getCursorType, getExtendedState, getFrames, getIconImage, getMaximizedBounds, getMenuBar, getState, getTitle, isResizable, isUndecorated, remove, removeNotify, setBackground, setCursor, setExtendedState, setMaximizedBounds, setMenuBar, setOpacity, setResizable, setShape, setState, setTitle, setUndecorated

Methods inherited from class java.awt.Window
addPropertyChangeListener, addPropertyChangeListener, addWindowFocusListener, addWindowListener, addWindowStateListener, applyResourceBundle, applyResourceBundle, createBufferStrategy, createBufferStrategy, dispose, getBackground, getBufferStrategy, getFocusableWindowState, getFocusCycleRootAncestor, getFocusOwner, getFocusTraversalKeys, getIconImages, getInputContext, getListeners, getLocale, getModalExclusionType, getMostRecentFocusOwner, getOpacity, getOwnedWindows, getOwner, getOwnerlessWindows, getShape, getToolkit, getType, getWarningString, getWindowFocusListeners, getWindowListeners, getWindows, getWindowStateListeners, hide, isActive, isAlwaysOnTop, isAlwaysOnTopSupported, isAutoRequestFocus, isFocusableWindow, isFocusCycleRoot, isFocused, isLocationByPlatform, isOpaque, isShowing, isValidateRoot, pack, paint, postEvent, processEvent, processWindowFocusEvent, processWindowStateEvent, removeWindowFocusListener, removeWindowListener, removeWindowStateListener, reshape, setAlwaysOnTop, setAutoRequestFocus, setBounds, setBounds, setCursor, setFocusableWindowState, setFocusCycleRoot, setIconImages, setLocation, setLocation, setLocationByPlatform, setLocationRelativeTo, setMinimumSize, setModalExclusionType, setSize, setSize, setType, setVisible, show, toBack, toFront

Methods inherited from class java.awt.Container
add, add, add, add, add, addContainerListener, applyComponentOrientation, areFocusTraversalKeysSet, countComponents, deliverEvent, doLayout, findComponentAt, findComponentAt, getAlignmentX, getAlignmentY, getComponent, getComponentAt, getComponentAt, getComponentCount, getComponents, getComponentZOrder, getContainerListeners, getFocusTraversalPolicy, getInsets, getLayout, getMaximumSize, getMinimumSize, getMousePosition, getPreferredSize, insets, invalidate, isAncestorOf, isFocusCycleRoot, isFocusTraversalPolicyProvider, isFocusTraversalPolicySet, layout, list, list, locate, minimumSize, paintComponents, preferredSize, print, printComponents, processContainerEvent, remove, removeAll, removeContainerListener, setComponentZOrder, setFocusTraversalKeys, setFocusTraversalPolicy, setFocusTraversalPolicyProvider, setFont, transferFocusDownCycle, validate, validateTree

Methods inherited from class java.awt.Component
action, add, addComponentListener, addFocusListener, addHierarchyBoundsListener, addHierarchyListener, addInputMethodListener, addKeyListener, addMouseListener, addMouseMotionListener, addMouseWheelListener, bounds, checkImage, checkImage, coalesceEvents, contains, contains, createImage, createImage, createVolatileImage, createVolatileImage, disable, disableEvents, dispatchEvent, enable, enable, enableEvents, enableInputMethods, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, firePropertyChange, getBaseline, getBaselineResizeBehavior, getBounds, getBounds, getColorModel, getComponentListeners, getComponentOrientation, getCursor, getDropTarget, getFocusListeners, getFocusTraversalKeysEnabled, getFont, getFontMetrics, getForeground, getGraphicsConfiguration, getHeight, getHierarchyBoundsListeners, getHierarchyListeners, getIgnoreRepaint, getInputMethodListeners, getInputMethodRequests, getKeyListeners, getLocation, getLocation, getLocationOnScreen, getMouseListeners, getMouseMotionListeners, getMousePosition, getMouseWheelListeners, getName, getParent, getPeer, getPropertyChangeListeners, getPropertyChangeListeners, getSize, getSize, getTreeLock, getWidth, getX, getY, gotFocus, handleEvent, hasFocus, imageUpdate, inside, isBackgroundSet, isCursorSet, isDisplayable, isDoubleBuffered, isEnabled, isFocusable, isFocusOwner, isFocusTraversable, isFontSet, isForegroundSet, isLightweight, isMaximumSizeSet, isMinimumSizeSet, isPreferredSizeSet, isValid, isVisible, keyDown, keyUp, list, list, list, location, lostFocus, mouseDown, mouseDrag, mouseEnter, mouseExit, mouseMove, mouseUp, move, nextFocus, paintAll, prepareImage, prepareImage, printAll, processComponentEvent, processFocusEvent, processHierarchyBoundsEvent, processHierarchyEvent, processInputMethodEvent, processKeyEvent, processMouseEvent, processMouseMotionEvent, processMouseWheelEvent, removeComponentListener, removeFocusListener, removeHierarchyBoundsListener, removeHierarchyListener, removeInputMethodListener, removeKeyListener, removeMouseListener, removeMouseMotionListener, removeMouseWheelListener, removePropertyChangeListener, removePropertyChangeListener, repaint, repaint, repaint, requestFocus, requestFocus, requestFocusInWindow, requestFocusInWindow, resize, resize, revalidate, setComponentOrientation, setDropTarget, setEnabled, setFocusable, setFocusTraversalKeysEnabled, setForeground, setIgnoreRepaint, setLocale, setMaximumSize, setName, setPreferredSize, show, size, toString, transferFocus, transferFocusBackward, transferFocusUpCycle

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.awt.MenuContainer
getFont, postEvent

- Field Detail
  - tf
```
protected TerminalFunction tf
```
    Terminal funciton for determining when episodes have ended.
  - agentWiseData
```
protected java.util.Map<java.lang.String,MultiAgentPerformancePlotter.DatasetsAndTrials> agentWiseData
```
    Datastructure for maintaining data for each agent playing in the game.
  - colCSR
```
protected org.jfree.data.xy.XYSeriesCollection colCSR
```
    All agent plot series for the the most recetent trial's cumulative reward per step
  - colCER
```
protected org.jfree.data.xy.XYSeriesCollection colCER
```
    All agent plot series for the the most recetent trial's cumulative reward per episode
  - colAER
```
protected org.jfree.data.xy.XYSeriesCollection colAER
```
    All agent plot series for the the most recetent trial's average reward per episode
  - colMER
```
protected org.jfree.data.xy.XYSeriesCollection colMER
```
    All agent plot series for the the most recetent trial's median reward per episode
  - colCSE
```
protected org.jfree.data.xy.XYSeriesCollection colCSE
```
    All agent plot series for the the most recetent trial's cumulative step per episode
  - colSE
```
protected org.jfree.data.xy.XYSeriesCollection colSE
```
    All agent plot series for the most recetent trial's steps per episode
  - colCSRAvg
```
protected org.jfree.data.xy.YIntervalSeriesCollection colCSRAvg
```
    All agent plot series for the average of all trial's cumulative reward per step
  - colCERAvg
```
protected org.jfree.data.xy.YIntervalSeriesCollection colCERAvg
```
    All agent plot series for the average of all trial's cumulative reward per episode
  - colAERAvg
```
protected org.jfree.data.xy.YIntervalSeriesCollection colAERAvg
```
    All agent plot series for the average of all trial's average reward per episode
  - colMERAvg
```
protected org.jfree.data.xy.YIntervalSeriesCollection colMERAvg
```
    All agent plot series for the average of all trial's median reward per episode
  - colCSEAvg
```
protected org.jfree.data.xy.YIntervalSeriesCollection colCSEAvg
```
    All agent plot series for the average of all trial's cumulative steps per episode
  - colSEAvg
```
protected org.jfree.data.xy.YIntervalSeriesCollection colSEAvg
```
    All agent plot series for the average of all trial's steps per episode
  - metricsSet
```
protected java.util.Set<PerformanceMetric> metricsSet
```
    A set specifying the performance metrics that will be plotted
  - trialMode
```
protected TrialMode trialMode
```
    specifies whether the most recent trial, average of all trials, or both plots will be displayed
  - collectData
```
protected boolean collectData
```
    Whether the data from action observations received should be recoreded or not.
  - lastTimeStepUpdate
```
protected int lastTimeStepUpdate
```
    The last time step at which the plots' series data was updated
  - lastEpisode
```
protected int lastEpisode
```
    The last episode at which the plot's series data was updated
  - curTimeStep
```
protected int curTimeStep
```
    the current time step that was recorded
  - curEpisode
```
protected int curEpisode
```
    the current episode that was recorded
  - delay
```
protected int delay
```
    the delay in milliseconds between which the charts are updated automatically
  - significance
```
protected double significance
```
    the significance level used for confidence intervals. The default is 0.05 (corresponding to a 95% CI).
  - needsClearing
```
protected boolean needsClearing
```
    Whether the current plots need their series data cleared for a new trial
  - freshStart
```
protected boolean freshStart
```
    Indicates whether this observer has observed any outcomes yet.
  - trialUpdateComplete
```
protected MultiAgentPerformancePlotter.MutableBoolean trialUpdateComplete
```
    Synchronization object to ensure proper threaded plot updating
  - debugCode
```
public int debugCode
```
    The debug code used for debug printing.
  - world
```
protected World world
```
- Constructor Detail
  - MultiAgentPerformancePlotter
```
public MultiAgentPerformancePlotter(TerminalFunction tf,
                                    int chartWidth,
                                    int chartHeight,
                                    int columns,
                                    int maxWindowHeight,
                                    TrialMode trialMode,
                                    PerformanceMetric... metrics)
```
    Initializes
    
    Parameters:
    
    tf - the terminal function that will be used for detecting the end of episdoes
    
    chartWidth - the width of a cart
    
    chartHeight - the height of a chart
    
    columns - the number of columns of charts
    
    maxWindowHeight - the maximum window height until a scroll bar will be added
    
    trialMode - the kinds of trail data that will be displayed
    
    metrics - which metrics will be plotted.
- Method Detail
  - setWorld
```
public void setWorld(World world)
```
  - insertChart
```
protected void insertChart(java.awt.Container plotContainer,
                           java.awt.GridBagConstraints c,
                           int columns,
                           int chartWidth,
                           int chartHeight,
                           java.lang.String title,
                           java.lang.String xlab,
                           java.lang.String ylab,
                           org.jfree.data.xy.XYSeriesCollection mostRecentCollection,
                           org.jfree.data.xy.YIntervalSeriesCollection averageCollection)
```
    Adds the most recent trial (if enabled) chart and trial average (if enabled) chart into the provided container. The GridBagConstraints will aumatically be incremented to the next position after this method returns.
    
    Parameters:
    
    plotContainer - the contain in which to insert the plot(s).
    
    c - the current grid bag contraint locaiton in which the plots should be inserted.
    
    columns - the number of columns to fill in the plot container
    
    chartWidth - the width of any single plot
    
    chartHeight - the height of any single plot
    
    title - the title to label thep plot; if average trial plots are enabled the word "Average" will be prepended to the title for the average plot.
    
    xlab - the xlab axis of the plot
    
    ylab - the y lab axis of the plot
    
    mostRecentCollection - the XYSeriesCollection dataset with which the most recent trial plot is associated
    
    averageCollection - the YIntervalSeriesCollection dataset with which the trial average plot is associated
  - createDeviationRenderer
```
protected org.jfree.chart.renderer.xy.DeviationRenderer createDeviationRenderer()
```
    Creates a DeviationRenderer to use for the trial average plots
    
    Returns:
    
    a DeviationRenderer
  - updateGBConstraint
```
protected void updateGBConstraint(java.awt.GridBagConstraints c,
                                  int maxCol)
```
    Increments the x-y position of a constraint to the next position. If there are still free columns in the current row, then the next position in the next column; otherwise a new row is started.
    
    Parameters:
    
    c - the constraint to increment
    
    maxCol - the maximum columns allowable in a container
  - startGUI
```
public void startGUI()
```
    Launches the GUI and automatic refresh thread.
  - launchThread
```
protected void launchThread()
```
    Launches the automatic plot refresh thread.
  - setRefreshDelay
```
public void setRefreshDelay(int delayInMS)
```
    sets the delay in milliseconds between automatic refreshes of the plots
    
    Parameters:
    
    delayInMS - the refresh delay in milliseconds
  - setSignificanceForCI
```
public void setSignificanceForCI(double signifcance)
```
    Sets the significance used for confidence intervals. The default is 0.05, which corresponds to a 95% confidence interval.
    
    Parameters:
    
    signifcance - the significance used for confidence intervals.
  - toggleDataCollection
```
public void toggleDataCollection(boolean collectData)
```
    Toggle whether performance data collected from the action observation is recorded or not
    
    Parameters:
    
    collectData - true if data collected should be plotted; false if not.
  - gameStarting
```
public void gameStarting(State s)
```
    Description copied from interface: WorldObserver
    
    This method is called whenever a new game in a world is starting.
    
    Specified by:
    
    gameStarting in interface WorldObserver
    
    Parameters:
    
    s - the state in which the world is starting.
  - observe
```
public void observe(State s,
                    JointAction ja,
                    double[] reward,
                    State sp)
```
    Description copied from interface: WorldObserver
    
    This method is called whenever an interaction in the world occurs.
    
    Specified by:
    
    observe in interface WorldObserver
    
    Parameters:
    
    s - the previous state of the world
    
    ja - the joint action taken in the world
    
    reward - the joint reward received by the agents
    
    sp - the next state of the world
  - gameEnding
```
public void gameEnding(State s)
```
    Description copied from interface: WorldObserver
    
    This method is called whenever a game in a world ends.
    
    Specified by:
    
    gameEnding in interface WorldObserver
    
    Parameters:
    
    s - the final state of the world when it ends.
  - startNewTrial
```
public void startNewTrial()
```
    Initializes the datastructures for a new trial. Nothing will happen if no data has been recorded in the last trial. This method will automatically end the current trial, finializing all data before the new trial is started.
  - writeStepAndEpisodeDataToCSV
```
public void writeStepAndEpisodeDataToCSV(java.lang.String pathAndBaseNameToUse)
```
    Writes the step-wise and episode-wise data to CSV files. The episode-wise data will be saved to the file <pathAndBaseNameToUse>Episodes.csv. The step-wise data will be saved to the file <pathAndBaseNameToUse>Steps.csv
    
    Parameters:
    
    pathAndBaseNameToUse - the base path and file name for the episode-wise and step-wise csv files.
  - writeStepDataToCSV
```
public void writeStepDataToCSV(java.lang.String filePath)
```
    Writes the step-wise data to a csv file. If the file path does not include the .csv extension, it will automatically be added.
    
    Parameters:
    
    filePath - the path to the csv file to write to.
  - writeEpisodeDataToCSV
```
public void writeEpisodeDataToCSV(java.lang.String filePath)
```
    Writes the episode-wise data to a csv file. If the file path does not include the .csv extension, it will automatically be added.
    
    Parameters:
    
    filePath - the path to the csv file to write to.
  - endTrial
```
protected void endTrial()
```
    Ends the current trial data and updates the plots accordingly.
  - endAllTrials
```
public void endAllTrials()
```
    Specifies that all trials are complete and that the average trial results and error bars should be plotted.
  - endAllTrialsHelper
```
protected void endAllTrialsHelper()
```
    The end all trial methods helper called at the end of a swing update.
  - endAllTrialsForAgent
```
protected void endAllTrialsForAgent(java.lang.String agentName)
```
    Ends all the trials, plotting the average trial data for the agent with the given name
    
    Parameters:
    
    agentName - the name of the agent whose trial data will be completed and plotted
  - updateTimeSeries
```
protected void updateTimeSeries()
```
    Updates all the most recent trial time series with the latest data
  - updateMostRecentSeriesHelper
```
protected void updateMostRecentSeriesHelper()
```
    Updates the series data for the most recent trial plots.
  - minStepAndEpisodes
```
protected int[] minStepAndEpisodes(java.util.List<MultiAgentPerformancePlotter.Trial> trials)
```
    Returns the minimum steps and episodes across all trials
    
    Parameters:
    
    trials - the trials to perform the min over
    
    Returns:
    
    a double array of length 2; the first entry is the minimum steps, the second entry tthe minimum episodes
  - accumulate
```
protected static void accumulate(java.util.List<java.lang.Double> list,
                                 double v)
```
    Computes the sum of the last entry in list and the value v and adds it to the end of list. Use for maintainly cumulative data.
    
    Parameters:
    
    list - the list to add and append to.
    
    v - the value to add to the last value of list and append
  - getCI
```
public static double[] getCI(org.apache.commons.math3.stat.descriptive.DescriptiveStatistics stats,
                             double significanceLevel)
```
    Returns the confidence interval for the specified significance level
    
    Parameters:
    
    stats - the summary including the array of data for which the confidence interval is to be returned
    
    significanceLevel - the significance level required
    
    Returns:
    
    a double array of length three in the form: {mean, lowerBound, upperBound}
  - updateCSRSeries
```
protected void updateCSRSeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)
```
    Updates the cumulative reward by step series. Does nothing if that metric is not being plotted.
    
    Parameters:
    
    agentData - the agent dataset
  - updateCERSeries
```
protected void updateCERSeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)
```
    Updates the cumulative reward by episode series. Does nothing if that metric is not being plotted.
    
    Parameters:
    
    agentData - the agent dataset
  - updateAERSeris
```
protected void updateAERSeris(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)
```
    Updates the average reward by episode series. Does nothing if that metric is not being plotted.
    
    Parameters:
    
    agentData - the agent dataset
  - updateMERSeris
```
protected void updateMERSeris(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)
```
    Updates the median reward by episode series. Does nothing if that metric is not being plotted.
    
    Parameters:
    
    agentData - the agent dataset
  - updateCSESeries
```
protected void updateCSESeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)
```
    Updates the cumulative steps by episode series. Does nothing if that metric is not being plotted.
    
    Parameters:
    
    agentData - the agent dataset
  - updateSESeries
```
protected void updateSESeries(MultiAgentPerformancePlotter.DatasetsAndTrials agentData)
```
    Updates the steps by episode series. Does nothing if that metric is not being plotted.
    
    Parameters:
    
    agentData - the agent dataset

Class MultiAgentPerformancePlotter

Nested Class Summary

Nested classes/interfaces inherited from class javax.swing.JFrame

Nested classes/interfaces inherited from class java.awt.Frame

Nested classes/interfaces inherited from class java.awt.Window

Nested classes/interfaces inherited from class java.awt.Container

Nested classes/interfaces inherited from class java.awt.Component

Field Summary

Fields inherited from class javax.swing.JFrame

Fields inherited from class java.awt.Frame

Fields inherited from class java.awt.Component

Fields inherited from interface javax.swing.WindowConstants

Fields inherited from interface java.awt.image.ImageObserver

Constructor Summary

Method Summary

Methods inherited from class javax.swing.JFrame

Methods inherited from class java.awt.Frame

Methods inherited from class java.awt.Window

Methods inherited from class java.awt.Container

Methods inherited from class java.awt.Component

Methods inherited from class java.lang.Object

Methods inherited from interface java.awt.MenuContainer

Field Detail

tf

agentWiseData

colCSR

colCER

colAER

colMER

colCSE

colSE

colCSRAvg

colCERAvg

colAERAvg

colMERAvg

colCSEAvg

colSEAvg

metricsSet

trialMode

collectData

lastTimeStepUpdate

lastEpisode

curTimeStep

curEpisode

delay

significance

needsClearing

freshStart

trialUpdateComplete

debugCode

world

Constructor Detail

MultiAgentPerformancePlotter

Method Detail

setWorld

insertChart

createDeviationRenderer

updateGBConstraint

startGUI

launchThread

setRefreshDelay

setSignificanceForCI

toggleDataCollection

gameStarting

observe

gameEnding

startNewTrial

writeStepAndEpisodeDataToCSV

writeStepDataToCSV

writeEpisodeDataToCSV

endTrial

endAllTrials

endAllTrialsHelper

endAllTrialsForAgent

updateTimeSeries

updateMostRecentSeriesHelper

minStepAndEpisodes

accumulate

getCI

updateCSRSeries