public class ExponentialDecayLR extends java.lang.Object implements LearningRate
Modifier and Type  Class and Description 

protected class 
ExponentialDecayLR.MutableDouble
A class for storing a mutable double value object

protected class 
ExponentialDecayLR.StateWiseLearningRate
A class for storing a learning rate for a state, or a learning rate for each action for a given state

Modifier and Type  Field and Description 

protected double 
decayRate
The exponential base by which the learning rate is decayed

protected java.util.Map<java.lang.Integer,ExponentialDecayLR.StateWiseLearningRate> 
featureWiseMap
The state feature dependent or state featureaction dependent learning rates

protected HashableStateFactory 
hashingFactory
How to hash and perform equality checks of states

protected double 
initialLearningRate
The initial learning rate value

protected int 
lastPollTime
The last agent time at which they polled the learning rate

protected double 
minimumLR
The minimum learning rate

protected java.util.Map<HashableState,ExponentialDecayLR.StateWiseLearningRate> 
stateWiseMap
The state dependent or stateaction dependent learning rates

protected double 
universalLR
The state independent learning rate

protected boolean 
useStateActionWise
Whether the learning rate is dependent on stateactions

protected boolean 
useStateWise
Whether the learning rate is dependent on the state

Constructor and Description 

ExponentialDecayLR(double initialLearningRate,
double decayRate)
Initializes with an initial learning rate and decay rate for a state independent learning rate.

ExponentialDecayLR(double initialLearningRate,
double decayRate,
double minimumLearningRate)
Initializes with an initial learning rate and decay rate for a state independent learning rate that will decay to a value no smaller than minimumLearningRate

ExponentialDecayLR(double initialLearningRate,
double decayRate,
double minimumLearningRate,
HashableStateFactory hashingFactory,
boolean useSeparateLRPerStateAction)
Initializes with an initial learning rate and decay rate for a state or stateaction (or state featureaction) dependent learning rate that will decay to a value no smaller than minimumLearningRate
If this learning rate function is to be used for state state features, rather than states,
then the hashing factory can be null;

ExponentialDecayLR(double initialLearningRate,
double decayRate,
HashableStateFactory hashingFactory,
boolean useSeparateLRPerStateAction)
Initializes with an initial learning rate and decay rate for a state or stateaction (or state featureaction) dependent learning rate.

Modifier and Type  Method and Description 

protected ExponentialDecayLR.StateWiseLearningRate 
getFeatureWiseLearningRate(int feature)
Returns the learning rate data structure for the given state feature.

protected ExponentialDecayLR.StateWiseLearningRate 
getStateWiseLearningRate(State s)
Returns the learning rate data structure for the given state.

protected double 
nextLRVal(double cur)
Returns the value of an input current learning rate after it has been decayed by one time step.

double 
peekAtLearningRate(int featureId)
A method for looking at the current learning rate for a state (action) feature without having it altered.

double 
peekAtLearningRate(State s,
Action ga)
A method for looking at the current learning rate for a stateaction pair without having it altered.

double 
pollLearningRate(int agentTime,
int featureId)
A method for returning the learning rate for a given state (action) feature and then decaying the learning rate as defined by this class.

double 
pollLearningRate(int agentTime,
State s,
Action ga)
A method for returning the learning rate for a given state action pair and then decaying the learning rate as defined by this class.

void 
resetDecay()
Causes any learnign rate decay to reset to where it started.

protected double initialLearningRate
protected double decayRate
protected double minimumLR
protected double universalLR
protected java.util.Map<HashableState,ExponentialDecayLR.StateWiseLearningRate> stateWiseMap
protected java.util.Map<java.lang.Integer,ExponentialDecayLR.StateWiseLearningRate> featureWiseMap
protected boolean useStateWise
protected boolean useStateActionWise
protected HashableStateFactory hashingFactory
protected int lastPollTime
public ExponentialDecayLR(double initialLearningRate, double decayRate)
initialLearningRate
 the initial learning ratedecayRate
 the exponential base by which the learning rate is decayedpublic ExponentialDecayLR(double initialLearningRate, double decayRate, double minimumLearningRate)
initialLearningRate
 the initial learning ratedecayRate
 the exponential base by which the learning rate is decayedminimumLearningRate
 the smallest value to which the learning rate will decaypublic ExponentialDecayLR(double initialLearningRate, double decayRate, HashableStateFactory hashingFactory, boolean useSeparateLRPerStateAction)
initialLearningRate
 the initial learning rate for each state or stateactiondecayRate
 the exponential base by which the learning rate is decayedhashingFactory
 how to hash and compare statesuseSeparateLRPerStateAction
 whether to have an independent learning rate for each stateaction pair, rather than just each statepublic ExponentialDecayLR(double initialLearningRate, double decayRate, double minimumLearningRate, HashableStateFactory hashingFactory, boolean useSeparateLRPerStateAction)
initialLearningRate
 the initial learning rate for each state or stateactiondecayRate
 the exponential base by which the learning rate is decayedminimumLearningRate
 the smallest value to which the learning rate will decayhashingFactory
 how to hash and compare statesuseSeparateLRPerStateAction
 whether to have an independent learning rate for each stateaction pair, rather than just each statepublic double peekAtLearningRate(State s, Action ga)
LearningRate
peekAtLearningRate
in interface LearningRate
s
 the state for which the learning rate should be returnedga
 the action from which the learning rate should be returnedpublic double pollLearningRate(int agentTime, State s, Action ga)
LearningRate
pollLearningRate
in interface LearningRate
agentTime
 the time index of the agent when polling.s
 the state for which the learning rate should be returnedga
 the action from which the learning rate should be returnedpublic double peekAtLearningRate(int featureId)
LearningRate
peekAtLearningRate
in interface LearningRate
featureId
 the state feature for which the learning rate should be returnedpublic double pollLearningRate(int agentTime, int featureId)
LearningRate
pollLearningRate
in interface LearningRate
agentTime
 the time index of the agent when polling.featureId
 the state feature for which the learning rate should be returnedpublic void resetDecay()
LearningRate
resetDecay
in interface LearningRate
protected ExponentialDecayLR.StateWiseLearningRate getStateWiseLearningRate(State s)
s
 the state to get a learning rate forprotected ExponentialDecayLR.StateWiseLearningRate getFeatureWiseLearningRate(int feature)
feature
 the state feature id to get a learning rate forprotected double nextLRVal(double cur)
cur
 the currently learning rate to be decayed by one time step.