burlap.behavior.policy

Class CachedPolicy

java.lang.Object

burlap.behavior.policy.CachedPolicy

All Implemented Interfaces:

EnumerablePolicy, Policy

public class CachedPolicy
extends java.lang.Object
implements EnumerablePolicy

This class can be used to lazily cache the policy of a source policy. This is useful when the source policy may require some degree of computation and you'd like to store the results of that computation to memory so that it does not have be recomputed for states that have their policy queried multiple times.

Author:

James MacGlashan.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Map<HashableState,java.util.List<ActionProb>>	actionSelection The cached action selection probabilities
protected HashableStateFactory	hashingFactory The hashing factory to use for indexing states
protected EnumerablePolicy	sourcePolicy The source policy that gets cached

Constructor Summary

Constructors

Constructor and Description
CachedPolicy(HashableStateFactory hashingFactory, EnumerablePolicy sourcePolicy) Initializes
CachedPolicy(HashableStateFactory hashingFactory, EnumerablePolicy sourcePolicy, int cacheCapacity) Initializes

Method Summary

Modifier and Type	Method and Description
Action	action(State s) This method will return an action sampled by the policy for the given state.
double	actionProb(State s, Action a) Returns the probability/probability density that the given action will be taken in the given state.
boolean	definedFor(State s) Specifies whether this policy is defined for the input state.
java.util.List<ActionProb>	policyDistribution(State s) This method will return action probability distribution defined by the policy.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

hashingFactory

protected HashableStateFactory hashingFactory

The hashing factory to use for indexing states

actionSelection

protected java.util.Map<HashableState,java.util.List<ActionProb>> actionSelection

The cached action selection probabilities

sourcePolicy

protected EnumerablePolicy sourcePolicy

The source policy that gets cached

Constructor Detail

CachedPolicy

public CachedPolicy(HashableStateFactory hashingFactory,
                    EnumerablePolicy sourcePolicy)

Initializes

Parameters:

hashingFactory - the HashableStateFactory to use for indexing states

sourcePolicy - the source policy that will be lazily cached.

CachedPolicy

public CachedPolicy(HashableStateFactory hashingFactory,
                    EnumerablePolicy sourcePolicy,
                    int cacheCapacity)

Initializes

Parameters:

hashingFactory - the HashableStateFactory to use for indexing states

sourcePolicy - the source policy that will be lazily cached.

cacheCapacity - the initial memory capacity to be set aside for the policy cache

Method Detail

action

public Action action(State s)

Description copied from interface: Policy

This method will return an action sampled by the policy for the given state. If the defined policy is stochastic, then multiple calls to this method for the same state may return different actions. The sampling should be with respect to defined action distribution that is returned by getActionDistributionForState

Specified by:

action in interface Policy

Parameters:

s - the state for which an action should be returned

Returns:

a sample action from the action distribution; null if the policy is undefined for s

actionProb

public double actionProb(State s,
                         Action a)

Description copied from interface: Policy

Returns the probability/probability density that the given action will be taken in the given state.

Specified by:

actionProb in interface Policy

Parameters:

s - the state of interest

a - the action that may be taken in the state

Returns:

the probability/probability density

policyDistribution

public java.util.List<ActionProb> policyDistribution(State s)

Description copied from interface: EnumerablePolicy

This method will return action probability distribution defined by the policy. The action distribution is represented by a list of ActionProb objects, each which specifies a grounded action and a probability of that grounded action being taken. The returned list does not have to include actions with probability 0.

Specified by:

policyDistribution in interface EnumerablePolicy

Parameters:

s - the state for which an action distribution should be returned

Returns:

a list of possible actions taken by the policy and their probability.

definedFor

public boolean definedFor(State s)

Description copied from interface: Policy

Specifies whether this policy is defined for the input state.

Specified by:

definedFor in interface Policy

Parameters:

s - the input state to test for whether this policy is defined

Returns:

true if this policy is defined for State s, false otherwise.