public class PrioritizedSweeping extends ValueIteration
Modifier and Type | Class and Description |
---|---|
protected class |
PrioritizedSweeping.BPTR
A back pointer and its max action probability of transition.
|
protected class |
PrioritizedSweeping.BPTRNode
A node for state thar contains a list of its back pointers, their max probability of transition to this state, and the priority of this nodes state.
|
protected static class |
PrioritizedSweeping.BPTRNodeComparator
Comparator for the the priority of BPTRNodes
|
ValueFunctionPlanner.StaticVFPlanner
QComputablePlanner.QComputablePlannerHelper
Modifier and Type | Field and Description |
---|---|
protected int |
maxBackups
THe maximum number Bellman backups permitted
|
protected HashIndexedHeap<PrioritizedSweeping.BPTRNode> |
priorityNodes
The priority queue of states
|
foundReachableStates, hasRunVI, maxDelta, maxIterations, stopReachabilityFromTerminalStates
transitionDynamics, useCachedTransitions, valueFunction, valueInitializer
actions, containsParameterizedActions, debugCode, domain, gamma, hashingFactory, mapToStateIndex, rf, tf
Constructor and Description |
---|
PrioritizedSweeping(Domain domain,
RewardFunction rf,
TerminalFunction tf,
double gamma,
StateHashFactory hashingFactory,
double maxDelta,
int maxBackups)
Initializes
|
Modifier and Type | Method and Description |
---|---|
protected PrioritizedSweeping.BPTRNode |
getNodeFor(StateHashTuple sh)
Returns or creates, stores, and returns a priority back pointer node for the given hased state
|
boolean |
performReachabilityFrom(State si)
This method will find all reachable states that will be used by the
ValueIteration.runVI() method and will cache all the transition dynamics. |
void |
planFromState(State initialState)
This method will cause the planner to begin planning from the specified initial state
|
void |
runVI()
Runs VI until the specified termination conditions are met.
|
recomputeReachableStates, resetPlannerResults, toggleReachabiltiyTerminalStatePruning
computeQ, computeQ, getActionsTransitions, getAllStates, getCopyOfValueFunction, getDefaultValue, getQ, getQ, getQs, getValueFunctionInitialization, hasComputedValueFor, initializeOptionsForExpectationComputations, performBellmanUpdateOn, performBellmanUpdateOn, performFixedPolicyBellmanUpdateOn, performFixedPolicyBellmanUpdateOn, setValueFunctionInitialization, toggleUseCachedTransitionDynamics, value, value, VFPInit
addNonDomainReferencedAction, getActions, getAllGroundedActions, getDebugCode, getDomain, getGamma, getHashingFactory, getRf, getRF, getTf, getTF, plannerInit, setActions, setDebugCode, setDomain, setGamma, setRf, setTf, stateHash, toggleDebugPrinting, translateAction
protected HashIndexedHeap<PrioritizedSweeping.BPTRNode> priorityNodes
protected int maxBackups
public PrioritizedSweeping(Domain domain, RewardFunction rf, TerminalFunction tf, double gamma, StateHashFactory hashingFactory, double maxDelta, int maxBackups)
domain
- the domain in which to planrf
- the reward functiontf
- the terminal state functiongamma
- the discount factorhashingFactory
- the state hashing factor to usemaxDelta
- when the maximum change in the value function is smaller than this value, VI will terminate.maxBackups
- the maximum number of Bellman backups. If set to -1, then there is no hard limit.public void planFromState(State initialState)
OOMDPPlanner
planFromState
in class ValueIteration
initialState
- the initial state of the planning problempublic void runVI()
ValueIteration
ValueIteration.planFromState(State)
method.
The ValueIteration.performReachabilityFrom(State)
must have been performed at least once
in the past or a runtime exception will be thrown. The ValueIteration.planFromState(State)
method will automatically call the ValueIteration.performReachabilityFrom(State)
method first and then this if it hasn't been run.runVI
in class ValueIteration
public boolean performReachabilityFrom(State si)
ValueIteration
ValueIteration.runVI()
method and will cache all the transition dynamics.
This method will not do anything if all reachable states from the input state have been discovered from previous calls to this method.performReachabilityFrom
in class ValueIteration
si
- the source state from which all reachable states will be foundprotected PrioritizedSweeping.BPTRNode getNodeFor(StateHashTuple sh)
sh
- the hashed state for which its node should be returned.