public class InvertedPendulum extends java.lang.Object implements DomainGenerator
CartPoleDomain
in which the movement of the pole depends only on gravity and the force applied, and not the velocity of the
underlying cart. The track is also always assumed to be infinite. Therefore, the state space for this domain is fully described by two variables:
the angle and angular velocity of the pole. However, there is also noise included in the actions
of this domain as well a noop action. This version of the inverted pendulum is the version used in the original
Least-Squares Policy Iteration paper [1].
1. Lagoudakis, Michail G., and Ronald Parr. "Least-squares policy iteration." The Journal of Machine Learning Research 4 (2003): 1107-1149.Modifier and Type | Class and Description |
---|---|
class |
InvertedPendulum.ForceAction
An action that applies a given force to the cart + uniform random noise in the range defined in the
physParams data member. |
static class |
InvertedPendulum.InvertedPendulumRewardFunction
A default reward function for this domain.
|
static class |
InvertedPendulum.InvertedPendulumTerminalFunction
A default terminal function for this domain.
|
static class |
InvertedPendulum.IPPhysicsParams |
Modifier and Type | Field and Description |
---|---|
static java.lang.String |
ACTIONLEFT
A constant for the name of the left action
|
static java.lang.String |
ACTIONNOFORCE
A constant for the name of the no force action (which due to stochasticity may include a small force)
|
static java.lang.String |
ACTIONRIGHT
A constant for the name of the right action
|
static java.lang.String |
ATTANGLE
A constant for the name of the angle attribute
|
static java.lang.String |
ATTANGLEV
A constant for the name of the angle velocity
|
static java.lang.String |
CLASSPENDULUM
The object class for the pendulum.
|
InvertedPendulum.IPPhysicsParams |
physParams |
Constructor and Description |
---|
InvertedPendulum() |
Modifier and Type | Method and Description |
---|---|
Domain |
generateDomain()
Returns a newly instanced Domain object
|
static State |
getInitialState(Domain domain)
Returns an initial state with 0 angle (perfectly vertical) and 0 angle velocity.
|
static State |
getInitialState(Domain domain,
double angle,
double angleVelocity)
Returns an initial state with the pole at the given angle and with the given angular velocity of the pole.
|
static void |
main(java.lang.String[] args) |
static void |
updateState(State s,
double controlForce,
InvertedPendulum.IPPhysicsParams physParams)
Updates the given state object given the control force.
|
public static final java.lang.String ATTANGLE
public static final java.lang.String ATTANGLEV
public static final java.lang.String CLASSPENDULUM
public static final java.lang.String ACTIONLEFT
public static final java.lang.String ACTIONRIGHT
public static final java.lang.String ACTIONNOFORCE
public InvertedPendulum.IPPhysicsParams physParams
public Domain generateDomain()
DomainGenerator
generateDomain
in interface DomainGenerator
public static void updateState(State s, double controlForce, InvertedPendulum.IPPhysicsParams physParams)
s
- the input statecontrolForce
- the control force acted upon the cart.physParams
- the InvertedPendulum.IPPhysicsParams
object specifying the physics to use for movementpublic static State getInitialState(Domain domain)
domain
- the domain object to which the state will be belong.public static State getInitialState(Domain domain, double angle, double angleVelocity)
domain
- the domain object to which the state will belong.angle
- the angle of the pole from the vertical axis.angleVelocity
- the angular velocity of the pole.public static void main(java.lang.String[] args)
args
- none expected