Implementation of Double Pole Cart Balancing task. More...
Classes | |
class | Action |
Implementation of action of Double Pole Cart. More... | |
class | State |
Implementation of the state of Double Pole Cart. More... | |
Public Member Functions | |
DoublePoleCart (const size_t maxSteps=0, const double m1=0.1, const double m2=0.01, const double l1=0.5, const double l2=0.05, const double gravity=9.8, const double massCart=1.0, const double forceMag=10.0, const double tau=0.02, const double thetaThresholdRadians=36 *2 *3.1416/360, const double xThreshold=2.4, const double doneReward=0.0) | |
Construct a Double Pole Cart instance using the given constants. More... | |
void | Dsdt (const State &state, const Action &action, arma::vec &dydx) |
This is the ordinary differential equations required for estimation of next state through RK4 method. More... | |
State | InitialSample () |
Initial state representation is randomly generated within [-0.05, 0.05]. More... | |
bool | IsTerminal (const State &state) const |
This function checks if the car has reached the terminal state. More... | |
size_t | MaxSteps () const |
Get the maximum number of steps allowed. More... | |
size_t & | MaxSteps () |
Set the maximum number of steps allowed. More... | |
void | RK4 (const State &state, const Action &action, arma::vec &dydx, State &nextState) |
This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation. More... | |
double | Sample (const State &state, const Action &action, State &nextState) |
Dynamics of Double Pole Cart instance. More... | |
double | Sample (const State &state, const Action &action) |
Dynamics of Double Pole Cart. More... | |
size_t | StepsPerformed () const |
Get the number of steps performed. More... | |
Implementation of Double Pole Cart Balancing task.
This is an extension of the existing CartPole environment. The environment comprises of a cart of a cart with two poles of different lengths and masses. The agent is meant to balance the poles by applying force on the cart.
Definition at line 27 of file double_pole_cart.hpp.
|
inline |
Construct a Double Pole Cart instance using the given constants.
maxSteps | The number of steps after which the episode terminates. If the value is 0, there is no limit. |
m1 | The mass of the first pole. |
m2 | The mass of the second pole. |
l1 | The length of the first pole. |
l2 | The length of the second pole. |
gravity | The gravity constant. |
massCart | The mass of the cart. |
forceMag | The magnitude of the applied force. |
tau | The time interval. |
thetaThresholdRadians | The maximum angle. |
xThreshold | The maximum position. |
doneReward | Reward recieved by agent on success. |
Definition at line 123 of file double_pole_cart.hpp.
This is the ordinary differential equations required for estimation of next state through RK4 method.
state | The current state. |
action | The action taken. |
dydx | The differential. |
Definition at line 197 of file double_pole_cart.hpp.
References DoublePoleCart::Action::action, DoublePoleCart::State::Angle(), and DoublePoleCart::State::AngularVelocity().
Referenced by DoublePoleCart::RK4(), and DoublePoleCart::Sample().
|
inline |
Initial state representation is randomly generated within [-0.05, 0.05].
Definition at line 289 of file double_pole_cart.hpp.
References DoublePoleCart::State::State().
|
inline |
This function checks if the car has reached the terminal state.
state | The desired state. |
Definition at line 301 of file double_pole_cart.hpp.
References DoublePoleCart::State::Angle(), Log::Info, and DoublePoleCart::State::Position().
Referenced by DoublePoleCart::Sample().
|
inline |
Get the maximum number of steps allowed.
Definition at line 327 of file double_pole_cart.hpp.
|
inline |
Set the maximum number of steps allowed.
Definition at line 329 of file double_pole_cart.hpp.
This function calls the RK4 iterative method to estimate the next state based on given ordinary differential equation.
state | The current state. |
action | The action to be applied. |
dydx | The differential. |
nextState | The next state. |
Definition at line 238 of file double_pole_cart.hpp.
References DoublePoleCart::State::Data(), DoublePoleCart::Dsdt(), and DoublePoleCart::State::State().
Referenced by DoublePoleCart::Sample().
Dynamics of Double Pole Cart instance.
Get reward and next state based on current state and current action.
state | The current state. |
action | The current action. |
nextState | The next state. |
When done is false, it means that the cartpole has fallen down. For this case the reward is 1.0.
Definition at line 159 of file double_pole_cart.hpp.
References DoublePoleCart::State::AngularVelocity(), DoublePoleCart::Dsdt(), DoublePoleCart::IsTerminal(), DoublePoleCart::RK4(), and DoublePoleCart::State::Velocity().
Referenced by DoublePoleCart::Sample().
Dynamics of Double Pole Cart.
Get reward based on current state and current action.
state | The current state. |
action | The current action. |
Definition at line 278 of file double_pole_cart.hpp.
References DoublePoleCart::Sample().
|
inline |
Get the number of steps performed.
Definition at line 324 of file double_pole_cart.hpp.