Key Moments
Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 5: Computational Methods
Stanford OnlineWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
The Pontryagin's Minimum Principle tightens optimality conditions for bounded controls, forcing a global minimization of the Hamiltonian, which can lead to 'bang' or 'bang-off-bang' control strategies.
Key Insights
The Pontryagin's Minimum Principle strengthens the optimality condition from a stationary point to a global minimizer of the Hamiltonian when controls are bounded.
Minimum time problems with bounded controls result in 'bang' control, where the system operates at maximum acceleration or braking.
Minimum fuel problems with bounded controls can lead to 'bang-off-bang' control, where the system coasts at zero control effort for portions of the trajectory.
Minimum energy problems with quadratic costs and bounded controls result in a saturating control profile, linear within bounds and constant at the bounds.
Singular arcs occur when optimality conditions provide no information on the control input, requiring more advanced methods to characterize the optimal control.
Computational methods for solving optimal control problems numerically include indirect shooting methods and collocation methods, often using libraries like SciPy's `solve_bvp`.
Extending optimality conditions to bounded controls
The lecture begins by revisiting optimal control problems, contrasting open-loop control (dependent only on initial time) with closed-loop control. It highlights the distinction between indirect methods (deriving optimality conditions first, then solving) and direct methods (discretizing first, then optimizing). This lecture focuses on refining indirect methods to handle bounded controls, a common practical constraint. Previously, optimality conditions for unbounded controls required the gradient of the Hamiltonian with respect to control (u) to be zero, analogous to finite-dimensional optimization. However, with bounded controls, the optimal control must be a global minimizer of the Hamiltonian, not just a stationary point. This shift is crucial because at the boundaries of the control set, arbitrary variations in u are no longer possible, changing the fundamental calculus of variations requirement from delta J = 0 to delta J >= 0.
The Pontryagin's Minimum Principle
The Pontryagin's Minimum Principle (PMP) is introduced as the extension of optimality conditions to incorporate bounded controls. The core change is in the third optimality condition: instead of requiring the partial derivative of the Hamiltonian with respect to u to be zero (dH/du = 0), the PMP demands that the optimal control u* globally minimizes the Hamiltonian H(x, u, p) for all admissible controls u. This means H(x, u*, p) <= H(x, u, p) for all valid u. This strengthening of the condition arises because at the control boundaries, only one-sided variations are permissible, preventing the arbitrary variation assumption that led to the gradient being zero in the unconstrained case. The principle essentially states that for any given state (x) and costate (p), the optimal control (u*) must be the one that minimizes the Hamiltonian.
Minimum energy problem with bounded controls
An example of a minimum energy problem illustrates the PMP. The objective is to minimize a quadratic cost functional (e.g., 1/2 * u^2) subject to system dynamics and a control bound (|u| <= 1). The Hamiltonian is formed as H = 1/2 * x1^2 + u^2 + p1*x2 - p2*(x2 - x1) + p2*u. The key step is to minimize H with respect to u. The terms involving u are u^2 + p2*u. Setting the derivative with respect to u to zero yields u + p2 = 0, so u = -p2. However, due to the bound |u| <= 1, the optimal control becomes a saturating function: u*(p2) = -p2 if -1 <= -p2 <= 1, u*(p2) = 1 if -p2 < -1, and u*(p2) = -1 if -p2 > 1. This results in a control profile that is linear within the bounds (-1, 1) and saturates at the bounds when -p2 falls outside this range. This exemplifies how bounds transform the optimal control strategy.
Minimum time problems and 'bang' control
Minimum time problems aim to reach a target state in the shortest possible time. The cost functional is typically the integral of 1 over time (∫ 1 dt). For a control-affine system x_dot = A(x) + B(x)u with bounded controls u_i in [m_i-, m_i+], the Hamiltonian is H = 1 + p^T(A + Bu). Minimizing H with respect to u involves minimizing each term p^T*b_i*u_i independently for each control dimension. If p^T*b_i is positive, the minimum is achieved at the lower bound u_i = m_i-. If p^T*b_i is negative, the minimum is at the upper bound u_i = m_i+. This results in a 'bang' control strategy, where the control is constantly at its maximum or minimum limit, driving the system as fast as possible towards the target. Singular arcs can occur if p^T*b_i = 0, where the optimality conditions offer no direct information about the control.
Minimum fuel problems and 'bang-off-bang' control
Minimum fuel problems typically minimize the integral of the absolute value of the control effort (∫ Σ c_i|u_i| dt). For a control-affine system with bounded controls, the Hamiltonian includes terms like c_i|u_i| + p^T*b_i*u_i. Minimizing this term for each i involves analyzing cases based on the sign of p^T*b_i relative to c_i. This leads to a 'bang-off-bang' control strategy. For example, if |p^T*b_i| < c_i, the optimal control u_i is 0, meaning the system coasts. If |p^T*b_i| > c_i, the control saturates at its maximum or minimum bound (m_i+ or m_i-). This strategy is intuitive: use minimal effort (coast) when possible, and only apply maximum thrust or braking when necessary to reach the target within the fixed time.
Computational methods for solving boundary value problems
The lecture concludes by introducing computational methods for solving the two-point boundary value problems that arise from indirect optimal control methods. These techniques are necessary when analytical solutions are intractable. Two main categories are discussed: indirect shooting methods and collocation methods. Shooting methods treat the boundary value problem as an initial value problem, making an initial guess for unknown initial conditions (often costates), propagating the system forward, and iteratively adjusting the guess based on the error at the final boundary. Collocation methods approximate the system dynamics using basis functions (like polynomials) and enforce the dynamics and boundary conditions at specific 'collocation points'. The use of SciPy's `solve_bvp` function is demonstrated with a simple example, showing how to define the differential equations, boundary conditions (in residual form), time discretization, and an initial guess to numerically solve for the state and costate trajectories.
Mentioned in This Episode
●Software & Apps
●Concepts
Optimal Control: Indirect Methods Cheat Sheet
Practical takeaways from this episode
Do This
Avoid This
Common Questions
Indirect methods first derive optimality conditions and then discretize for solving. Direct methods discretize the problem first and then optimize.
Topics
Mentioned in this video
A necessary condition for optimality in optimal control problems, particularly relevant when controls are bounded, strengthening the stationarity condition to a global minimization of the Hamiltonian.
An analog of the Lagrangian function in finite dimensional optimization, defined as the stage cost plus the dynamics multiplied by costates. It plays a central role in deriving optimality conditions.
More from Stanford Online
View all 121 summaries
83 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 3: Calculus of Variations
80 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 2: Optimization Theory
85 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 4: Optimal Control
74 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 1: Course Overview
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free