Key Moments
Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 4: Optimal Control
Stanford OnlineWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Optimal control problems can be solved using indirect methods that transform infinite-dimensional optimization into differential equations, but solving them analytically is rare, with numerical solvers being the practical approach.
Key Insights
Indirect methods for optimal control transform the problem of optimizing functionals (functions of functions) into necessary optimality conditions, analogous to setting the gradient to zero in finite-dimensional optimization.
The Euler equation, derived from the fundamental theorem of calculus of variations, is a second-order nonlinear ODE with split boundary conditions that characterizes extremals for simplified optimal control problems.
Relaxing fixed boundary conditions (final time or final state) requires more sophisticated boundary conditions derived from a specific equation that accounts for admissible variations, leading to case-by-case analysis.
Optimal control problems with unbounded controls are formulated using a Hamiltonian, which augments the cost function with dynamics via costate variables, leading to a system of differential and algebraic equations.
Solving optimal control problems analytically is difficult; typically, the derived differential equations and boundary conditions are fed into numerical solvers, making indirect methods 'optimized and discretized'.
From functionals to necessary optimality conditions
This lecture introduces indirect methods for optimal control, which are used to derive necessary conditions for optimality in infinite-dimensional optimization problems. These problems involve optimizing over functionals, essentially functions of functions, where the goal is to find an optimal control signal u*(t) over a time interval [t0, tf]. The core mathematical tool used is the calculus of variations. The fundamental theorem of calculus of variations states that for a function x* to be an extremal (a local minimum), the variation of the cost functional J must vanish at x*. This condition, which states that the variation of J must be zero for all admissible variations delta x, is analogous to the gradient being zero in finite-dimensional optimization. While this condition characterizes potential optima, it is not directly actionable until translated into more specific forms like the Euler equation.
The Euler equation for simplified problems
For a simplified optimal control problem where the cost functional is defined by a stagewise cost g(x(t), ẋ(t), t) and controls are implicitly defined by the state dynamics (no explicit control input 'u'), the calculus of variations leads to the Euler equation. This equation is a second-order nonlinear ordinary differential equation. A key characteristic of the Euler equation derived in this simplified setting with fixed initial and final states and times is that it often results in split boundary conditions, meaning boundary conditions are specified at both the initial and final times. This contrasts with Cauchy problems where boundary conditions are typically all at the initial time. The process of deriving the Euler equation involves taking the increment of the functional, linearizing it, and setting the variation to zero, which is analogous to applying Newton's method.
Illustrating the Euler equation: the shortest path
A concrete example of applying the Euler equation is finding the shortest path between two points. The functional to minimize is the integral of the arc length, ds, over the path. Approximating ds using infinitesimal time dt and displacement dx, we get ds = sqrt(dt^2 + dx^2) = dt * sqrt(1 + ẋ(t)^2). Thus, the functional J is the integral of sqrt(1 + ẋ(t)^2) dt from t0 to tf. To apply the Euler equation, we need to compute ∂g/∂x and ∂g/∂ẋ. Since g = sqrt(1 + ẋ(t)^2) does not depend on x, ∂g/∂x = 0. The derivative ∂g/∂ẋ is ẋ(t) / sqrt(1 + ẋ(t)^2). The Euler equation, ∂g/∂x - d/dt(∂g/∂ẋ) = 0, simplifies to d/dt(ẋ / sqrt(1 + ẋ^2)) = 0. Differentiating this leads to ẍ(t) / (1 + ẋ(t)^2)^(3/2) = 0. For this to hold, ẍ(t) must be 0. Integrating ẍ(t) = 0 twice with respect to time yields x(t) = c1*t + c2, which is the equation of a straight line. The two integration constants c1 and c2 are determined by the fixed initial and final position boundary conditions, formally proving that a straight line is the shortest path between two points.
Relaxing fixed boundary conditions
When boundary conditions are not fixed (e.g., the final time tf or the final state x(tf) are free), the Euler equation remains valid, but additional boundary conditions are needed to determine the constants of integration. These new conditions arise from considering the variation of the functional with respect to the free final state and time. A general equation is used, and by considering whether delta xf and delta tf are fixed (zero variation) or free (any variation allowed), specific boundary conditions can be derived. For instance, if tf is fixed but x(tf) is free, the condition derived is ∂g/∂ẋ evaluated at tf must be zero. This systematic approach allows the formalism to handle a broader range of optimal control problems, such as finding the shortest path from a fixed point to a line at a fixed final time, where the final x position is determined by the optimization.
Incorporating constraints: The Hamiltonian
To handle optimal control problems with explicit control inputs 'u' and system dynamics defined by differential equations (e.g., ẋ = f(x, u, t)), the concept of the Hamiltonian is introduced. This is analogous to using Lagrange multipliers in finite-dimensional optimization to handle constraints. The Hamiltonian H is defined as the stage cost g(x, u, t) plus the product of costate variables p(t) (analogous to Lagrange multipliers) and the system dynamics f(x, u, t). The optimality conditions then consist of differential equations for the states x and costates p, along with algebraic equations relating u to p and x, derived from setting the derivative of the Hamiltonian with respect to u to zero. The system must also satisfy the original dynamics and specific boundary conditions, which depend on whether initial/final states or times are fixed or free.
Solving optimal control problems: The practical approach
The necessary optimality conditions for optimal control problems, derived using indirect methods, typically result in a system of differential and algebraic equations. For analytical solutions, these equations, along with derived boundary conditions, must be solved to find the constants of integration. However, analytical solutions are rare. In practice, these derived differential equations and boundary conditions are fed into numerical solvers. This process is referred to as 'optimized and discretized,' where 'optimized' refers to the formulation of optimality conditions and 'discretized' refers to the numerical solution. The system typically involves 2N first-order differential equations for N states and N costates, plus M algebraic equations if there are M control inputs.
Example: Particle on a line with free final time
A detailed example demonstrates the application of these principles to optimize the motion of a particle on a line. The goal is to reach the origin (x=0, ẋ=0) from an initial state (x=10, ẋ=0) at a free final time tf. The cost functional balances control effort (u^2) and minimizing final time (tf^2), weighted by parameters b and alpha. The problem is first cast as a first-order system. The Hamiltonian is formulated as H = 0.5*b*u^2 + p1*x2 + p2*u. The optimality conditions yield differential equations for x and p, and an algebraic equation for u (u = -p2/b). Solving these equations with the given boundary conditions (initial state, final state, and a condition derived from the free final time which involves the Hamiltonian and terminal cost) allows for the determination of the constants of integration and the final time. This example highlights the systematic application of the formalism, including casting second-order dynamics into first-order form and using generalized boundary conditions.
Mentioned in This Episode
●Concepts
Common Questions
Indirect methods derive necessary optimality conditions by extending calculus of variations, which leads to differential equations. Direct methods, discussed for the following week, discretize the problem and solve it as a finite-dimensional optimization problem.
Topics
Mentioned in this video
A method used in constrained optimization problems where constraints are added to the objective function using multipliers to create an unconstrained problem.
An augmented cost function used in optimal control problems, formed by adding the system dynamics multiplied by costate variables to the original cost.
More from Stanford Online
View all 121 summaries
83 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 3: Calculus of Variations
80 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 2: Optimization Theory
79 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 5: Computational Methods
74 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 1: Course Overview
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free