Key Moments
Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 3: Calculus of Variations
Stanford OnlineWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Optimal control problems are infinite-dimensional optimization problems, and necessary optimality conditions can be derived using calculus of variations, leading to the Euler-Lagrange equations.
Key Insights
Inequality constraints in optimization problems can be treated as active (binding) or inactive, and a local minimum for the full problem is also a local minimum for the problem considering only active constraints.
The Karush-Kuhn-Tucker (KKT) conditions are necessary optimality conditions for optimization problems with inequality constraints, requiring specific non-negativity and complementarity conditions for Lagrange multipliers.
Indirect methods for optimal control, also known as 'optimize then discretize,' derive optimality conditions first (often differential equations) and then solve them numerically, contrasting with direct methods ('discretize then optimize').
Calculus of variations extends calculus to functionals (functions of functions) and uses the concept of variations to derive necessary optimality conditions, analogous to how gradients are used in finite-dimensional optimization.
The fundamental theorem of calculus of variations states that for a differentiable functional, the variation must vanish at an extremum (analogous to gradient being zero), leading to the Euler-Lagrange equation for simplified optimal control problems.
Handling inequality constraints in optimization
The lecture begins by revisiting finite-dimensional optimization, specifically addressing inequality constraints. An inequality constraint $g_j(x) leq 0$ is considered 'active' if $g_j(x) = 0$ at a given point $x$, and 'inactive' if $g_j(x) < 0$. A key insight is that a local minimum of an optimization problem with inequality constraints is also a local minimum of the same problem with only the active constraints considered. This is because any small perturbation around the local minimum will respect the active constraints as equalities (or slightly violate them without leaving the feasible region), while inactive constraints will remain strictly negative. This simplification allows the use of methods developed for equality constraints, specifically by treating active inequality constraints as equalities.
The KKT conditions for inequality constraints
Building on the simplification of active constraints, the lecture introduces the Karush-Kuhn-Tucker (KKT) conditions, which are necessary optimality conditions for problems with inequality constraints. These conditions extend the gradient-zero condition from unconstrained optimization by incorporating Lagrange multipliers for both equality and inequality constraints. For inequality constraints $g_j(x) leq 0$, the corresponding multipliers $mu_j$ must be non-negative ($mu_j geq 0$). Furthermore, a complementarity condition states that either the multiplier $mu_j$ is zero (for inactive constraints) or the constraint is active ($mu_j > 0$ implies $g_j(x) = 0$). This set of conditions, including the gradient of the Lagrangian being zero, non-negativity of $mu_j$, and the complementarity condition, helps filter candidate optimal solutions.
Introduction to indirect methods in optimal control
The lecture then transitions to optimal control, framing it as an infinite-dimensional optimization problem where the goal is to find an optimal control signal $u(t)$ that minimizes an objective functional $J$. Indirect methods are introduced as a primary approach for solving these problems. These methods follow an 'optimize then discretize' paradigm. They first derive necessary optimality conditions for the continuous-time optimal control problem, which typically result in differential equations (like the Euler-Lagrange equations). These derived differential equations are then solved numerically using discretization techniques. This contrasts with direct methods, which discretize the problem first and then optimize the resulting finite-dimensional problem.
Calculus of variations: functionals and norms
To tackle infinite-dimensional optimization, the lecture introduces calculus of variations. A functional, denoted by $J(x)$, is a mapping that assigns a scalar number to a function $x(t)$. This is analogous to how a function maps a number to a number in finite-dimensional calculus. To define concepts like 'closeness' and 'neighborhoods' for functions, norms are used, similar to how they are used for vectors. A norm assigns a non-negative number to a function, satisfying properties like non-negativity and the triangle inequality. The distance between two functions $y(t)$ and $z(t)$ can then be measured by the norm of their difference, $|y - z|$. This allows for the definition of local minima for functionals: a functional $J$ has a local minimum at $x^*(t)$ if $J(x^*) leq J(x)$ for all functions $x$ within a certain norm-bounded neighborhood of $x^*$. The control signal $u(t)$ will be the function being optimized in optimal control problems.
The fundamental theorem of calculus of variations
The core of calculus of variations for finding extrema is the fundamental theorem. Analogous to finding critical points in finite-dimensional calculus by setting the gradient to zero, in infinite dimensions, the 'variation' of the functional must vanish at an extremum. The variation, denoted $delta J$, represents the linear part of the increment of the functional. For a differentiable functional $J$, the increment $Delta J$ can be approximated by $delta J$. The fundamental theorem states that if $x^*(t)$ is an extremum of $J$, then $delta J(x^*, delta x) = 0$ for all admissible variations $delta x$. This condition, $delta J = 0$, is the necessary optimality condition for functionals.
Deriving the Euler-Lagrange equation
The lecture applies the fundamental theorem to a simplified optimal control problem: minimizing $int_{t_0}^{t_f} g(x(t), dot{x}(t), t) dt$ subject to fixed initial and final states and times. By calculating the increment of this functional and using Taylor expansion and integration by parts, the linear part ($delta J$) is derived. Applying the condition $delta J = 0$ and using the fundamental lemma of calculus of variations (which states that if $int h(t) delta x(t) dt = 0$ for all admissible $delta x(t)$, then $h(t)=0$), leads to the Euler-Lagrange equation: $ rac{partial g}{partial x} - rac{d}{dt}left( rac{partial g}{partial dot{x}} ight) = 0$. This equation provides the necessary condition for optimality for this class of problems.
Mentioned in This Episode
●Concepts
Common Questions
Optimal control is a field focused on determining control signals for a dynamical system that minimize or maximize a given objective function, often involving costs over time and at a final state.
Topics
Mentioned in this video
More from Stanford Online
View all 121 summaries
85 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 4: Optimal Control
80 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 2: Optimization Theory
79 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 5: Computational Methods
74 minStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 1: Course Overview
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free