How are optimal control problems with free final time handled?

Problems with free final time can be reformulated using time rescaling, where a normalized time variable from 0 to 1 is introduced. A dummy state variable with trivial dynamics is also added to represent the final time.

What are the two main categories of direct methods for optimal control?

The two main categories are state and control parameterization (collocation) methods, where both states and controls are optimization variables, and control parameterization (shooting) methods, where only controls are optimization variables and states are computed recursively.

What are the pros and cons of state/control parameterization vs. control parameterization?

State and control parameterization handles complex state constraints better but involves more optimization variables. Control parameterization has fewer variables, making optimization problems smaller, but handling state constraints is more challenging.

Why are direct methods becoming more popular in robotics?

Advances in nonlinear optimization have made solving direct method problems feasible, even in real-time. They are often simpler to implement compared to the complex derivations required for indirect methods.

How does sequential convex programming (SCP) solve nonlinear optimal control problems?

SCP iteratively solves a sequence of convex optimization problems. It linearizes and convexifies the nonlinear dynamics and cost around a nominal trajectory, solves the resulting convex problem, and updates the trajectory.

What is the challenge with implementing direct methods, like the river crossing example?

Direct methods are sensitive to the quality of the initial guess and require careful tuning of parameters and optimizers. Poor initial guesses or restrictive constraints can lead to the solver failing to find a feasible solution.

How can one improve the initial guess for direct methods?

A common strategy is to first solve a simplified version of the problem, perhaps by relaxing constraints, and then use that solution as an initial guess for the full problem.

What is the role of Jacobians in sequential convex programming?

Jacobians (derivatives of the dynamics with respect to states and controls) are used to linearize the nonlinear dynamics around a nominal trajectory, forming the basis of the convex subproblems in SCP.

How does SCP handle potential issues like artificial infeasibility?

Artificial infeasibility is typically addressed using slack variables, which are introduced to relax constraints and are penalized in the cost function, guiding the optimization towards feasibility.

What is the advantage of solving convex optimization problems within SCP?

Convex optimization problems are guaranteed to find the global optimum, are computationally efficient, and can be solved reliably, making them ideal for the iterative steps within SCP.

What is the main difference between 'shooting' and 'multiple shooting' methods?

Shooting (control parameterization) propagates dynamics from the initial condition. Multiple shooting breaks the trajectory into smaller segments, performing shooting within each segment and adding constraints at segment boundaries.

Key Moments

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 6: Optimal Control

Stanford Online

Education6 min read81 min video

Aug 12, 2026|182 views|15

Stanford Stanford Online Robotics

Save to Pod

Want to know something specific about what's covered?

We've already dissected every moment. Ask and we will deliver (with timestamps).

Key Moments

On this page

TL;DR

Direct methods for optimal control discretize time and turn continuous problems into nonlinear optimization, offering simpler implementation than indirect methods but requiring more tuning.

Key Insights

Indirect methods for optimal control problems boil down to solving two-point boundary value problems, which can be computationally challenging.

Time rescaling and introducing a dummy state variable are "tricks" used to reformulate optimal control problems with free final time into a standard form amenable to classical solvers.

Direct methods discretize time, transforming optimal control problems into nonlinear, constrained optimization problems solvable by numerical optimizers.

State and control parameterization methods (collocation) and control parameterization methods (shooting) are two main categories of direct methods, differing in whether states and controls are optimization variables.

Sequential convex programming (SCP) iteratively solves a sequence of convex problems to find a solution to a nonlinear optimization problem, linearizing and convexifying dynamics and costs around nominal trajectories.

The choice between state and control parameterization methods often depends on the complexity of state constraints; control parameterization might be simpler for few state constraints, while state and control parameterization is better for complex ones.

Bridging indirect and direct methods: Reformulating free final time problems

The lecture begins by reviewing indirect methods for optimal control, which involve deriving optimality conditions and solving them as two-point boundary value problems. A key challenge addressed is handling problems where the final time is not fixed. The common "time rescaling" trick is introduced: time 't' is rescaled to a normalized time 'tau' (t/TF), making the final time known (1). This requires adjusting time derivatives by multiplying them by TF. Additionally, a dummy state variable 'R' with trivial dynamics (R dot = 0) is introduced to represent the final time. This reformulation transforms problems with free final time into a standard form suitable for classical solvers. An example of controlling a particle on a line to move from x0=10 to xf=0 with a cost on final time and control effort is used to illustrate the analytical solution and its computational implementation via a BVP solver.

Computational solution of optimal control problems via BVP solvers

The lecture demonstrates the application of a Python BVP solver to an actual optimal control problem, specifically the particle on a line example where the final time is free. The system dynamics are defined, including the Hamiltonian and optimality conditions (dynamics, costate dynamics, and control optimality). Boundary conditions are established, including initial states and target final states. For the free final time case, an additional master boundary condition is applied, which, after derivation, results in a condition involving the costate and final time. The implementation involves defining the differential equations, boundary conditions as residuals, and a time array from 0 to 1 (due to rescaling). The solver successfully finds the final time to be approximately 4.47, matching the analytical solution, and visualizes the state variables and control profile. This highlights how indirect methods, when computationally tackled, involve setting up and solving complex boundary value problems.

Transitioning to direct methods: A more user-friendly approach

The lecture then introduces direct methods as an increasingly popular alternative to indirect methods in robotics, primarily due to their ease of use. While indirect methods focus on deriving optimality conditions and solving them (optimized then discretized), direct methods discretize time first, transforming the continuous-time optimal control problem into a nonlinear, constrained optimization problem that can be solved by off-the-shelf nonlinear solvers. This shift is enabled by significant advances in nonlinear optimization. The core idea is to approximate the continuous problem with a discrete one, where the primary challenge shifts from formulating complex optimality conditions to tuning the optimization process itself.

Two main categories of direct methods: Parameterization strategies

Direct methods are broadly categorized into two main sub-strategies based on how the continuous-time problem is transcribed into a discrete optimization problem: 1. State and control parameterization methods (collocation): Both states and controls are treated as optimization variables, with dynamics acting as constraints linking them. 2. Control parameterization methods (shooting): Only controls are optimization variables; states are computed recursively by unrolling the dynamics. Control parameterization methods tend to result in smaller optimization problems as there are fewer variables. However, enforcing state constraints can be more challenging. Conversely, state and control parameterization methods allow direct enforcement of state constraints by treating states as optimization variables, though this can lead to larger problems. Hybrid methods, like multiple shooting, also exist.

State and control parameterization: Discretizing dynamics and costs

In state and control parameterization methods, time is discretized (e.g., using Euler discretization), and a zero-order hold is assumed for controls (constant within a time step). Continuous-time dynamics are approximated using Euler integration: x(t + delta t) ≈ x(t) + delta t * f(x(t), u(t)). The continuous cost integral is also discretized, often approximated as the integrand evaluated at a time step multiplied by the step size. This process transforms the continuous optimal control problem into a discrete nonlinear optimization problem where both discretized states and controls are optimization variables, linked by the discretized dynamics, which become constraints.

Illustrating direct methods with Zerloc's river crossing problem

The lecture uses Zerloc's river crossing problem to demonstrate direct methods. The goal is to cross a river with a spatially varying flow, minimizing control effort (squared angle of velocity relative to flow). The problem is transcribed using state and control parameterization. Time is discretized, and a zero-order hold on the control angle (u) is assumed. The dynamics are approximated using Euler integration. The cost function is the sum of squared control efforts over discretized time steps. Constraints include initial and final position requirements, bounds on control angles, and the discretized dynamics themselves. Initial attempts with restrictive control bounds (u_max = 0.75) fail without a good initial guess, highlighting the sensitivity of direct methods to this aspect. Using a solution from a relaxed problem (u_max = 1) as an initial guess for the more restrictive problem enables convergence. This demonstrates the practical challenges and tuning required for direct methods.

Control parameterization (shooting) for optimal control

In control parameterization methods, only the control inputs (u_i) at each discrete time step are optimization variables. The dynamics are used to recursively propagate the state trajectory forward from an initial condition. This approach is also known as 'shooting' because the optimization 'shoots' forward through the dynamics with chosen controls. For the Zerloc's problem, this means the optimization problem's variables are only the control angles over time. The constraints function takes only 'u' as input, and an inner loop propagates the dynamics forward for each candidate 'u' solution to check feasibility and cost. This method can be less sensitive to initial guesses compared to state and control parameterization but makes enforcing complex state constraints more difficult, as states are not explicit optimization variables.

Sequential convex programming: Iterative refinement of nonlinear problems

Sequential Convex Programming (SCP) is presented as an advanced direct method that tackles challenging nonlinear optimization problems by iteratively solving a sequence of convex subproblems. The core idea is to linearize and convexify the nonlinear dynamics and cost functions around a nominal trajectory. In each iteration, a linearized and convexified surrogate problem is solved, yielding an updated nominal trajectory. This process repeats, refining the solution with each step. The advantage is that convex optimization problems are efficiently and reliably solvable to a global optimum. Practical implementation involves using automatic differentiation (like JAX) to compute Jacobians for linearization and discretizing the resulting linear optimal control problem into a convex optimization problem. Challenges like artificial infeasibility and unboundedness are addressed using techniques such as trust regions and slack variables, which are incorporated into the cost or constraints to guide the optimizer and ensure convergence.

Mentioned in This Episode

●Software & Apps

●Concepts

Common Questions

Indirect methods derive optimality conditions and solve two-point boundary value problems, often leading to analytical solutions. Direct methods discretize time first, transforming the problem into a nonlinear optimization problem, which is generally easier to implement computationally.

Topics

Technology & Innovation Science & Mathematics Robotics Control Optimal Control Convex Optimization Direct Methods Indirect Methods Nonlinear Optimization Sequential Convex Programming Trajectory Optimization

Mentioned in this video

Software & Apps

solve_bvp

A solver for two-point boundary value problems used in optimal control, demonstrated in Python for both abstract problems and specific optimal control examples.

Python

Programming language used for implementing optimal control solvers, specifically the solve_bvp function for two-point boundary value problems.

Jax

A library used for automatic differentiation and linearizing dynamics in sequential convex programming.

Concepts

Hamiltonian

A function formulated in optimal control problems that combines system dynamics, costates, and control variables, used to derive optimality conditions.

Ask anything from this episode.

Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.

Get Started Free