How does LQR differ from PID control?

While PID control is often used at the actuator level for direct signal manipulation, LQR sits higher in the control hierarchy. LQR offers a more intentional way to specify objectives and can be used for trajectory tracking, whereas PID's response shaping is more indirect.

Can LQR be used for trajectory tracking, not just regulation to the origin?

Yes, LQR can be adapted for trajectory tracking. By reformulating the problem in terms of deviation variables from a nominal trajectory, the tracking problem can be turned into a regulation problem for these deviations, allowing LQR to be applied.

How is trajectory tracking handled for nonlinear systems?

For nonlinear systems, tracking is achieved by linearizing the dynamics around a nominal trajectory. This allows the problem to be reformulated in terms of deviation variables, transforming it into a linear system problem that can then be solved using LQR techniques.

Can LQR be used to compute optimal trajectories for nonlinear systems?

Yes, LQR ideas can be used to derive new methodologies for computing trajectories for nonlinear systems, such as Iterative LQR. This involves deformiing a nominal trajectory to achieve a lower cost, rather than driving deviations to zero.

What is Iterative LQR (iLQR)?

iLQR is an algorithm for trajectory optimization in nonlinear systems. It works by iteratively linearizing dynamics, approximating the cost quadratically, solving an LQR problem for deviation variables, and then propagating the exact dynamics to get a new, lower-cost trajectory.

What are the key differences between iLQR and SCP (Successive Convex Programming)?

Both iLQR and SCP use successive approximations. SCP can handle general convex approximations and constraints, while iLQR relies on LQR's analytical solutions and typically doesn't handle constraints explicitly. iLQR is simpler to implement due to its analytical nature, whereas SCP requires convex solvers.

What is Differential Dynamic Programming (DDP)?

DDP is a more powerful approach than iLQR. Instead of approximating dynamics and cost separately, DDP directly approximates the Bellman equation quadratically. It uses second-order derivatives of the dynamics, making it a more sophisticated algorithm.

What are the main challenges in implementing iLQR?

Challenges include ensuring new trajectories stay close to linearization points, careful handling of constraints (iLQR doesn't explicitly support them), the need for good initialization, and potential issues with cost matrices not being positive definite.

Key Moments

AStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 8: Nonlinearity

Stanford Online

Education5 min read75 min video

Aug 12, 2026|334 views|26

Stanford Stanford Online Robotics

Save to Pod

Want to know something specific about what's covered?

We've already dissected every moment. Ask and we will deliver (with timestamps).

Key Moments

TL;DR

LQR, a core optimal control algorithm, can be adapted for trajectory tracking and trajectory optimization, extending its utility beyond simple state regulation.

Key Insights

LQR is highly complementary to PID control, with PID handling low-level actuator manipulation and LQR focusing on trajectory optimization.

LQR can be generalized to handle affine dynamics (linear dynamics with a constant forcing term) and cost functions with linear, constant, and cross-penalty terms.

Trajectory tracking with nonlinear dynamics can be achieved by linearizing the dynamics around a nominal trajectory and then applying LQR to the deviation variables.

Iterative LQR (iLQR) computes optimal open-loop trajectories by iteratively linearizing dynamics, quadratizing the cost, and solving LQR problems with respect to deviation variables.

Differential Dynamic Programming (DDP) approximates the Bellman equation directly, using second-order derivatives of dynamics and cost, making it a more sophisticated variant of iLQR.

Constraints cannot be explicitly handled in standard LQR or iLQR; Sequential Convex Programming (SCP) is a more powerful alternative for problems with constraints.

LQR's adaptability beyond state regulation

While originally designed for Linear Quadratic Regulator (LQR) problems, aiming to drive system states to the origin, the lecture emphasizes LQR's versatility. It's not just for stabilization but also a powerful tool for trajectory tracking and even for computing trajectories for optimal control problems. This is a significant extension from its basic 'regulation' purpose. The instructor clarifies that LQR is often placed higher in the control hierarchy than PID, focusing on trajectories rather than direct actuator manipulation, making them complementary tools rather than competing ones. The control architecture can involve PID at the lowest level, trajectory optimization (like LQR-derived methods or SCP), and high-level objective setting, with physical knowledge granularity decreasing at higher levels.

Generalizing LQR for affine dynamics and mixed costs

The core LQR formulation assumes linear dynamics (x_k+1 = Ax_k + Bu_k) and a quadratic cost. However, the lecture details generalizations that significantly broaden its applicability. First, LQR can handle affine dynamics, meaning linear dynamics with an added constant forcing term (x_k+1 = Ax_k + Bu_k + d_k). Second, the cost function can be extended beyond simple quadratic penalties on state and control. It can include cross-terms (x'Qxu) and even linear and constant terms in the state and control. These extensions are crucial for practical applications, especially when LQR is used for trajectory tracking, where linearization processes naturally introduce such terms into the cost and dynamics.

Trajectory tracking with LQR: linear and nonlinear systems

A key application discussed is trajectory tracking, where the goal is to follow a desired reference trajectory (x_bar, u_bar) rather than driving the state to zero. For linear systems, this is achieved by reformulating the problem in terms of deviation variables (delta_x = x - x_bar, delta_u = u - u_bar). The system dynamics in terms of deviations become linear, allowing a standard LQR problem to be solved for delta_u to drive delta_x to zero. The final control is then the sum of the nominal control and the computed deviation control (u = u_bar + delta_u). For nonlinear systems, the same principle applies, but with an initial step of linearizing the nonlinear dynamics around the nominal trajectory. This linearization provides linear dynamics for the deviation variables, enabling the application of the LQR framework. The resulting tracking controller has an intuitive structure: a feedforward term (the nominal control) plus a feedback term that corrects deviations from the reference trajectory.

Iterative LQR for computing optimal trajectories

Moving beyond tracking a pre-computed trajectory, the lecture introduces how LQR ideas can be used to compute optimal open-loop trajectories for nonlinear systems. This is the domain of Iterative LQR (iLQR). The process involves starting with a feasible nominal trajectory and then iteratively improving it. In each iteration, the nonlinear dynamics are linearized and the cost function is quadratized around the current nominal trajectory. An auxiliary LQR problem is then solved for the deviation variables (delta_u) to minimize this approximated cost. The resulting delta_u values are used to update the control sequence, and then the exact nonlinear dynamics are propagated forward to generate a new, hopefully lower-cost, nominal trajectory. This process, comprising a backward pass (LQR solution) and a forward pass (dynamics propagation), repeats until convergence, yielding an optimized trajectory. A critical aspect is that the forward pass uses the actual nonlinear dynamics, not the linearized ones.

Differential Dynamic Programming: A more refined approach

Differential Dynamic Programming (DDP) is presented as a refinement of iLQR. While iLQR linearizes dynamics and approximates the cost before applying LQR, DDP directly approximates the Bellman equation itself, typically quadratically, around the nominal trajectory. This involves taking second-order derivatives of both the dynamics and the cost function. The optimization then proceeds by solving for control deviations (delta_u) that minimize this approximated Bellman equation. The key difference is DDP's use of second-order derivatives for dynamics, making it more sophisticated and potentially more accurate than iLQR, which relies on first-order derivatives for linearization. The outcome is similar: a feedforward control plus a feedback gain, but the gains themselves are derived through this more advanced approximation of the dynamic programming recurrence.

Limitations and practical considerations for LQR-based methods

Despite their power, LQR, iLQR, and DDP have limitations. A major one is the inability to handle explicit state or control constraints. While penalties can be added to the cost function, these methods do not enforce hard constraints. This is where algorithms like Sequential Convex Programming (SCP) become more advantageous, as they can explicitly incorporate constraints. For iLQR, ensuring that the trajectory updates stay close to the linearization points is critical for convergence; otherwise, the approximations break down, and the cost might even increase. Good initialization is also vital, and modern techniques often use data-driven methods (like neural networks) to provide warm starts. Regularization of cost matrices is sometimes necessary if they become non-positive definite during linearization or quadratization.

Mentioned in This Episode

●Software & Apps

●Concepts

Common Questions

LQR is a control theory algorithm that finds optimal control policies for linear systems with quadratic costs. It's widely used in robotics and control because it provides an explicit way to optimize operations and is flexible for stabilization and trajectory tracking.

Topics

Technology & Innovation Science & Mathematics Control Theory Optimal Control Iterative Methods Trajectory Tracking Nonlinear Systems

Mentioned in this video

Concepts

Iterative LQR

An algorithm that leverages LQR ideas to compute optimal open-loop trajectories by iteratively refining a nominal trajectory through linearization and quadratic approximation.

Dynamic Programming

A fundamental algorithm used to compute closed-loop optimal policies, derived here for discrete time and extended to continuous time.

Riccati equation

Equations used in LQR and its generalizations to update matrices and gains, crucial for deriving optimal control policies and costs.

Software & Apps

Large Language Models

Mentioned as a data-driven technique for setting objectives in control systems, positioned at a higher level of decision-making logic.

Ask anything from this episode.

Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.

Get Started Free