How does the Hamilton-Jacobi-Bellman equation help compute reachable sets?

The HJB equation can compute reachable sets by reframing set membership as a cost function. By removing the running cost and defining a final cost that represents belonging to the target set, the HJB equation can be used to solve differential games, which are central to computing these sets.

Can the Hamilton-Jacobi-Bellman formalism handle set membership directly?

Yes, the HJB formalism can encode the boolean outcome of belonging to a set by removing the running cost and using a final cost function. A target set can be represented by the zero level set of a function 'h', where belonging to the set corresponds to h(x) <= 0.

What is a reachable tube, and how does it differ from a reachable set?

A reachable tube considers the entire trajectory, not just the final state. It ensures that the system stays outside an avoidance set throughout the entire time horizon, strengthening the safety guarantee beyond just the endpoint.

How does MPC combine open-loop speed with closed-loop power?

MPC achieves this by solving an open-loop optimal control problem over a finite horizon at each time step. It then uses only the first control action and re-solves the problem from the new state, effectively incorporating feedback and closing the loop.

Why was MPC initially developed for chemical engineering?

MPC was suited for chemical engineering due to the need to manage constraints (like temperature limits) which are hard to handle with frequency-based controls, and because chemical processes often operate on slower time scales, allowing for the computation of online optimizations.

What are the key tuning knobs in MPC?

The main tuning knobs in MPC are the terminal cost, the terminal constraint set, and the prediction horizon length. These are used to ensure persistent feasibility and stability of the control system.

What is persistent feasibility in MPC?

Persistent feasibility ensures that the optimal control problem remains solvable at every time step. This means that for any current state, there's always a feasible control sequence that satisfies all constraints and objectives.

How can invariant sets help guarantee MPC feasibility?

If a set X1 (the truncated feasibility set) is control invariant, it means that if you start within X1, you can always find a control to stay within X1. This property can be leveraged to guarantee that the MPC problem remains feasible for all future steps.

What is the role of the HJ Reachability package?

The HJ Reachability package is a computational tool developed in the lab for solving Hamilton-Jacobi equations, used for calculating reachable sets. It's noted for its speed and implementation in Python.

Key Moments

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 11: Introduction to MPC

Q: What is the difference between a reachable set and an avoidance set?

A reachable set defines states from which you can reach a target set under all disturbances and for some control. An avoidance set defines states from which you can avoid a target set under all disturbances and for all controls, meaning you want to start outside this set to guarantee safety.

Q: What is a reachable tube, and how does it differ from a reachable set?

A reachable tube considers the entire trajectory, not just the final state. It ensures that the system stays outside an avoidance set throughout the entire time horizon, strengthening the safety guarantee beyond just the endpoint.

Q: How does MPC combine open-loop speed with closed-loop power?

MPC achieves this by solving an open-loop optimal control problem over a finite horizon at each time step. It then uses only the first control action and re-solves the problem from the new state, effectively incorporating feedback and closing the loop.

Q: Why was MPC initially developed for chemical engineering?

MPC was suited for chemical engineering due to the need to manage constraints (like temperature limits) which are hard to handle with frequency-based controls, and because chemical processes often operate on slower time scales, allowing for the computation of online optimizations.

Q: What are the key tuning knobs in MPC?

The main tuning knobs in MPC are the terminal cost, the terminal constraint set, and the prediction horizon length. These are used to ensure persistent feasibility and stability of the control system.

Q: What is persistent feasibility in MPC?

Persistent feasibility ensures that the optimal control problem remains solvable at every time step. This means that for any current state, there's always a feasible control sequence that satisfies all constraints and objectives.

Q: How can invariant sets help guarantee MPC feasibility?

If a set X1 (the truncated feasibility set) is control invariant, it means that if you start within X1, you can always find a control to stay within X1. This property can be leveraged to guarantee that the MPC problem remains feasible for all future steps.

Q: What is the role of the HJ Reachability package?

The HJ Reachability package is a computational tool developed in the lab for solving Hamilton-Jacobi equations, used for calculating reachable sets. It's noted for its speed and implementation in Python.

Stanford Online

Education5 min read74 min video

Aug 13, 2026|83 views|5

Stanford Stanford Online Robotics

Save to Pod

Want to know something specific about what's covered?

We've already dissected every moment. Ask and we will deliver (with timestamps).

Key Moments

TL;DR

Model Predictive Control (MPC) offers a practical approach to complex control problems by repeatedly solving finite-horizon open-loop problems, balancing computational cost with closed-loop performance.

Key Insights

The Hamilton-Jacobi-Bellman (HJB) equation can compute reachable sets by reframing set membership as a cost function, where belonging to a set corresponds to a negative cost.

Reachable tubes extend reachable sets to ensure safety across an entire trajectory, not just the endpoint, by minimizing a cost function over the minimum of the HJB value function across the horizon.

The two key objectives for tuning MPC are persistent feasibility (ensuring the optimization problem is always solvable) and stability (ensuring trajectories converge to the origin).

MPC applies to systems with constraints (state and control) that are difficult to handle with classical frequency-based control methods.

The core idea of MPC involves solving a finite-horizon optimal control problem, using only the first control action, and then re-planning from the new state.

The computation of reachable sets using HJB equations scales poorly beyond 5-6 dimensional state spaces, necessitating approximations like neural networks or sampling-based methods for higher dimensions.

Computing reachable sets using the Hamilton-Jacobi-Bellman equation

The lecture begins by revisiting the concept of reachable sets, crucial for optimal control applications. These sets define states from which a system can either avoid a target set (avoidance set, backward reachable set) or reach a target set (reach set). The Hamilton-Jacobi-Bellman (HJB) equation, a partial differential equation, is presented as a powerful tool for computing these sets. The key insight is that the binary condition of set membership can be encoded into a continuous cost function. Specifically, a target set T can be represented by the zero level set of a function h(x), where membership in T corresponds to h(x) <= 0. For avoidance sets, the objective is to maximize h(x) (staying outside the set), while for reach sets, the objective is to minimize h(x) (entering the set). This reframing allows the problem to be solved using optimal control formulations that can then be tackled by the HJB equation. The magnitude of h(x) also provides a measure of 'buffer' or safety margin.

From reachable sets to reachable tubes for full trajectory safety

While reachable sets guarantee safety at a specific final time, real-world applications often require ensuring safety throughout the entire trajectory. This leads to the concept of reachable tubes. A trajectory might end up outside an avoidance set, but still pass through a dangerous region at an intermediate time. To address this, the cost function is modified. Instead of just considering the final cost h(x(T)), the cost becomes the minimum of h(x(t)) over the entire time horizon [0, T]. This ensures that even in the worst-case scenario across the trajectory, the system remains outside the danger zone. Computing reachable tubes involves a more complex HJB equation due to this min-over-time cost. This extension is crucial for applications like collision avoidance, where a brief incursion into a safety zone is unacceptable.

The challenge of high-dimensional state spaces in reachability

A significant limitation of the HJB-based approach for computing reachable sets is its scalability. The computational complexity grows exponentially with the state-space dimension. The lecture notes that exact solutions are generally feasible only for state spaces up to about five or six dimensions. For systems with more complex dynamics or more states, approximations become necessary. These can include methods using neural networks, sampling-based approaches, or other techniques that may trade some accuracy for computational tractability. This scalability issue motivates the search for alternative control strategies, such as Model Predictive Control (MPC).

Introduction to Model Predictive Control (MPC)

Model Predictive Control (MPC) is introduced as a powerful and widely used control strategy that aims to combine the speed of open-loop methods with the robustness of closed-loop control. It originated in the chemical engineering domain in the 1970s and 80s due to the need for handling constraints effectively and the availability of computational resources for solving optimization problems at moderate frequencies (e.g., 0.1-0.5 Hz). MPC relies on a model of the system's dynamics and repeatedly solves an optimal control problem over a finite future horizon.

The MPC receding horizon optimization process

The core mechanism of MPC involves a receding horizon optimization. At each time step 't', the controller measures the current state x(t) and solves an open-loop optimal control problem over a finite horizon (e.g., from t to t+N). This optimization yields a sequence of control inputs. However, only the first control input, u(t), is applied to the system. The system then evolves to a new state, x(t+1). At the next time step (t+1), the entire process is repeated: the new state is measured, and a new finite-horizon optimal control problem is solved. This replanning at each step effectively closes the loop, providing closed-loop control while still solving open-loop problems at each iteration. This approach offers a balance between computational tractability and achieving desirable system behavior, especially in the presence of constraints.

Key MPC formulation elements: costs, constraints, and tuning knobs

A standard MPC formulation for a discrete-time linear time-invariant system (x_{t+1} = Ax + Bu) involves minimizing an objective function subject to state (x ∈ X) and control (u ∈ U) constraints, which are often polyhedral. The objective function typically includes a running cost (e.g., quadratic in state and control) and a terminal cost. Crucially, MPC incorporates a finite prediction horizon 'N', a terminal constraint (x_{t+N} ∈ X_f), and potentially an auxiliary terminal cost 'P'. These elements—the horizon length, the terminal cost matrix 'P', and the terminal set 'X_f'—serve as 'tuning knobs' for the MPC designer. Their selection is critical for ensuring two primary objectives: persistent feasibility (the optimization problem is always solvable) and stability (the system converges to the desired state, usually the origin).

Ensuring persistent feasibility and stability in MPC

Achieving persistent feasibility and stability is paramount for a robust MPC system. Persistent feasibility guarantees that at every planning step, a valid control sequence can be found, preventing the system from becoming stuck in an unmanageable state. Stability ensures that the control actions drive the system towards its objective, typically regulation to the origin. While early MPC design was often heuristic, theoretical advancements have provided ways to tune the horizon 'N', terminal cost 'P', and terminal set 'X_f' to guarantee these properties. The lecture outlines that by defining specific sets like the one-step controllable set and control invariant sets, and by leveraging tools from invariance theory, designers can derive conditions for these critical objectives. The focus will shift to how these tuning knobs can be manipulated to guarantee persistent feasibility and stability by design.

Mentioned in This Episode

●Software & Apps

●Concepts

Common Questions

A reachable set defines states from which you can reach a target set under all disturbances and for some control. An avoidance set defines states from which you can avoid a target set under all disturbances and for all controls, meaning you want to start outside this set to guarantee safety.

Topics

AI & Machine Learning Technology & Innovation Science & Mathematics Control Theory Optimal Control Reachability Analysis Differential Games State Constraints

Mentioned in this video

Software & Apps

Hamilton-Jacobi-Bellman Equation

The formalism for solving differential games and computing backward reachable sets, by setting the running cost to zero and defining the final cost to represent set membership.

HJ Reachability

A package developed in the speaker's lab, written in Python, for solving Hamilton-Jacobi equations, noted for being one of the fastest.

MPT

A MATLAB tool used for computing control invariant sets, mentioned as a resource for homework assignments.

Concepts

Model Predictive Control

A control strategy that uses an optimal control formulation over a finite horizon, repeatedly re-solving the problem as new state information becomes available.

Ask anything from this episode.

Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.

Get Started Free