Key Moments
Stanford Robotics Seminar ENGR319 | Spring 2026 | Towards Trustworthy Autonomy
Stanford OnlineWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Robots are going into the real world at scale, but are prone to "semantic anomalies" that confuse them. New "fast and slow" reasoning systems use AI to detect and mitigate these failures, but they are not perfect.
Key Insights
Robots are now performing nearly 500,000 rides per week in the US, indicating a significant shift from controlled environments to real-world deployment.
Existing out-of-distribution detection methods fail to catch "semantic anomalies" where unusual contexts of ordinary objects cause confusion, unlike traditional physical safety concerns.
A two-stage "fast and slow" reasoning framework utilizes fast, embedding-based similarity queries for initial anomaly detection and slower, large language models for deeper reasoning and safety interventions.
Embedding-based anomaly detectors can achieve high accuracy, outperforming generative reasoning and showing that smaller models (e.g., ~100 million parameters) perform comparably to larger ones for detection.
The Cupid algorithm, based on influence functions from statistics, can causally relate training data to deployment-time performance, enabling data curation that dramatically improves success rates (e.g., from 40% to 90% with 2/3 fewer samples).
While LLMs add safety by making systems more conservative with false positives, their current accuracy in detecting semantic anomalies is not perfect, requiring significant further improvement for full reliability.
Robots are leaving controlled environments for widespread real-world deployment
Robotics is shifting from highly choreographed, repetitive tasks in controlled settings to large-scale deployment in dynamic, real-world environments. Companies like Waymo are now conducting approximately 500,000 rides per week in the US, signifying that these systems are no longer prototypes. This scalability is largely attributed to advances in learning algorithms that enable robots to interpret complex sensor data and make decisions in unpredictable situations. The emergence of end-to-end AI architectures promises further advancements, with robots potentially impacting how we move, work, and live. However, this scale also introduces significant safety risks as current systems are imperfect and prone to failures. Examples include self-driving cars misinterpreting sunsets as traffic lights, stopping due to billboards, or getting confused by inactive traffic lights on trucks. Manipulation systems can also make nonsensical errors, like placing utensils in ovens. These failures are particularly concerning with black-box learning systems, making it difficult to diagnose their root causes.
Failure modes stem from data limitations and spurious correlations
The core issue with current data-driven robotic systems is their reliance on training data, which inherently limits their performance. Models are only as good as the data they are trained on, leading to poor performance when encountering "out-of-distribution" (OOD) inputs—situations dissimilar to the training data. These OOD cases include rare "long-tail" events or sudden shifts in input distribution, such as a model trained on daytime data being deployed at night. Fundamentally, models learn correlations, not causation, making them susceptible to spurious correlations that do not hold true in general. For instance, a classifier might incorrectly associate a cow with its background rather than its actual features. This necessitates "guardrails" to manage the practical realities of deploying learning-enabled systems. Beyond validating against known unknowns, runtime monitors are crucial for guarding against "unknown unknowns"—rare events unforeseen during design. The focus is shifting from building perfect models to creating trustworthy systems that understand their capabilities and limitations, recover safely from challenging conditions, and allow for interpretable diagnosis and improvement.
Semantic anomalies challenge traditional safety and OOD detection
A significant challenge in ensuring robot safety is dealing with "semantic anomalies." These are unusual situations where the context of ordinary objects leads to system-level confusion, rather than a clear physical obstruction. Examples include a self-driving car confused by inactive traffic lights on a truck, stopping for a stop sign on a t-shirt, or a robot misplacing utensils due to the scene's context. Traditional OOD detection methods, which often focus on visual novelty or model uncertainty for physical safety, are ineffective here. A stop sign on a billboard isn't visually novel, and the model may be highly confident about its recognition. These semantic errors defy simple component-level blame and depend heavily on scene context. To address this, the research explores leveraging the common sense reasoning capabilities of large language models (LLMs), which, trained on vast internet-scale data, possess a form of common sense derived from lifetimes of human experience. This common sense is hypothesized to help mitigate semantic anomalies by understanding holistic context.
A 'thinking fast and slow' system for real-time semantic anomaly detection
Operationalizing LLMs for real-time robot control presents two key challenges: computational cost and integration into dynamic systems. LLMs are notoriously slow due to their auto-regressive token-by-token generation process. To overcome this, a two-stage reasoning framework is proposed: a "fast" stage using intermediate model outputs like embeddings for reactivity, and a "slow" stage employing full LLM generative capabilities for zero-shot decisions on unseen OOD scenarios. The "fast" stage detects semantic anomalies by comparing the semantic embedding of the current robot observation with a database of embeddings from prior experiences. If the current observation is deemed anomalous (dissimilar to past experiences), the "slow" stage is invoked. This involves querying the full LLM to reason about the anomaly and determine a safety-preserving intervention. The framework integrates these reasoners into a model predictive controller (MPC) that maintains fallback options and accounts for LLM decision latency, ensuring dynamically feasible interventions and guaranteeing reachability of the chosen recovery set. This approach allows for anomaly detection at speeds like 20 Hz on an NVIDIA Jetson, while the LLM provides deeper reasoning.
Smaller models excel at detection, while larger ones are needed for reasoning
Experiments with various embedding models on synthetic datasets demonstrate that grounding anomaly detectors in prior experiences via embeddings outperforms pure generative reasoning for anomaly detection. Notably, smaller models like MPNet and BERT (around 100 million parameters) achieve very high accuracy, comparable to larger models, in detecting a wide range of semantic anomalies. This suggests that the anomaly detection task itself is relatively easier for models. The accuracy of these detectors increases linearly with the percentage of nominal concepts in the database, indicating they primarily detect deviations from prior experiences. This means that while fast, smaller models are sufficient for detecting *that* something is different, robustly assessing *why* it's anomalous and what to do requires the deeper reasoning capabilities of larger LLMs. The "fast" reasoner provides reactivity, enabling real-time detection, while the "slow" reasoner is essential for thoroughly assessing safety criticality.
Improving models through data curation with influence functions
The second part of the work focuses on systematically improving end-to-end robotic policies by addressing failures observed during deployment. Instead of fixing black-box models manually, a data-centric approach is advocated: a policy is only as good as its training data. Failures can stem from diverse human demonstrator skill levels, varied collection strategies, or spurious correlations. To systematically identify these issues, methods that counterfactually reason about the impact of training data inclusion or exclusion are needed. This involves "data attribution"—predicting how policy performance changes if certain data is removed. The research leverages "influence functions" from statistics, a tool that computes the sensitivity of a model's performance to individual training samples. By applying policy gradient techniques, an estimator is developed to causally link training data to deployment-time performance outcomes, even without explicit reward functions or known environment dynamics. This enables the "Cupid" algorithm, which uses performance influence to curate data, removing low-quality samples or selecting valuable new ones, creating a "data flywheel" for continuous improvement.
Cupid algorithm dramatically enhances policy performance through data pruning
The Cupid algorithm has shown significant success in improving policy performance through data curation. Experiments demonstrate that pruning low-quality samples can lead to drastic improvements in success rates, for example, increasing performance from 40% to 90% with two-thirds fewer samples in some tasks. This highlights that a large portion of collected data might be problematic or even detrimental to learning. Cupid can identify the most robust training strategies under test-time distribution shifts and root out spurious correlations. Compared to heuristic methods like "demonth," which measures trajectory noisiness, Cupid's causal approach is more effective because human intuition about data quality might not align with how models actually learn. The algorithm has also been applied to curate post-training data for visual-inertial odometry (VIO) systems, significantly boosting their performance. Cupid's ability to causally link training data to key performance indicators makes it a powerful tool for systematic policy improvement.
Trustworthy autonomy requires guardrails and a data-centric view
The overarching challenge in robotics is that models are limited by their training data, making out-of-distribution edge cases an ever-present risk. A safety-first process that tightly links development and deployment is crucial, aiming for systems users can trust rather than solely perfect models. This involves creating "guardrails"—runtime monitors to detect impending failures, safety interventions to avoid negative consequences, and deployment data analysis to transparently improve model performance. Two key takeaways are that runtime monitors offer a pragmatic paradigm for failure detection and mitigation, and that a data-centric view is essential for understanding how data quality and composition drive the performance of learning systems. These insights help demystify deep learning systems. Future work may involve further refining LLM accuracy, integrating them more deeply into control loops, and expanding data curation techniques to even more complex robotics applications.
Mentioned in This Episode
●Products
●Software & Apps
●People Referenced
Anomaly Detection Model Performance Comparison
Data extracted from this episode
| Model Type | Embedding Cache Size | Anomaly Detection Accuracy |
|---|---|---|
| Embedding Detectors | Increasing | High Accuracy |
| GPT Models (Generative Reasoning) | N/A | Lower than Embedding Detectors |
| Small Models (e.g., MPNet, BERT) | N/A | Extremely High Accuracy (comparable to larger models) |
Common Questions
Semantic safety in robotics refers to preventing failures arising from the holistic context and interrelations of objects in a scene, rather than just physical obstacles. For example, mistaking a stop sign on a billboard for a real traffic signal.
Topics
Mentioned in this video
Mentioned as an example of a large language model that could potentially handle common sense reasoning tasks for robots.
A small language model mentioned as performing well in anomaly detection tasks, capable of running at high frequencies.
A small language model mentioned alongside MPNet for its high accuracy in anomaly detection tasks, despite its relatively small size.
A Vision-Language Model (VLM) whose post-training performance was significantly increased by data curation using the Cupid algorithm.
An algorithm developed for data attribution and curation, designed to systematically improve robot policy performance by identifying and selecting valuable training data.
More from Stanford Online
View all 82 summaries
58 minStanford CS153 Frontier Systems | Building the Frontier Ecosystem
50 minStanford MS&E435 Economics of the AI Supercycle | Spring 2026 | Applications, Coding AI
50 minStanford MS&E435 Economics of the AI Supercycle | Spring 2026 | Building AI Factories
42 minStanford CS153 Frontier Systems | Scale, AGI, and the Future of Everything
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free