How can robots achieve high reliability for complex real-world tasks?

High reliability can be achieved by iterating on models, collecting more detailed data (especially for edge cases), and by enabling the AI system itself to automatically seek out areas where it needs more data or supervision, similar to reinforcement learning.

What are the challenges in applying reinforcement learning from language models to robotics?

RL algorithms like PPO/TRPO require millions of attempts, which translates to prohibitively long times (e.g., 700 robot days for one minute of task data) when applied to physical robots due to hardware constraints and real-world operation costs.

How does memory improve robotic capabilities?

Most current foundation models lack memory, operating only on current sensor data. Memory is critical for long-horizon tasks that involve multiple sequential steps, allowing robots to track progress and operate autonomously for extended periods.

What are the key advancements that have brought robotics to a 'ChatGPT-like' era?

Key advancements include the development of general-purpose foundation models trained on diverse data, incorporating memory, and demonstrating compositional generalization, allowing a single model to perform many tasks effectively out-of-the-box.

How does compositional generalization apply to robotics?

Compositional generalization allows robots to combine known skills and concepts in novel ways, such as interacting with an appliance not seen in training data or applying folding skills to a different robot platform, demonstrating a deeper conceptual understanding.

What is the robotics equivalent of the internet-scale data used to train large language models?

The equivalent is data from robots operating in real-world circumstances, collected through teleoperation and autonomous experience. Videos of humans and captioned images are also useful, but direct robot experience remains critical.

How can someone with a software engineering background break into robotics?

One can join a robotics company as a software engineer, or gain hands-on experience by purchasing an affordable robot, experimenting with fine-tuning open-source models, and showcasing these projects on a resume.

Key Moments

Chelsea Finn: This is the State of the Art in Robotics

Y Combinator

Science & Technology6 min read59 min video

Aug 12, 2026|1,840 views|143|6

YC Y Combinator

Save to Pod

Want to know something specific about what's covered?

We've already dissected every moment. Ask and we will deliver (with timestamps).

Key Moments

TL;DR

Robots can now perform complex tasks autonomously for hours, but achieving human-level reliability and dexterity still requires significant advancements in reinforcement learning and memory systems.

Key Insights

Google's Med-Gemini outperforms doctors at diagnosis, raising the question of whether AI should be deployed even when we can't explain its reasoning.

Sauna 4-7x/week reduces cardiovascular death risk by 50% vs once/week (BMC Medicine, n=1,688).

A year ago, Waymo passed a quarter of a million weekly autonomous rides, demonstrating the feasibility of trustworthy autonomous operation in the physical world.

Reinforcement learning with human intervention to prevent dead-end trajectories and amortized value estimation can significantly reduce the millions of attempts typically needed for robot learning.

A single general-purpose robot model (PIO7) can now match or outperform specialized, fine-tuned models in various tasks without post-training.

The PIO7 model demonstrates compositional generalization, enabling robots to interact with unseen appliances and perform tasks on different robot platforms without prior training data for those specific combinations.

The critical need for reliable general-purpose robots

The current state of robotics, while capable of impressive feats like folding laundry and making espresso, struggles with reliability and long-term autonomy. Unlike AI applications where users make decisions based on model recommendations (e.g., ChatGPT, ad ranking), physical robots must directly and reliably interact with the real world. This necessitates a much higher degree of accuracy and fewer mistakes. The success of autonomous ride-sharing services like Waymo, which completed over a quarter million weekly rides, offers a beacon of hope, indicating that trustworthy autonomous operation in the physical world is achievable. The core challenge lies in developing general-purpose robots that can perform a wide variety of tasks in unknown environments, autonomously, and for extended periods, rather than being limited to repetitive or highly specific functions.

Reinforcement learning for scalable robot reliability

Achieving high reliability in robotics, such as a 90% success rate for making espresso, is a significant hurdle. Traditional machine learning workflows involve iterative data collection and model tuning, which can be tedious and time-consuming for humans. A more scalable approach involves reinforcement learning (RL), where the AI system autonomously seeks out areas needing more data or supervision. However, applying standard RL algorithms like PPO/PPO to robotics is computationally prohibitive due to the immense number of attempts required—millions of trajectories could equate to hundreds of robot days. To overcome this, two key improvements are crucial: first, identifying and terminating 'dead-end' trajectories early, often with human intervention, to avoid wasting robot time and resources on unproductive attempts. Second, amortizing the cost of value estimation across different tasks and scenarios by training a general-purpose value function. This function can predict what constitutes good or bad progress, enabling more efficient learning from experience. These techniques allow for rapid iteration and fine-tuning of foundation models to achieve high reliability in complex manipulation tasks.

Memory as a crucial ingredient for long-term autonomy

Most current state-of-the-art foundation models for robotics lack memory, operating solely on current sensor observations. While this is sufficient for short, repetitive tasks, it becomes a critical bottleneck for long-horizon tasks requiring multi-step sequences. naively incorporating past sensor data, such as 10 seconds of video at 50Hz across multiple camera streams, can lead to millions of tokens, making real-time processing prohibitively expensive. To address this, a multi-scale memory system is needed. This includes short-term video memory (e.g., 10 seconds) processed efficiently, and for longer durations (minutes to hours), a compressed textual summary of past events. By combining these, robots can autonomously perform complex, non-repetitive tasks like cleaning a kitchen for 10-15 minutes, tracking progress through various sub-steps such as wiping counters, drying them, and putting away dishes, without human oversight.

Towards a single, out-of-the-box general-purpose robot model

The evolution of generalist AI has moved from training models from scratch (ImageNet, 2012) to using pre-trained models (BERT, ImageNet pre-trained) and finally to using generalist models out-of-the-box (GPT-2 onwards). Robotics has lagged, traditionally requiring bespoke data sets and training from scratch for each project. The goal is to reach a similar 'GPT-like' era for robotics, where a single model can perform diverse tasks without task-specific fine-tuning. This requires training a foundation model on a vast and diverse dataset, incorporating robot demonstrations, policy rollouts, human videos, and web data. Crucially, detailed prompting, including memory, instruction, subtask instructions, and metadata, is key to enabling the model to effectively leverage this heterogeneous data. This approach has led to models like PIO7, which, out-of-the-box, matches or exceeds the performance of specialized, RL-post-trained models on tasks like folding shirts and assembling components.

Compositional generalization: applying learned skills to novel situations

Compositional generalization, inspired by models like DALL-E, allows AI systems to combine known concepts to understand and generate novel ones. In robotics, this translates to applying learned skills to unseen objects, appliances, or even different robot platforms. For example, a robot trained on various manipulation tasks might generalize to operating an air fryer, even if air fryers were not explicitly part of its training data. Similarly, a model trained to fold clothes on one robot platform could potentially fold clothes on a completely different, unseen robot platform. This ability demonstrates a deeper conceptual understanding and data efficiency, as the model doesn't need explicit training for every possible combination of task and environment. Such generalization is critical for creating truly adaptable and versatile robots.

The importance of diverse data and detailed prompting

Experiments with the PIO7 model highlight the critical role of data diversity and detailed prompting. Removing highly diverse data significantly degrades performance on unseen tasks, whereas removing less diverse data has a minor impact. This underscores that breadth of experience is key for generalization. Furthermore, prompting the model with metadata (e.g., data quality, episode length) is crucial. Without metadata prompting, incorporating low-quality data can decrease performance. However, with metadata prompting, even low-quality data can contribute to improved performance, suggesting the model can extract more value from a wider range of data when guided by detailed contextual information. This detailed prompting, combined with a large, diverse dataset, is fundamental to achieving out-of-the-box performance and compositional generalization in robotics.

Current status and future outlook for physical intelligence

Robotics is now entering a 'GPT and DALL-E like era' with the development of general-purpose models capable of complex, autonomous tasks. These advancements are transitioning from research labs to real-world deployments, with companies like Ultra and Weave already fine-tuning PI models for tasks like laundry folding and warehouse packaging. The models are becoming adaptable to diverse embodiments, including drones, quadcopters, surgical robots, and tractors. While significant progress has been made in achieving long-term autonomy and generalization, challenges remain, including improving robot speed to match or exceed human teleoperation and further enhancing reliability. The path forward involves continued innovation in reinforcement learning, memory systems, and data collection strategies to make robots more capable, faster, and more useful in everyday applications.

Mentioned in This Episode

●Software & Apps

●Companies

●People Referenced

Key Principles for Building Useful General-Purpose Robots

Practical takeaways from this episode

Do This

Develop general-purpose models that can perform many tasks.

Focus on bringing models to the real world to have an impact.

Prioritize developing physical AI systems that make fewer mistakes than current ML systems.

Enable robots to operate autonomously for long periods.

Incorporate memory at multiple time scales for long-horizon tasks.

Train foundation models on diverse data with sufficient capacity.

Leverage detailed prompting, including metadata, subtask instructions, and subgoal images.

Utilize open-source generalist policies and fine-tune them for specific applications.

Avoid This

Rely on systems where humans make the final decision based on AI recommendations (for physical tasks).

Expect robots to learn complex tasks solely from observing humans without their own platform experience.

Assume that current state-of-the-art foundation models have memory or context.

Naively feed large amounts of raw video data into models, which is computationally prohibitive.

Train models from scratch on bespoke datasets for every individual project if aiming for general-purpose capabilities.

Forget the importance of hardware and software stacks, as well as ML and data infrastructure, in robotics development.

Common Questions

AI in software, like recommendation systems, allows for a margin of error because the user can recognize and correct mistakes. Physical AI, however, must operate autonomously and directly affect the real world, requiring far fewer errors to be trustworthy and useful.

Topics

Reinforcement Learning Artificial Intelligence AI & Machine Learning Technology & Innovation Science & Mathematics Autonomous Systems Machine Learning Foundation Models Compositional Generalization

Mentioned in this video

Organizations

Physical Intelligence

The company founded by Chelsea Finn two years prior to the talk, focused on developing general-purpose robots capable of performing any task in the real world.

Software & Apps

ChatGPT

A general-purpose AI model launched in 2022, significant for being the first to be widely used by many people in the real world, reaching a million users in five days.

PPO

Reinforcement learning algorithms used in language models that have scaled to enable complex reasoning.

TRPO

Reinforcement learning algorithms used in language models that have scaled to enable complex reasoning.

BERT

A pre-trained model that became a norm for fine-tuning to downstream tasks, showing better performance than training from scratch.

GPT-2

An early generalist model that marked a transition towards using models out-of-the-box without fine-tuning.

Dolly

A model that showed signs of compositional generalization in 2021, demonstrating conceptual understanding and data efficiency.

PIO7

A general-purpose foundation model developed by Physical Intelligence, capable of performing diverse tasks out-of-the-box and exhibiting compositional generalization.

Gemma

An open-source model mentioned in the context of companies supporting open source to build ecosystems.

Weave

A YC company that has taken PI models and post-trained them for packaging tasks in a warehouse.

Companies

Dandelion Chocolate Factory

A local chocolate factory near the speaker's office where a robot workflow was trained to construct, label, and stack cardboard boxes.

Ultra

A YC company that has taken PI models and post-trained them for tasks like folding laundry in deployment.

People

Roger Federer

Mentioned as an example of a highly skilled individual whose actions, if only observed, do not directly enable someone else to perform at the same level, illustrating the need for direct experience in robotics.

Ask anything from this episode.

Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.

Get Started Free