Key Moments
Jeff Dean: The 1% Rule for Building in AI
Y CombinatorWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Jeff Dean argues AI is now at a junior engineer level and predicts automated ML system improvement, emphasizing specialized inference hardware and long-running agent tasks.
Key Insights
AI models are now capable enough to perform tasks previously thought to require human junior engineers, with their ability in complex and agent-based tasks growing faster than anticipated.
The inference hardware market is identified as the next significant area for innovation, with a focus on high-performance, low-energy solutions to reduce latency, potentially 50x better than current systems.
A key assumption to challenge is the limitation of AI agents; they can and will run for days or weeks on complex tasks like implementing new software versions in different programming languages.
The origin of the TPU stemmed from a 2013 calculation showing that three minutes of daily speech recognition per user would require doubling Google's server fleet, highlighting the need for specialized hardware.
Data movement energy cost is approximately 1000x higher than computation energy cost, necessitating strategies like batching to amortize these costs and influencing system design in AI.
For aspiring founders, focus on problems where current general models succeed only 0-1% of the time, indicating an opportunity for specialized solutions, or on personal information organization where general models lack access.
AI's current capabilities and future predictions
Jeff Dean states that AI models have reached the level of a junior engineer, a prediction he made in May 2025. He notes that models are rapidly improving in agent-based coding tasks and are showing significant promise in other domains. Dean's bold prediction for 2027 is the widespread automation of ML systems themselves. This involves ML systems improving their own capabilities through automated loops of experimentation, problem decomposition into sub-problems, and integration of results. He believes this automated scientific method, where ML systems run experiments, break down problems, and iterate on solutions, will not only revolutionize ML but also other fields of science and engineering where measurable objectives exist.
The 'it fits in memory' moment for inference hardware
Drawing a parallel to the 2001 realization that Google's entire search index could fit in RAM, Dean identifies a similar 'it fits in memory' moment for 2026 centered around high-performance and low-energy inference hardware. He emphasizes that inference is crucial for making AI agents accessible and that reducing latency is paramount. Specialized hardware, he explains, can achieve significantly better energy efficiency and lower latency compared to general-purpose devices like GPUs or TPUs. The goal is to create systems where users no longer have to wait for responses, imagining the possibilities with a 50x reduction in latency. This focus on specialized inference hardware is key to enabling more sophisticated and responsive AI applications.
Challenging assumptions about AI agent capabilities
Dean highlights an underappreciated assumption: the potential for agent-based AI systems to run for extended periods—days or even weeks—to tackle highly complex tasks. He notes that current models can be instructed to implement entirely new software versions in different programming languages, focusing on improved safety or performance properties. This capability goes far beyond short-term tasks and suggests a future where agents can perform sophisticated, long-duration projects. The implication is that we should not underestimate the future capacity of AI agents to undertake and complete complex, multi-stage problems that currently require significant human effort and time.
The genesis of the TPU and the importance of napkin math
Recounting the origin of the Tensor Processing Unit (TPU), Dean describes a 2013 calculation where he realized that if every Google user engaged with speech recognition for just three minutes daily, the system would require doubling Google's server fleet. This prompted the development of custom hardware. He explains that deep learning models for speech had halved error rates but were computationally expensive. The realization that increased usage due to better quality would strain existing infrastructure led to the creation of TPUs. These specialized chips, designed for low-precision dense linear algebra, proved to be 30-80 times more energy-efficient and 20-30x lower latency than CPUs and GPUs of the time, laying the foundation for modern machine learning inference.
Energy as the new unit of measurement and data IO bottlenecks
Dean points out that energy has become the primary metric for evaluating computational tasks. He states that performing a single multiplier operation costs about one picojoule, while moving data and performing I/O costs about a thousand times more. This significant gap in energy cost between computation and data movement shapes many decisions in AI system design. For instance, batching is necessary to amortize data movement costs, but it hinders low-latency applications. Founders should recognize that many perceived 'model problems' are actually rooted in energy constraints or data I/O challenges, which dictate system architecture and algorithm design.
Context engineering and the power of specific prompts
Dean emphasizes that AI progress is now about more than just model size; it encompasses the surrounding ecosystem, including retrieval tools, memory, and agent orchestration. He calls this 'context engineering,' where models learn to use tools, retrieve information, and leverage past interactions. Unlike the undifferentiated soup of training data, the context provided directly to a model for a specific problem is clearer and more actionable. The ability to decompose problems, sequence tool calls, and evaluate different approaches is becoming crucial. This area is particularly accessible to startups, as it requires API access and creative prompt engineering rather than massive compute resources for training.
Finding startup opportunities in specialized AI domains
Dean advises aspiring founders to identify problems where current general AI models perform poorly (0-1% success rate). He suggests two main avenues for startups: organizing personal information, which general models lack access to, or developing highly specialized models for niche domains. He cites DeepMind's AlphaFold for protein folding as an example of a highly successful niche model. Other potential domains include material science and chip design. Founders should assess whether their chosen problem is a durable opportunity or likely to be addressed by advancing general models within six to twelve months. Passion and identifying truly underserved areas are key.
The scarce skill of taste and setting clear specifications
With AI agents capable of writing code, Dean identifies 'taste'—having excellent judgment on what problems to tackle—as the scarce skill. He likens it to research, where problem selection is paramount. Building taste comes from experience, trying new ideas, and evaluating their importance over time. He also stresses the critical importance of clear specifications when working with AI agents. Just as translating software between languages is effective due to detailed specifications, clearly defining desired outcomes for agents enables them to perform tasks more reliably. The ability to provide these precise instructions and guide AI-assisted computation is what will drive significant impact.
Questioning fundamental assumptions and historical breakthroughs
Dean advocates for 'crazy thought experiments' that question deeply ingrained assumptions, citing the assumption of near-perfect transistors in chip design. He proposes considering systems built from transistors with higher error rates, which could lead to new design methodologies and fabrication techniques, potentially mirroring biological systems like the brain with its less reliable but redundant neural pathways. Historically, such thought experiments led to consequential systems like TPUs (specializing hardware for a nascent problem) and MapReduce (abstracting parallelization and reliability from application logic), demonstrating the value of revisiting fundamental design choices.
The automated scientific method and faster validation
Dean is excited about systems that automate the scientific method: proposing, implementing, running experiments, and evaluating results. He envisions highly automated loops capable of running numerous experiments efficiently, accelerating progress in ML, science, and engineering. This includes AI systems that can recursively self-improve by running extensive experiments, optimizing discovery per unit of compute. He also highlights the potential for much faster validation models, such as neural approximations to computationally intensive simulators, which can drastically speed up experimental loops and enable screening of millions of possibilities, transforming scientific discovery and research processes. An example is a quantum chemistry simulator that was made 300,000 times faster.
Navigating rejection and defining career impact
Reflecting on his career, Dean shares the experience of a paper on model distillation being rejected from NeurIPS in 2014 for 'unlikely to have significant impact.' He emphasizes that even with rejections, persistence is key, as demonstrated by distillation's widespread adoption and use in capable models like Gemini's flash models. When considering future paths, Dean advises focusing on problems one deeply cares about, collaborating with liked colleagues, and aiming for positive global impact. Whether joining a frontier lab or starting a company, the core question remains: 'If I work on this problem and the best possible outcome happens, will the world be a lot better in some way?'
Future problems and building effective teams
Dean hopes future consequential systems will tackle problems in new hardware approaches, more data-efficient ML algorithms, continuous learning, and multi-agent interactions. He also points to societal challenges like improving discourse among people and facilitating connections based on interests. When building teams, he stresses finding individuals with complementary skills, low ego, and good team player attitudes. The ideal scenario involves small teams where members learn from each other, continuously adding to their 'tool belt' of techniques. This collaborative growth is essential for tackling complex problems that no single individual could solve alone.
Mentioned in This Episode
●Products
●Software & Apps
●Companies
●People Referenced
Common Questions
Jeff Dean predicts significant automation of Machine Learning (ML) systems, where ML systems will improve their own capabilities by running experiments, breaking down problems into sub-tasks, and using automated experimentation loops.
Topics
Mentioned in this video
An open-source software library for machine intelligence, developed by the Google Brain team. It is widely used for numerical computation and large-scale machine learning.
An AI system developed by DeepMind (part of Google) that predicts the 3D structure of proteins. It's a highly successful example of a specialized AI model in a specific scientific domain.
A highly capable yet efficient model from Google's Gemini family, which benefits from distillation techniques to achieve strong performance relative to its size and speed.
A distributed storage system developed at Google, designed to handle very large amounts of data across many commodity servers. It provides high scalability and performance.
Graphics Processing Unit, a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device. They are also used for general-purpose computing, including AI.
A multimodal large language model developed by Google AI, designed to understand and operate across different types of information including text, code, audio, image, and video. It is used as an API for context engineering.
A programming model for processing large data sets with a computer cluster in a distributed manner. It was foundational for distributed computing at Google and was developed by Jeff Dean, Sanjay Ghemawat, and others.
A computer scientist at Google, known for his contributions to large-scale distributed systems like MapReduce, Bigtable, TensorFlow, and TPUs. He is a prominent figure in AI and systems engineering.
A computer scientist at Google, a key collaborator with Jeff Dean on foundational systems like MapReduce and Bigtable, and instrumental in early Google search development.
More from Y Combinator
View all 616 summaries
31 minPatrick Collison: "What If You Succeed?"
77 minMulti-GPU Kernels, Intelligence per Watt, Heterogeneous Inference, and More | YC Paper Club
33 minAlexandr Wang: “This is a Once-in-a-Civilization Opportunity”
51 minBlake Scholl: The Problem With Becoming an Expert
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free