Key Moments
Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology
All-In PodcastWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
AI's exponential growth faces an energy crisis, potentially running out of power in 3 years. Unconventional AI's new '4D computing' promises 1000x efficiency gains by mimicking biology and reducing data movement.
Key Insights
Google alone processes 3.2 quadrillion tokens per month, with AI services consuming 12 gigawatts, a significant fraction of the world's total data center energy.
The cost of energy accounts for 50% of serving a token, highlighting the critical need for efficiency in AI computation.
Biological brains, like the human brain (20 watts) or a squirrel's brain (8 milliwatts), are orders of magnitude more power-efficient than current computing systems.
Unconventional AI's new physical 'dynamical computer' prototype was built in 5 months and generates images using only 500 nanowatts per image, a thousand times more efficient than GPUs (milliwatts).
The new '4D computing' approach integrates compute and memory, utilizing physical 3D die stacking and the time dimension to drastically reduce data movement.
The company aims to achieve 1000x power efficiency within three and a half years, ultimately seeking to 'beat biology' in computational performance and efficiency.
The looming energy crisis in AI
Naveen Rao, CEO of Unconventional AI, highlights a critical bottleneck facing artificial intelligence: energy consumption. He points to Google's staggering usage of 3.2 quadrillion tokens per month, which, even at a conservative 10 joules per token, demands 12 gigawatts of power – a significant portion of the world's total data center energy capacity. Rao estimates that at current growth rates, the world could run out of energy for AI within approximately three years. This energy demand is not a distant problem; it's already impacting data center strategy, with energy contracts now being the primary consideration, superseding even floor space. He notes that 50% of the cost of serving a single token is purely energy, underscoring the urgent need for radical efficiency improvements.
Biology as a blueprint for efficient computation
Rao contrasts the massive energy demands of current AI systems with the incredible efficiency of biological brains. The human brain, responsible for complex cognition, operates on a mere 20 watts. Scaling down, a monkey's brain uses about 1 watt, comparable to a smartphone. Even smaller animals exhibit remarkable efficiency: a rat's brain uses milliwatts, and a squirrel's, known for its agile and accurate movements, runs on just 8 milliwatts. This biological precedent suggests that intelligence does not inherently require vast amounts of power. The inefficiency in current computing, Rao explains, stems from excessive data movement. While the human cortex moves about 16 billion bits per second, high-end GPUs move trillions of bits in and out of memory per second, driving up energy consumption.
Moving beyond abstractions: towards dynamical systems
Traditional digital computing, built on layers of abstraction like ones and zeros, has reached its limits. These abstractions, while enabling speed, create inefficiencies by not fully utilizing the underlying physics. Rao's Unconventional AI is pursuing a different path: simplifying abstractions and connecting them directly to the physics of semiconductors, mimicking how biological intelligence emerges from the physics of neurons. This approach draws inspiration from dynamical systems theory, observed in phenomena like flocking birds or ant colonies, where complex emergent behavior arises from simple rules governing individual components. The company's research explores systems where computation is inherent in the physical dynamics, eliminating the need for traditional separate compute and memory units.
The UNO model: simulating synchronized oscillators for image generation
As a proof of concept, Unconventional AI developed the UNO model, an image generation system based on synchronized oscillators. Inspired by the synchronization of metronomes placed on a rolling plank, this system demonstrates how physical interactions can lead to emergent, intelligent behavior. Even with hundreds of oscillators starting at different phases, they naturally synchronize. UNO simulates this by using oscillators to generate images, demonstrating that this dynamical system approach can produce useful outputs. The company has made this simulation open-source, allowing others to explore this novel computational paradigm. Analysis of UNO’s state space trajectory shows how conditioning on desired outputs like 'airplane' or 'car' guides the system's evolution.
The holy grail: a physical dynamical computer
Rao reveals the creation of the first physical 'dynamical computer,' developed in just five months. This prototype leverages the concept of 'sparsity,' which involves strategically removing connections in a system without sacrificing performance, and in some cases, even improving it. Unlike traditional systems with 'N-squared' scaling (where connections grow quadratically with the number of elements), sparsity offers better scalability. The physical chip was designed and sent to fabrication in June, with results already demonstrating its capability. This system generates images using only 500 nanowatts per image, a thousand-fold improvement over the milliwatts typically consumed by GPUs. This dramatically reduced energy consumption is attributed to the absence of extensive data movement characteristic of traditional architectures.
Introducing '4D computing' and beating biology
This new paradigm is termed '4D computing.' It moves beyond the standard Von Neumann architecture (separate compute and memory) by integrating compute and memory within each element. The '4D' aspect refers to utilizing the physical three dimensions of die stacking (vertical integration) and the fourth dimension of time in its dynamic operations. The overarching goal of Unconventional AI is ambitious: to 'beat biology' in terms of computational efficiency. While current AI is billions of times less efficient than the thermodynamic limit of intelligence, Rao believes they can achieve significant gains within three and a half years, potentially reaching the limits of current 2D lithography. This efficiency leap could enable pervasive computing, including advanced robotics and decentralized data centers.
Productization and ecosystem challenges
The path to product involves creating a full-rack data center system within two years, designed to run AI models with radically different internal workings but standard network interfaces. Porting existing models will require work at the model layer, not the operations layer, as fundamental computational elements like matrix multiplication are implemented differently, as time-varying dynamics. The core challenge lies in bridging the gap between theoretical physicists and chip architects. Unconventional AI is building a Python-based library to express these time-varying stochastic behaviors, serving as a potential equivalent to CUDA for this new architecture. The ambition is to disrupt the trillion-dollar AI market, potentially triggering a 'Jevons paradox' effect where drastically reduced costs lead to even greater overall consumption and market expansion.
Mentioned in This Episode
●Software & Apps
●Companies
●Concepts
●People Referenced
Energy Consumption Comparison: Biology vs. AI
Data extracted from this episode
| System | Energy Consumption | Neurons/Elements |
|---|---|---|
| Human Brain | ~20 Watts | ~86 Billion |
| Monkey Brain | ~1 Watt | N/A |
| Rat/Bat Brain | Milliwatts | N/A |
| Squirrel Brain | ~8 Millwatts | N/A |
| Cell Phone | ~1 Watt | N/A |
| High-End GPU (System) | Order of Megawatts | N/A |
| Dynamical Computer (per image) | ~500 Nanojoules | N/A |
AI Data Center Energy Consumption Example (Google)
Data extracted from this episode
| Metric | Value |
|---|---|
| Tokens processed per month | 3.2 Quadrillion |
| Energy per token (estimated low) | 10 Joules |
| Estimated energy consumption for AI services | 12 Gigawatts |
Common Questions
4D computing refers to a new approach that utilizes the time dimension in system dynamics, combined with the three physical dimensions of die stacking (vertical and planar), to create a more efficient computing paradigm.
Topics
Mentioned in this video
Acquired Nirvana Systems, where Naveen Rao started and ran the AI group.
An AI chip startup focused on rethinking computer foundations for power efficiency.
Used as an example to illustrate the immense energy consumption of AI services, processing 3.2 quadrillion tokens per month.
The observation that the number of transistors on a dense integrated circuit doubles about every two years, which is noted as having largely ended for efficiency gains.
The concept that increased efficiency in resource use tends to increase rather than decrease overall consumption of that resource.
More from All-In Podcast
View all 457 summaries
24 minAdam Foroughi, Applovin CEO: Surviving a 92% Drawdown, Ads as ML 1.0 & the $50B Game Ad Market
39 minBill Gurley: Searching for Feynman
39 minJared Isaacman: A New Era for NASA and American Space Exploration
33 minNick Shirley: Exposing Government Fraud, Suing California & Taking on the Media
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free