Key Moments
The AI Frontier: from FLOPs to Megawatts — Anjney Midha, AMP
Latent Space PodcastWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
AI compute utilization is surprisingly low, often below 70% MFU, indicating massive waste that common sense infrastructure practices could fix, and community backlash against data centers is a significant bottleneck.
Key Insights
96% node utilization should be standard, but most single-down clusters are not running at that, and best-in-class MFU (Model FLOPs Utilization) is only 60-70%.
Up to 20% of data centers in the US might face community backlash this year, risking project approval due to concerns about power grids and the environment.
Amp aims to be an independent system operator for compute, analogous to the electric grid, pooling supply and demand to make 'megaflops flow like megawatts'.
The 'bitter lesson' for AI scaling doesn't excuse abandoning common sense in infrastructure; AI scaling should increase the premium on robust infrastructure due to higher costs of wastage.
Anthropic's success is attributed to years of 'preparedness' and efficiency, focusing on a P0 mission (like coding) and having a strong 'culture of safety' that acts as a moat.
Venture capitalists often fail to recognize dynamic agents in AI, boxing researchers into narrow roles and overlooking that high-level scientific achievement often translates to strong CEO potential.
Wasted compute: the hidden cost of AI scaling
Anjney Midha highlights a critical inefficiency in AI compute utilization. While 96% node utilization is considered standard (an outage at Google if below 95%), most AI clusters operate far below this. Even more concerning is Model FLOPs Utilization (MFU), where the best-in-class performance is only between 60-70%. This indicates massive underutilization of expensive GPU resources. Midha attributes this not to a lack of funding or compute, but to a 'culture' problem and a lack of alignment between those funding compute and those deploying it. He argues that the rapid scaling demands in AI have led to compounded wastage, a phenomenon that common sense and iterative bring-ups, principles long understood in the semiconductor industry, could significantly mitigate.
Community backlash and infrastructure reliability
The expansion of data centers, crucial for AI development, faces significant community resistance. Midha notes that up to 20% of data centers in the US may be at risk this year due to community backlash, stemming from concerns over power grids, environmental impact, and permitting. This highlights a critical bottleneck that goes beyond technical capabilities. He proposes an innovative solution: data center operators could charge a marginal premium on compute (e.g., an extra $0.50 per hour) and direct these funds to local communities. This would create a clear public benefit, fostering community support and ensuring more reliable infrastructure, effectively turning potential opposition into partnership.
Amp's vision for a compute grid
Amp aims to revolutionize AI infrastructure by creating a 'compute grid' that functions like the electric grid, making 'megaflops flow like megawatts.' This involves a horizontal, multi-cloud, and multi-silicon approach focused on pooling and utilization, rather than vertical integration. Acting as an independent system operator (ISO), Amp will coordinate supply from various partners and demand from research labs and AI companies. This model, inspired by historical grid operators like PJM, focuses on neutrality and aggregation of uncorrelated demand to maximize utilization and create a fungible compute market, addressing the current fragmentation and stranded pools of compute.
The limits of 'move fast and break things' in AI
While the hustle and hacker mindset is valuable for startups, Midha argues that AI infrastructure requires a shift towards 'responsible infrastructure.' The uncontrolled pursuit of speed without stable foundations can lead to systemic failures. He draws a parallel to Mark Zuckerberg's evolution from 'move fast, break things' to emphasizing stable infrastructure. In AI, the margin for error and the cost of wastage are significantly higher, making common sense and robust infrastructure non-negotiable. Abandoning these principles in the name of AI progress is a mistake that will inevitably lead to an accounting for unforeseen consequences.
Venture capital's view on talent and culture
Midha criticizes the venture capital community's tendency to pigeonhole individuals, particularly scientists and researchers, into predefined roles. He argues that top-tier scientists, who have already demonstrated immense performance and discipline, often possess the qualities needed to be great CEOs. The example of Anastasia, co-creator of ChatGPT and founder of LM Arena, illustrates this point, showcasing her ability to excel not only in research but also in building impactful projects. Midha advocates for funding these 'star athletes of the mind,' recognizing that the drive for scientific rigor can translate into strong leadership and a mission-driven approach, rather than forcing them into conventional CEO molds.
Anthropic's preparedness and the culture of safety
Anthropic's success, particularly in coding capabilities, is attributed to a deliberate, long-term strategy of 'preparedness,' not just luck. For four years, the company has been highly efficient, focusing on a P0 mission (coding) and cultivating a 'culture of safety.' This involves a constant awareness of the risks associated with powerful AI systems and a commitment to responsible development, even if it means delaying product launches. This focus on safety, despite potential accusations of over-caution, has served as a durable moat, reinforcing their mission alignment and making them uniquely positioned to handle mission-critical AI applications.
The mission-driven approach at Periodic Labs
Periodic Labs exemplifies a mission-driven approach, centered on scientific breakthroughs, particularly in areas like superconductivity and end-of-life prediction in healthcare. The company's challenges, like attracting talent and navigating technical hurdles, serve as the 'hardship' that hones its culture. Midha emphasizes that true mission alignment, especially when resources are scarce, forces teams to define their P0 (priority zero) and make difficult trade-offs. This contrasts with labs that raise excessive capital too quickly, diluting focus and potentially leading to a fragile culture that fails to reach its full potential.
Output maximization and the nature of progress
The core philosophy driving Midha's work, and exemplified by 'output maxing,' is achieving optimal outcomes through AI. This involves pushing the frontier of capabilities while minimizing waste – whether it's computational resources, human potential, or healthcare expenditures. He sees AI as a tool to move beyond simplistic analogies and reasoning, encouraging a return to first principles. Progress, he suggests, comes from either standardizing protocols for lossless communication or developing entirely new capabilities that unlock abundance, making standardization less critical. This pursuit of maximal, efficient output is central to his vision for the future of AI infrastructure and research.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●Concepts
●People Referenced
Common Questions
Many AI labs struggle to ship products despite having sufficient cash and compute. The speaker diagnoses this as a cultural issue, where a lack of consistent action demonstrating mission alignment causes culture to fray.
Topics
Mentioned in this video
A scheduler developed at Google that Seb, the co-founder of Amp, built, highlighting the high utilization standards expected in data centers.
Amp's system which is a compute grid designed to function like the electric grid, pooling and utilizing compute across clouds to make megaflops flow like megawatts.
Mentioned as a company that built its own WebRTC (voice and video infrastructure) in-house, serving as a counter-example to using third-party infra and maximizing utilization by pooling demand.
Amp's capital business that incubates and invests in new frontier AI labs, similar to a venture capital arm.
A project started by Anastasios (a former debate champion and researcher) that was being used by millions of people as a side project.
Anthropic's AI model, discussed in the context of its coding capabilities and the company's early focus on safety.
Mentioned as a product co-created by Liam, who also worked on Gnomics at DeepMind.
Described as one of the most important tools to come out of DeepMind, created by Doge (a skip-level from Demis), and relevant to benchmarking frontier models.
An AI infrastructure business and compute grid company aiming to make megaflops flow like megawatts, acting as an independent system operator.
The parent company structure resembling Google, under which Amp Holdings operates, with subsidiaries for infrastructure and a capital business (Foundry) for incubating AI labs.
A company that Amp's fund invested in, highlighting its velocity and focus on the transformer architecture. It's also discussed in terms of its early focus on safety and the 'prepared mind' philosophy.
Mentioned as a place where extraordinary research has happened but much of it has never seen the light of day, sometimes due to publication embargoes, leading to adverse selection.
A chip company whose chips plug into NVIDIA's reference architecture, enabling them to innovate on systems co-design without needing to compete on every front.
Mentioned in the context of shifting from 'move fast, break things' to 'move faster, stable infrastructure', highlighting the need for responsible infrastructure.
Founder of General Matter, who spoke at Stanford about energy bottlenecks and proposed charging data centers an additional fee to benefit local communities.
A professor at Stanford who the speaker apprenticed with, focusing on end-of-life prediction using deep learning on patient data.
Founder of Rishi Valley School, a boarding school in India that enforced a minimalist and austere lifestyle, which the speaker experienced.
CEO of NVIDIA, credited with enabling companies like Matrox by publishing NVIDIA's reference architecture, making it open for others to build upon.
CEO of Anthropic, highlighted as an example of a scientist who has achieved immense success, demonstrating that being a great CEO requires performance comparable to top scientists.
Former head of AI at Meta, credited with creating LLaMA before starting Mistral, serving as another example of a leader demonstrating strong human leadership.
More from Latent Space
View all 227 summaries
77 min🔬 The Limits of AI in Science - Why We Need Self-Driving Labs — Joseph Krause, Radical AI
41 min⚡️Making DeepSeek v4 outperform Opus 4.7 with Taste — @AhmadAwais , CommandCode.ai
78 minWhen AI Agents Run Businesses — Lukas Petersson and Axel Backlund of Andon Labs
42 minSatya Nadella on AI: @NoPriorsPodcast x Latent Space Crossover Special at Microsoft Build 2026
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free