Key Moments
The Case for AI That Improves Itself | Deep Dives with a16z
a16zWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Self-improving AI can accelerate scientific discovery by drastically reducing the need for human AI research teams. This disruptive technology, while powerful, raises questions about accessibility and control, pushing towards a future where businesses can own and optimize their own AI infrastructure.
Key Insights
Mirendil's founders believe self-accelerating AI can reduce the need for large, specialized AI research teams from hundreds of people down to as few as two, significantly speeding up scientific and technological progress.
The jump from models like Sonnet 3.5 to 4.5 shows material improvements in longevity and oversight reduction, where even tiny decreases in oversight can lead to significant increases in token spend and outcome efficiency.
The current business model for many AI companies, which is to train a model and charge for its use, creates an incentive misalignment for sharing technology that could enable others to train their own models.
Accelerating scientific discovery with AI is seen as a positive good, a counterpoint to negative narratives about job automation, with a preferred outcome of general prosperity and scientific advancement.
While scaling up AI models is important, Mirendil emphasizes scaling up systems composed of people and AI agents, focusing on favorable system scaling where increased agents lead to proportionally faster problem-solving, not diminishing returns.
From Google and Anthropic to Mirendil: A Drive for Accelerated Science
Behnam Neyshabur and Harsh Mehta, formerly of Google and Anthropic, founded Mirendil with a core question: what happens when AI can meaningfully contribute to its own development? Their journey, starting from early research at Google's Blue Shift Labs during the scaling law revolution at OpenAI, was driven by a desire to accelerate scientific and technological progress. They observed that major AI labs were diverging from this goal, leading them to establish Mirendil to focus on building and making available self-accelerating AI, which they see as a disruptive yet crucial technology for advancing science and technology broadly.
Redefining the scientist: AI as a research accelerator
The traditional scientific method relies on deep domain expertise and iterative experimentation. Mirendil aims to equip AI systems with the primitives to conduct research and engineering across various scientific domains, particularly those with a digital component. They contrast their approach with how major AI labs operate, emphasizing that advancing science often requires superhuman capabilities. Instead of building an AI scientist to simply do the work of a human scientist, Mirendil focuses on creating AI systems that can autonomously conduct research and engineering at a high throughput, minimizing the bottleneck of assembling and iterating with human AI teams. This approach aims to reduce the necessity of large, specialized AI teams, potentially from hundreds of experts down to a handful.
Enabling self-accelerating AI: From primitives to research capabilities
Mirendil's vision of self-accelerating AI goes beyond simple task execution. They define it as AI systems capable of conducting AI research and engineering itself. This includes everything from low-level system optimization to high-level scientific inquiry. They refer to instances like AlphaGo's self-play loops as early forms of self-acceleration, and more recently, systems that learn and improve within unknown environments, such as the ArcAGI benchmark. Their specific focus is on developing AI that can perform the work of an AI researcher or engineer, iterating on solutions at a high rate. This capability, they believe, is the shortest path to broadly accelerating all areas of science. The progress is evident in model improvements, like the jump from Sonnet 3.5 to 4.5, where even small reductions in required human oversight lead to significant efficiency gains and reduced token expenditure.
Disruptive technology and the business model challenge
Self-accelerating AI is inherently disruptive, posing challenges to existing business models. Mirendil argues that companies whose primary model is training large models and charging for their use are not incentivized to share this technology. Sharing would empower others to train their own models, reducing reliance on the provider. This creates a fundamental misalignment with the goal of broad scientific acceleration. Mirendil's founders observed that disruptive technologies often require rethinking company structure, culture, and incentives to flourish. They aim to build a company and a business model that actively promotes accessibility and enables others to leverage this powerful technology, rather than hoarding it.
Beyond narrow AI: The importance of general capabilities
The effectiveness of AI in conducting research and engineering hinges on its general capabilities. Mirendil suggests that focusing too narrowly on specific tasks can lead to 'dumb mistakes,' like incorrectly wiring code or lacking the mindset of a domain expert. The significant advancements observed, such as the jump between Sonnet model versions, demonstrate material improvements in how long an AI can sustain focus on a problem and where it requires human oversight. Reducing this oversight, even marginally, has substantial implications for token usage and overall effectiveness. Therefore, cultivating more general and intelligent AI models is crucial for systems that can autonomously conduct research and engineering tasks.
From individual agents to cooperative systems for progress
Mirendil views self-acceleration not just as a single model improving itself, but as an evolving ecosystem of AI systems, potentially including humans. This system, functioning as a more intelligent whole, can then aim to improve itself. This 'gradual view' of self-acceleration is more realistic than purely sci-fi scenarios. They emphasize that these systems, composed of specialized and generalist models, and potentially human collaborators, aim for continuous self-improvement by developing the next generation of AI to be part of future, more capable systems. This is analogous to how a company, as an organized system, can achieve more than an individual brilliant person, but it requires solving complex organizational and coordination problems within the AI ecosystem itself.
Scaling systems, not just models, for favorable progress
A key focus for Mirendil is the concept of 'favorable scaling' within systems. Unlike human organizations, where productivity can decrease as size increases, they envision AI systems where doubling the number of agents leads to a proportional increase in problem-solving speed. They liken this to the competitive advantage companies gain from reaching breakthroughs faster. This requires solving complex technical challenges related to inter-agent communication, resource allocation, and incentive structures, areas mirroring issues faced by human researchers. Mirendil sees itself as an experiment in building these scalable systems, starting small to refine the scaling laws before larger deployments.
A positive vision for AI's future: Accelerating science for humanity
Mirendil aims to counter a more negative narrative surrounding AI, which often focuses on job automation. Instead, they champion a future centered on accelerating scientific discovery, offering a 'pure good for humanity.' Their mission is to direct powerful AI technology towards long-standing, challenging problems like solving Alzheimer's disease, which require more than just incremental AI scientific improvements. They believe that by removing intelligence as a bottleneck, other constraints can be addressed, leading to significant breakthroughs and moving beyond the limitations of current models that may take decades to achieve such goals. This focus on solving critical global challenges represents a purposeful direction for AI development.
Mentioned in This Episode
●Software & Apps
●Companies
●Books
●Concepts
Self-Accelerating AI for Scientific Advancement: Key Principles
Practical takeaways from this episode
Do This
Avoid This
Common Questions
Merindel is a new company focused on developing self-accelerating AI technology. Their main goal is to make this technology widely available to accelerate scientific research and engineering, ultimately contributing to general prosperity and a deeper understanding of ourselves and the universe.
Topics
Mentioned in this video
A previous company where Benham was a founding member.
The company where Benham and Harsh previously worked as research scientists and where they discussed ideas for accelerating science with AI.
The company where Benham and Harsh worked together previously.
A new company founded by Benham and Harsh, focused on developing self-accelerating AI to advance scientific research and engineering.
An example of self-accelerating AI from the history of the field, utilizing self-play loops.
Mentioned as a project where they first built math-specialized models.
A machine learning library that Merindel's models aim to be proficient with.
A previous AI model mentioned for its improvements and the jump in capabilities.
A machine learning library that Merindel's models aim to be proficient with.
The state-of-the-art model mentioned from when the speakers joined Anthropic.
More from a16z Deep Dives
View all 59 summaries
53 minWhy Retention Still Defines Product-Market Fit | Deep Dives with a16z
49 minTeaching AI the Language of Design | Deep Dives with a16z
42 minBuilding the Future of Image Generation with Ideogram's CEO
50 minHow Exa is Building the Perfect Search Engine | Deep Dives with a16z
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free