Key Moments
A Worm With 302 Neurons Inspired Their Architecture — Ramin Hasani, Liquid AI
Latent Space PodcastWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
Liquid AI's neural networks, inspired by a 302-neuron worm, can control robots with uncanny efficiency, potentially revolutionizing on-device AI by shifting computation away from data centers.
Key Insights
Liquid AI's neural networks are inspired by the C. elegans worm, which has only 302 neurons but can control 95 muscle cells, demonstrating remarkable efficiency.
The company has scaled its Liquid Foundation Models from tens of millions to 70 billion parameters, releasing LFM2 and LFM2.5 instances optimized for CPUs and other devices.
Liquid AI is partnering with companies like AMD and Mercedes-Benz to deploy its models on 300 million AI PCs annually and in next-generation cars, respectively.
Shopify is using Liquid Foundation Models, processing over a billion requests per month through its platform, showcasing significant production adoption.
Liquid AI's open-source models have achieved over 40 million downloads, with approximately 1.5 million downloads per week, indicating strong community interest.
The company is developing a self-serve platform for model development and customization, aiming to empower enterprises to build and deploy their own AI solutions.
Worm-inspired neural networks achieve high efficiency
Liquid AI's journey began over 12 years ago with a desire to integrate continuous-time mathematics into machine learning, drawing inspiration from biological nervous systems. Rather than focusing on the complex spiking neurons of the human or fruit fly brain, the researchers turned to the C. elegans worm, a simple organism with just 302 neurons. This choice was strategic: the worm's neurons exhibit electrotonic behavior (they don't spike) and their dynamics are governed by first-order differential equations, making them mathematically tractable and differentiable. This allowed the team to develop 'liquid neural networks,' also known as Liquid Time Constant Neural Networks. A key insight was that with only 302 neurons, the worm could control 95 muscle cells with greater efficacy than contemporary robotic systems. This efficiency, combined with the fact that humans share approximately 78% of their genome with C. elegans, suggested a powerful, scalable model for artificial intelligence. This foundational research led to the development of Liquid Foundation Models (LFMs).
Scaling intelligence from worms to foundation models
The initial success with controlling robots using small subsets of these worm-inspired networks (tens to hundreds of neurons) spurred further research. The team, including co-founders from MIT, explored continuous-time dynamical systems and their application to sequence modeling. While early recurrent neural networks (RNNs) suffered from sequential computation bottlenecks, and transformer architectures offered scalability through parallelizable matrix multiplications but at a high computational cost (quadratic complexity with attention), Liquid AI sought a balance. They investigated techniques like linearization, as seen in State Space Models (SSMs), to enable parallel computation. However, they found that linearization often sacrificed expressivity, particularly for tasks like language understanding. This led Liquid AI to develop a meta-AI system that searches for optimal 'hybrid architectures' by combining elements from various successful models (like convolutions, attention, and their original liquid networks) to achieve efficiency without sacrificing quality, minimizing memory and latency, and maximizing speed.
Liquid Foundation Models: Efficiency and diverse applications
Liquid AI has evolved its architectures significantly, moving from early 'messy' combinations to more refined models like LFM2, which is approximately 80% 1D convolutions and 20% grouped query attention, optimized for CPU performance. The company now operates at LFM 2.5 instances and is scaling these models from tens of millions to 70 billion parameters. These models are designed to be highly efficient and adaptable, making them suitable for deployment outside traditional data centers. Liquid AI focuses on bringing intelligence to edge devices, working with enterprises in automotive (Mercedes-Benz), PCs (AMD), and mobile sectors. For instance, their models are being integrated into cars for in-car intelligence and AI PCs for privacy filters and proactive agents, aiming to reduce cloud dependency and costs. The company also emphasizes the production-readiness of its models, noting that developing enterprise-grade AI for devices is significantly more challenging than simple benchmarking.
Production deployments and enterprise partnerships
Liquid AI's technology is not just theoretical; it's actively used in production. Shopify, a major partner, leverages Liquid Foundation Models across its platform, handling over a billion requests per month. These models are deployed in private environments where low latency and high quality are crucial for merchant and client-facing products. Another significant client is Mercedes-Benz, which is integrating a multimodal foundation model (around 600MB) into its North American Generation 3 vehicles, enabling voice interaction, reasoning, and camera-connected functionalities. This deployment highlights the company's ability to deliver production-grade AI for resource-constrained environments. The open-source release of their models has also seen substantial traction, with over 40 million downloads, demonstrating broad adoption and interest from developers.
The future: Self-serve platforms and advanced research
Liquid AI is developing a self-serve platform to democratize AI model development and customization for enterprises. This platform aims to automate the complex process of building, deploying, and continuously learning models, mirroring the quality Liquid AI achieves internally. While currently in beta, it represents a shift towards enabling clients to own their intelligence, moving beyond inference token monetization. In terms of research, Liquid AI is heavily invested in massively multimodal systems capable of processing and generating various data types (video, audio, text) simultaneously. They are also focusing on longer-horizon reasoning, reliability for on-device intelligence, and efficiency improvements across their entire infrastructure. Exploration into non-human understandable data, such as DNA foundation models, and advancing embodied and adaptive intelligence are also key research areas.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●Studies Cited
●Concepts
●People Referenced
Common Questions
Liquid AI's architecture was initially inspired by the C. elegans worm, specifically its 302 neurons that exhibit electrotonic behavior similar to artificial neurons. This inspiration helped in developing continuous-time, differentiable neural networks.
Topics
Mentioned in this video
A microscopic nematode worm with 302 neurons that served as a key inspiration for Liquid AI's neural network architecture due to its neurons' behavior resembling artificial neurons and its efficiency in controlling muscle cells.
A type of continuous-time dynamical system used in machine learning, mentioned as part of the broader field Liquid AI contributes to.
A toolkit for developing and comparing reinforcement learning algorithms, used by OpenAI during the period mentioned.
An alternative AI architecture that was brought together with other approaches for research and development at Liquid AI.
Graphics Processing Unit, hardware commonly used for accelerating AI computations, mentioned in the context of deployment environments for Liquid AI models.
The second generation of Liquid Foundation Models, optimized for CPU performance with a simplified architecture combining convolutions and attention.
A large language model developed by OpenAI, mentioned as an example of a cloud-based AI service that Liquid AI's on-device models can complement or replace for certain tasks.
Apple's AI features, mentioned in the context of on-device vs. cloud-based AI usage, noting that most current implementations rely heavily on the cloud.
Samsung's AI features, discussed alongside Apple Intelligence as examples of current on-device vs. cloud AI distribution.
A programming language developed by Modular, aiming to simplify AI development and custom kernel work, but potentially becoming ecosystem-locked.
A machine learning framework, which Liquid AI switched from to PyTorch, indicating a shift in their development tools.
A popular open-source machine learning framework, which Liquid AI adopted after previously using Jax.
An optimized C++ implementation for running LLaMA models, considered a baseline for on-device AI deployment that Liquid AI currently finds superior to some alternatives.
A non-transformer architecture inspired by RNNs, discussed as another popular alternative in the AI landscape.
The university where the founders of Liquid AI began their research into continuous time mathematics within machine learning systems.
Massachusetts Institute of Technology, where Liquid AI's founders continued their research and development, collaborating with leading figures in robotics.
A research group contributing to the field of continuous time dynamical systems.
A research lab that contributed to the field of continuous time dynamical systems.
An institution whose researchers contributed to the field of continuous time dynamical systems.
Professor at MIT, considered a pioneer in robotics, and a co-founder of Liquid AI, who invited the founders to MIT to further their research.
Co-founder of Liquid AI and researcher at MIT, instrumental in the development of continuous time dynamical systems and their application across various domains.
Researcher whose lab contributed to the field of continuous time dynamical systems, mentioned as part of the broader research community.
A respected figure in computer science, known for his work on LLVM and Swift, and now leading the Mojo development at Modular.
An individual who discussed State Space Models (SSMs) and their effectiveness, particularly for voice processing, with the speaker.
Founder of Groq, known for developing custom AI accelerators (TPUs), mentioned in the context of fundamental hardware work.
A class of models used in machine learning, particularly for sequence modeling and dynamical systems, representing a linearized version of recurrent neural networks that allows for more scalable computation.
A prominent deep learning architecture known for its scalability and effectiveness, particularly in natural language processing, contrasted with recurrent neural networks.
The foundational architecture for many modern large language models, celebrating its 10-year anniversary, with discussion on whether attention is the only component needed.
A leading AI research lab, mentioned as a competitor and benchmark in the field of reinforcement learning and robotics research during the 2017-2020 period.
A technology giant mentioned in the context of competing with Liquid AI from a research standpoint, particularly in scalable AI development.
The company founded by Ramin Hasani and others, focused on developing novel AI architectures inspired by biological systems, emphasizing efficiency and continuous time dynamics.
An e-commerce platform that partners with Liquid AI, utilizing their foundation models to improve latency and quality for merchant and client-facing products.
A technology company partnered with Liquid AI to bring their foundation models to AI PCs, focusing on local deployment of intelligence.
A luxury car manufacturer partnering with Liquid AI to integrate advanced in-car intelligence, including multimodal capabilities, into their vehicles.
A company that collaborated with Liquid AI, showcasing a demonstration of their work using Liquid's vision-language models for industrial robot operations.
An Italian company working with Liquid AI, utilizing their vision-language models for modular robots, particularly for instruction-following capabilities.
A technology company whose ecosystem Mojo is becoming locked into, a point of interest for Liquid AI's strategy regarding hardware-specific optimization.
More from Latent Space
View all 251 summaries
87 minThe Watchdogs of AGI — Rune Kvist of AI Underwriting Company
94 minRecursive Self-Improvement: from Auto Research to Superintelligence — Richard Socher, Recursive
41 minInside the Team That Killed Mandatory Code Review — Quinn Slack, AMP
28 minFaster Chips Designs That Don't Melt — Anima Anandkumar & Benedikt Jenik, Accelerated Understanding
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free