How does Accelerated Understanding's approach differ from traditional AI models for physics?

Unlike models trained on specific physical tasks, Accelerated Understanding builds a single, universal model. They incorporate the laws of physics as a guiding principle and leverage shared learning across different physics domains, rather than relying solely on data-driven methods.

What kind of context lengths can Accelerated Understanding's models handle?

Their models can handle extremely large context lengths, achieving up to a trillion context input during training and 5 trillion context during inference. This is significantly larger than typical language models and is enabled by their unique data handling and architecture.

How does Accelerated Understanding generate training data for physical simulations?

They primarily use numerical simulators to generate vast amounts of training data. They also employ curriculum engineering, starting with simpler equations and lower resolutions, and utilize PDEs as a training signal for self-improvement beyond the initial data quality.

What are the main application areas for Accelerated Understanding's technology?

Key application areas include semiconductors and energy. In semiconductors, they aim to improve chip design and production processes. In energy, they are involved in areas like geothermal exploration and designing physical objects with specific capabilities.

How does Accelerated Understanding address the 'sim-to-real' gap in physics simulations?

They address the sim-to-real gap by ensuring their simulation domains are wide enough to guarantee they are in distribution. They also use PDEs as a training signal for self-improvement, leading to models that can potentially surpass the quality of the training data itself.

What role do neural operators play in Accelerated Understanding's models?

Neural operators are fundamental to their architecture, enabling resolution invariance and flexibility in context lengths. This allows their models to efficiently handle different resolutions and scales within physics problems, unlike fixed-resolution video or vision models.

Key Moments

Faster Chips Designs That Don't Melt — Anima Anandkumar & Benedikt Jenik, Accelerated Understanding

Latent Space Podcast

Science & Technology6 min read28 min video

Sep 4, 2026|14 views|1

Save to Pod

Want to know something specific about what's covered?

We've already dissected every moment. Ask and we will deliver (with timestamps).

Key Moments

TL;DR

Accelerated Understanding is building a single AI model for the physical world, akin to GPT for language, capable of simulating diverse physics like fluid dynamics and semiconductors, but faces challenges in data availability and the inherent complexity of physical systems.

Key Insights

Accelerated Understanding aims to create a universal foundation model for the physical world, analogous to GPT's success in language, by training a single AI across diverse domains like fluid dynamics, semiconductors, and energy.

The model is designed to handle 4-dimensional data (3 spatial + time) and has been trained with up to a trillion parameters and achieved inference at 5 trillion context length, necessitating a reinvented sharding infrastructure due to data sample sizes exceeding typical accelerator memory.

Physics-informed neural networks (PINNs) are integrated, using the laws of physics as a guiding principle for self-improvement, providing dense feedback signals that are more effective than the sparse feedback (e.g., human feedback) common in language models.

Neural operators form the basis of the architecture, enabling resolution invariance and flexible context lengths, which is crucial for engineering design and scientific discovery where fixed resolutions assumed by video or vision models are insufficient.

The company is focusing initial commercialization efforts on semiconductors and energy sectors, aiming to unlock performance gains through physics-informed chip design and optimize processes like geothermal energy exploration.

While language models learn from a vast, often unstructured 'mess' of internet data, Accelerated Understanding can use numerical simulators to generate training data and engineer a curriculum, starting with simpler equations and lower resolutions for more effective learning.

Building a GPT-style model for physical systems

Accelerated Understanding is developing a singular foundation model designed to understand and simulate a wide array of physical systems, drawing inspiration from the success of large language models like GPT. The core hypothesis is that the universality and scalability observed in language models can be mirrored in the physical sciences. This approach aims to move beyond training specialized models for individual tasks, such as weather forecasting or fluid dynamics in catheters, towards a single, overarching model that can learn across diverse domains. The company's bet is on the emergence of similar scale and generalization capabilities across physics as has been seen in language, potentially revolutionizing scientific discovery and engineering design.

The challenge of physical data and the role of physics laws

Unlike language, readily available, structured data for all physical phenomena is scarce. Scientific discovery, by definition, involves exploring the unknown, meaning novel data won't exist in training sets. This reliance on purely data-driven AI is insufficient. Accelerated Understanding emphasizes the critical role of integrating the laws of physics into their models. These fundamental principles act as a guiding framework, ensuring that the AI's predictions and simulations remain grounded in physical reality. This integration addresses the 'sim-to-real' gap often encountered in AI for science, allowing the model to learn not just from observed data but also from established scientific principles.

Handling 4D data and achieving massive context lengths

The model is architected to handle complex, multi-dimensional data, including three spatial dimensions and the temporal dimension, without compression. This contrasts with approaches used for video models, which often average pixels or use patching. Accelerated Understanding's model processes this data in its raw form. This has led to the development of models capable of handling context lengths in the trillions – a scale far beyond typical language models. For instance, they've achieved trillion-context input during training and 5 trillion-context inference. This requires a fundamentally new sharding infrastructure because data samples are so large they don't fit into standard accelerators or even entire nodes, necessitating custom solutions for distributed training and computation.

The power of shared learning across diverse physics

A key finding is that a single, larger model trained on multiple areas of physics performs better than creating separate, equally large models for each domain. This demonstrates a significant benefit from shared learning, where knowledge gained from one physical system can enhance performance in another. The company has deliberately selected diverse physics areas – such as semiconductors, energy, and aerospace – to test this hypothesis. They've observed that even though these domains appear different, they share underlying principles like energy conservation, causality, and object permanence, which neural models can effectively learn. This cross-domain learning is crucial for achieving the envisioned universality.

Neural operators enable flexible resolution and inference

The model's architecture is based on neural operators, which are essential for achieving resolution invariance. This flexibility is vital because not all physical simulations require extreme detail. While a 5 trillion context length is achievable, it's reserved for the most complex problems. For design exploration or initial simulations, lower resolutions and context lengths are sufficient, allowing for efficient computation. This adaptability distinguishes Accelerated Understanding's models from other 'world models' that often assume a fixed resolution during training and inference, which can be a limitation for precise engineering or scientific applications. Neural operators allow the model to dynamically adjust its resolution based on the task's requirements.

Physics-informed learning and self-improvement

Accelerated Understanding leverages differential equations (PDEs) not just for generating training data via numerical simulators but also as a direct training signal. By checking how well the model adheres to PDEs, it receives dense feedback, enabling it to improve beyond the quality of the initial training data. This 'self-improvement' mechanism is more potent than the sparse feedback (like human ratings) typical in language model reinforcement learning. Furthermore, the company can engineer a curriculum, starting with simpler physics problems and gradually increasing complexity, which is a more effective learning strategy than the chaotic, unfiltered data ingested by language models. This approach allows the model to achieve a higher quality of results.

Commercialization in semiconductors and energy

The company is initially targeting the semiconductor and energy industries, where the need for advanced physical simulation is high. In semiconductors, this includes both chip design (digital-analog co-design) and the production process itself. The goal is to leverage the AI's physics understanding to unlock new performance levels by optimizing designs that might currently be constrained by traditional design flows. In the energy sector, applications range from designing physical energy systems to analyzing observational data for resource exploration, such as geothermal energy or critical mineral identification. The universality of the model means that as it improves across various physics domains, more doors open for diverse applications.

Future scaling and impact

Accelerated Understanding plans to continue scaling both its models and its company, responding to significant inbound interest following its emergence from stealth. The journey involves tackling increasingly complex physics phenomena relevant to scientific discovery and invention, while simultaneously ensuring value delivery to commercial customers. The company emphasizes that this is just the beginning, with considerable room for growth in model scaling and application breadth. The ultimate vision is to build AI capable of addressing some of the world's most challenging scientific and technological problems through a deep, universal understanding of the physical world.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●Concepts

●People Referenced

Common Questions

Accelerated Understanding is developing universal AI models for physical simulation and understanding. They aim to apply the success of large language models, which handle diverse language tasks with a single model, to various physical domains like energy, semiconductors, and aerospace.

Topics

AI & Machine Learning Technology & Innovation Science & Mathematics Physics Simulation Scientific Discovery Energy Sector Universal AI Models Large Context Windows Neural Operators Semiconductor Design Multiscale Physics

Mentioned in this video

People

Anima Anandkumar

Co-founder of Accelerated Understanding, recognized as one of Time magazine's 100 most influential people.

Companies

Accelerated Understanding

A company developing universal AI models for physical simulation and understanding, aiming to apply the success of large language models to the physical world.

OpenAI

Company that pioneered the approach of training single, large models for multiple language tasks, which inspired the strategy of Accelerated Understanding for physics.

TSMC

A semiconductor manufacturer mentioned in the context of PDKs (Process Design Kits) and manufacturing requirements for chip design.

Organizations

Time Magazine

Publication that recognized Anima Anandkumar as one of the 100 most influential people of the year.

Concepts

Fourier neural operators

A specific type of neural operator mentioned as an example of inductive biases used in models, operating in Fourier space and having biases about geometry.

Software & Apps

transformer architectures

Architecture widely used for language models, which Accelerated Understanding's models move away from due to its quadratic complexity, which is infeasible for large physical simulations.

Ask anything from this episode.

Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.

Get Started Free