Key Moments
Jim Keller: The Future of Computing, AI, Life, and Consciousness | Lex Fridman Podcast #162
Lex FridmanKey Moments
Jim Keller discusses computing, AI, consciousness, engineering, and leadership. He emphasizes craftsmanship, simplicity, and the evolution of hardware and software.
Key Insights
Engineering success relies on deep craftsmanship and attention to fundamental details, not just groundbreaking invention.
The greatest leaps in computing performance often come from unexpected, simple ideas or timely innovations, like JavaScript.
The evolution of computing hardware is moving towards more specialized, graph-native architectures for AI, contrasting with traditional GPU approaches.
Leadership involves balancing order with controlled chaos to foster innovation, and understanding human dynamics is crucial.
The future of computing includes widespread, efficient AI integration, potentially blurring the lines between computation and everyday objects.
Consciousness may be an emergent property of complex computational systems, driven by narrative, simulation, and interaction.
THE INTERPLAY OF THEORY AND ENGINEERING
Jim Keller draws a distinction between pure theory and engineering, defining engineering as the practical application of known methods and science as the pursuit of the unknown. He highlights that while groundbreaking ideas are essential, the true power often lies in the meticulous engineering and craftsmanship applied to those ideas. Examples like branch prediction in processors and the widespread adoption of JavaScript underscore how practical implementation and timing can be as crucial as the initial concept. Keller emphasizes that engineering is fundamentally about building things well, akin to a skilled artisan.
INNOVATION, CRAFTSMANSHIP, AND COMPUTING EVOLUTION
Keller argues that innovation, while critical, is relatively infrequent compared to the vast amount of engineering required to elaborate on those innovations. He uses the analogy of a building constructed from bricks, emphasizing that the quality of the bricks (components) and the skill in laying them (engineering) are paramount. He critiques a corporate culture that excessively rewards patent filing over fundamental craftsmanship, leading to neglected basics. The evolution of instruction sets like RISC vs. x86 and the rise of ARM in mobile are discussed through the lens of simplicity, accessibility, and the ability to adapt to market needs.
THE DRAMATIC SHIFT IN AI HARDWARE AND SOFTWARE
The conversation delves into the future of computing, particularly AI. Keller explains that traditional parallel computing structures like GPUs, optimized for tasks like rendering pixels, are not natively suited for graph-based AI computations. He introduces the concept of 'given parallelism' in AI, where computations are represented as graphs, a paradigm Tenstorrent's hardware is designed to address natively. This shift from emulating graph computations to fundamentally supporting them is seen as a key evolutionary step, distinct from the incremental improvements in traditional hardware.
LEADERSHIP, IDEAS, AND THE HUMAN ELEMENT
Keller reflects on leadership, drawing comparisons between figures like Steve Jobs and Elon Musk. He notes Jobs's ability to generate ideas and select talent, acting as a visionary who grounded his concepts in engineering reality through trusted intermediaries. The necessity of 'controlled chaos' in leadership is discussed: too much order stifles progress, while too little leads to disarray. Keller suggests that strong leaders often introduce a disruptive force to counter organizational ossification, a trait observed in both Jobs and Musk, though manifesting differently.
CONSCIOUSNESS, SIMULATION, AND THE BRAIN
The discussion touches upon the nature of consciousness, exploring it as a potential emergent property of complex computational systems. Keller posits that a sufficiently advanced AI, by simulating worlds, processing data, and creating narratives, might exhibit traits we recognize as consciousness. He considers the brain itself a complex, parallel processor with unique structural and functional characteristics, questioning current biological limitations and imagining future enhancements. The concept of 'software 2.0,' where data and networks program themselves, is presented as a path toward creating more sophisticated AI, potentially indistinguishable from conscious entities.
THE FUTURE OF COMPUTING AND HUMAN LIMITATIONS
Keller discusses the increasing integration of computing into everything, driven by cost-effectiveness and the ubiquity of basic computational capabilities. He suggests that while hardware performance will continue to improve, the richest source of progress may come from scaling computations across vast networks and exploring new software paradigms like 'software 2.0.' He also touches on the limitations of human perception and cognition, suggesting that future AI might not only surpass us in raw computation but also in its understanding of fundamental principles in ways we can't yet comprehend.
THE CHALLENGES OF SCALING AND EMBEDDED AI
The conversation touches upon the complexities of scaling AI systems, from small, power-efficient devices like potential phone integrations to massive, high-power training clusters. Keller highlights the importance of efficiency, particularly with power and memory, and the role of techniques like conditional graphs and sparsity in managing complex networks on limited hardware. This broad range of scaling requirements, from milliwatts to megawatts, necessitates flexible and robust architectures capable of adapting to diverse computational demands.
THE ROLE OF LOVE, FAMILY, AND FRIENDSHIP
Keller views love, family, and friendship not just as emotional constructs but as functional elements that drive novelty, surprise, and sustained engagement. He uses the analogy of parents closely monitoring children because they love them, preventing habituation. Similarly, loving one's work fuels the dedication needed for mastery. He emphasizes that these human connections provide balance and spaciousness, preventing mental stagnation and fostering resilience, ultimately contributing to a richer, more meaningful life beyond pure professional achievement.
NAVIGATING REGRET AND PERSONAL GROWTH
Reflecting on his career, Keller acknowledges experiencing regrets, particularly concerning personal interactions and a tendency to take slights too personally. He shares a psychologist's insight that others' actions are often driven by their own patterns rather than being solely about the recipient. Keller wishes he had developed a better understanding of human dynamics, particularly political maneuvering within organizations, to react less emotionally and more strategically. He sees growth in recognizing that others' behaviors are rarely personal, allowing for a more measured response.
THE NATURE OF INTELLIGENCE AND POTENTIAL ADVANCEMENTS
Keller contemplates the nature of alien intelligence and its potential technological divergence from human understanding. He suggests that sufficiently advanced beings might transcend the constraints of our perceived reality, perhaps uploading consciousness into more complex computational substrates or evolving in ways we can't fathom. This perspective challenges our anthropocentric view of intelligence and raises questions about whether our current forms of computation and AI are merely stepping stones toward entirely different, perhaps incomprehensible, forms of cognition.
THE COMPLEXITY OF HUMAN SUFFERING AND RESILIENCE
The discussion touches upon the dual nature of human experience, acknowledging both immense capacity for joy and profound potential for suffering. Keller shares his experience with Jordan Peterson's struggles with benzodiazepine withdrawal, highlighting the severe physical and psychological toll such dependencies can exact. He reflects on how the human brain's complexity might correlate with its susceptibility to intense suffering, while also noting the resilience and adaptive capacity observed in individuals and humanity navigating profound challenges.
THE INTERSECTION OF BIOLOGY, COMPUTING, AND THE FUTURE
Keller explores the accelerating convergence of biology, computing, and AI. He touches on brain-computer interfaces, the potential for AI to mediate our sensory experiences, and the concept of neuroplasticity. He questions whether limitations in human cognition are essential for our current form of intelligence or if they can be expanded through technological augmentation. The possibility of creating consciousness through computational means, or enhancing it through interfaces and AI, remains a central, albeit speculative, theme.
THE ROLE OF SURPRISE AND LEARNING FROM FLAWS
Keller emphasizes the importance of embracing surprise and learning from failures, viewing them as integral to significant innovation. He recounts his personal experience with early processor designs that, despite being groundbreaking, had critical flaws. He learned to reframe these 'failures' not as personal shortcomings but as valuable lessons that informed subsequent successes. This perspective highlights that true progress involves a willingness to tackle the novel and imperfect, rather than shying away from challenges due to the fear of mistakes.
ADVICE FOR YOUNG PEOPLE AND NAVIGATING LIFE'S JOURNEYS
Keller advises young people to pursue genuine interests, develop deep skills, and avoid being trapped by societal pressures or the opinions of previous generations. He stresses the importance of self-awareness, critical thinking, and avoiding groupthink. He also advocates for finding balance beyond work, nurturing relationships, and engaging in activities that foster mental well-being and resilience. Facing fears, particularly the fear of humiliation, is essential for personal freedom and growth.
STARTUP LEADERSHIP AND THE DYNAMICS OF TEAMS
When discussing startups, Keller highlights the critical role of a strong, passionate team. He notes that while powerful ideas can propel a company, understanding and managing people are crucial for sustained success. He emphasizes the importance of valuing and nurturing talent at all levels, not just those with groundbreaking ideas. Building a capable team requires astute observation of human dynamics and a commitment to fostering collaboration and engagement.
Mentioned in This Episode
●Products
●Software & Apps
●Companies
●Organizations
●Books
●Concepts
●People Referenced
Common Questions
Theory involves constructing models and generalizing how things work (science and discovering new things), while engineering focuses on building and implementing pragmatic solutions using known methods (craftsmanship). Good designs integrate both.
Topics
Mentioned in this video
A sponsor of the podcast, offering grass-fed meat.
A platform where the podcast can be supported.
A microprocessor company that initially licensed x86 to multiple parties, then out-innovated competitors to dominate the market.
A platform where the podcast can be followed.
A platform where the podcast can be subscribed to.
Used as an example of a company that achieved massive scale with its search engine, factoring out indexing and distributing data across many machines.
A sponsor of the podcast, offering an all-in-one nutrition drink.
A social media platform mentioned for connecting with the host and Jim Keller later referenced for communication during the pandemic and WallStreetBets.
A microprocessor company that competes with Intel, and Jim Keller worked there on the Zen design.
Company under Steve Jobs that pivoted multiple times, lauded for its innovation in product design and material choices.
Recently published papers on exponential improvement in neural network training efficiency.
Used as an example of adding computers to everyday objects, where a computer sensor in a battery was cheaper than a resistor.
A semiconductor manufacturing company from which chip designers buy wafers.
A company whose stock was at the center of a saga where a large number of distributed individuals took on elite hedge funds.
A sponsor of the podcast.
Jim Keller's current company, which builds hardware specifically for deep learning, focusing on natively running graph computations.
A company known for its GPUs, discussed in the context of their emulation of graph computations and their pivot attempts in autonomous vehicles.
An autonomous vehicle technology company that uses a self-supervised approach to driving, but questions remain about its data engine compared to Tesla.
Elon Musk's neurotechnology company, mentioned in the context of brain-computer interfaces to expand human capabilities.
A semiconductor manufacturing company from which chip designers buy wafers.
Mentioned as another company that demonstrated the financial value of scaling computing systems.
One of the autonomous driving startups using powerful servers in their vehicles, fitting the NVIDIA approach of "take the biggest thing and make it work."
Elon Musk's aerospace manufacturer and space transport services company, mentioned for its engineers' perspective on large power consumption being a small number.
A trading platform whose CEO, Vlad Tenev, demonstrated a lack of understanding of the underlying finance system during the GameStop saga.
An early microprocessor mentioned by Jim Keller.
An early microprocessor by Intel, mentioned with 6502 and Z80.
Considered by Jim Keller as an early 'great' processor from Intel.
Tensor Processing Unit, Google's custom-designed ASIC for neural network machine learning, mentioned in reference to Chris Lattner's work.
Tesla's supercomputer for AI training, an unconstrained computer solution for very large training problems. Jim was there when it started.
Tesla's advanced driver-assistance system, being iteratively improved with data-driven methods and a focus on self-supervised learning.
An early microprocessor, which some thought was a better instruction set than the 8086 but was proprietary.
Mentioned as Intel's first integrated microprocessor.
Another Intel processor considered 'great' by Jim Keller.
A product created by Apple under Steve Jobs' leadership.
Tesla's custom-designed chip for autonomous driving, with an open question whether its compute capabilities are sufficient for full self-driving.
An NVIDIA GPU system, mentioned in comparison to Tenstorrent's Grayskull processor's performance metrics.
A product created by Apple under Steve Jobs' leadership.
Apple computers, praised for their design (e.g., made of aluminum).
A brand of virtual reality headsets, mentioned as a technology that will become so good, projections will appear real.
A pioneering convolutional neural network that won the ImageNet LSVRC-2012 competition, mentioned as a point in the steady improvement of compute resolution for neural networks.
The Low-Level Virtual Machine, created by Chris Lattner, which became the intermediate representation for all compilers.
A programming language with a similar 'crappy but took over the world' story to JavaScript.
A platform where the podcast can be reviewed.
A product created by Apple under Steve Jobs' leadership.
A residual neural network architecture, mentioned as showing steady improvements over prior architectures like AlexNet.
A popular open-source machine learning framework, described as the native language for AI network programs and favored over TensorFlow by many.
A programming language mentioned as an attempt to solve many problems, but was complicated compared to JavaScript.
An NVIDIA computing platform and API model that enables parallel computation on GPUs, initially for HPC and later benefiting from the AI trend.
A free and open-source software library for machine learning, with some users reportedly switching to PyTorch.
Mentioned as an innovation that created massive opportunities for software engineering.
A Reddit community whose actions led to the GameStop trading saga, highlighting the power of distributed action against centralized powers.
A programming language developed by Apple, on which Chris Lattner worked, that became very successful after initial adoption resistance.
A programming language mentioned as a contrast to scripting languages like JavaScript.
A programming language created by Brendan Eich that became widely popular despite its initial perceived flaws, due to its timing and accessibility.
A large visual database designed for use in visual object recognition software research, mentioned in context of neural network training efficiency.
Mid-Level Intermediate Representation, a project by Chris Lattner to represent graph computation and coordinate heterogeneous computers, inspiring Tenstorrent's software stack.
An autonomous driving company that takes a different approach to NVIDIA, building simpler, cheaper chips for automotive cost and form factor.
A language model by OpenAI, remarkable for demonstrating unsupervised learning capabilities with essentially infinite data.
Mentioned as a key thinker in AI, who gave a talk on 'Software 2.0', advocating for data-programmed computers.
Credited with Intel's pivotal shift from DRAMs to microprocessors.
Creator of LLVM and Swift, and developer of MLIR; regarded as a brilliant engineer who creates platforms for other developers.
A writer quoted expressing a cynical view of love as a 'fog that dissipates with the first light of reality'.
Jim Keller's brother-in-law, a public figure and psychologist whose struggles with benzodiazepine withdrawal are discussed.
CEO of Tesla and SpaceX, described as more engineering-centric than Steve Jobs, with a deep interest in manuals and craftsmanship.
Science fiction author whose "Culture" novels depict super-smart AIs living in worlds of 'infinite fun'.
Creator of JavaScript, who famously wrote the language in 10 days.
Co-founder of Apple, remembered for his strong intuition, talent selection, and 'first principles' approach, creating a dynamic work environment.
Famous for generating a high volume of ideas, not just a high percentage of good ones.
A pioneering computer scientist, quoted at the end saying, 'Those who can imagine anything can create the impossible.'
Founder of Tenstorrent, and Jim Keller's first investor.
A researcher from Johns Hopkins conducting large-scale studies on psychedelics, opening a door to the community of psychonauts.
Author of 'The Black Swan', whose concept of lowering short-run volatility while creating 'tail events' in systems is mentioned.
Scientist and founder of Wolfram Research, known for his work on computational physics and his project on 'physics by equation'.
CEO of Robinhood, who was criticized for his limited understanding of the financial system's underlying mechanisms during the GameStop events.
A method for slowly tapering off benzodiazepines, described as involving "unbelievable hell" for patients.
A modular processor design Jim Keller worked on at AMD, known for its well-defined interfaces that improved quality.
A microprocessor instruction set architecture, described as arguably the worst but most popular due to early licensing strategies and Intel's innovation.
A concept described by Andrej Karpathy, where software is 'programmed' by data and networks, signifying a fundamental shift in software development.
The markup language for web pages, JavaScript code was initially inserted as comments into HTML.
A company that designs processor architectures, known for its diverse 'A, R, M series' processors and its friendly approach to the synthesis IP environment, dominating the mobile platform.
The observation that computing capabilities double approximately every two years; discussed in the context of hardware vs. software improvements.
An open-source instruction set architecture, which allows anyone to change it, but also risks too much random change without a common subset.
A news channel mentioned as a source of opinions that young people might spout without independent thought.
A news channel mentioned as a source of opinions that young people might spout without independent thought.
Where Jim Keller studied engineering, where tests were open book and emphasized methods over memorized facts.
A concept relating to RISC-V being open source, allowing anyone to change it, but with potential for too much random variation.
More from Lex Fridman
View all 505 summaries
154 minRick Beato: Greatest Guitarists of All Time, History & Future of Music | Lex Fridman Podcast #492
23 minKhabib vs Lex: Training with Khabib | FULL EXCLUSIVE FOOTAGE
196 minOpenClaw: The Viral AI Agent that Broke the Internet - Peter Steinberger | Lex Fridman Podcast #491
266 minState of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490
Found this useful? Build your knowledge library
Get AI-powered summaries of any YouTube video, podcast, or article in seconds. Save them to your personal pods and access them anytime.
Try Summify free