Matt Botvinick: Neuroscience, Psychology, and AI at DeepMind | Lex Fridman Podcast #106

He increasingly sees them as a single science. For him, neuroscience's purpose is to study what the brain is for—producing adaptive behavior, which is what psychology investigates at a functional level. The rest of neuroscience is about understanding how these operations are mechanistically carried out at a neuronal level. (Timestamp: 335)

While psychology in its own right is hugely important and generative, a complete understanding of the mind, for Botvinick, requires grounding cognitive metaphors in physical, neuronal mechanisms. He draws an analogy to Mendelian genetics preceding the discovery of DNA structure, where metaphors were useful but eventually needed a physical basis. (Timestamp: 895)

He was drawn into science by connectionism (what is now called deep learning). Early neural networks were primarily used to model human cognition, focusing on the richness and complexity of human processes like past tense formation, which involved 'quasi-regular' rules and exceptions. (Timestamp: 1179)

He is fascinated by the paradox that the brain, while so mysterious and distant in terms of our understanding, is simultaneously responsible for the 'full transparency of everyday life' – what makes everything obvious and familiar. It's the intimately close organ that produces our entire subjective experience. (Timestamp: 1385)

The prefrontal cortex, located at the front of the brain, is primarily involved in flexibility and guiding behaviors that require overriding habits or doing something unusual. It allows us to adapt to new contexts and make goal-directed decisions rather than relying on automatic, habitual responses. (Timestamp: 2701)

Matt Botvinick holds the classical view that the primary mechanism is a 'rate code' – the frequency at which individual neurons are spiking. While acknowledging ongoing debates about precise spike timing or dendritic computations, he believes the rate code, analogous to unit activity in deep learning, remains a reasonable approximation. (Timestamp: 3229)

Meta-learning is 'learning to learn,' where one learning algorithm gives rise to another. DeepMind research found that in recurrent neural networks trained on interrelated tasks, meta-learning spontaneously emerges, resembling the distinction between synaptic and activity-based memory in the brain. The prefrontal cortex, with its recurrent connectivity, might implement reinforcement learning within its dynamic activity patterns, shaped by slower, dopamine-mediated synaptic learning. (Timestamp: 3653)

Traditional reinforcement learning represents future rewards as a single expected value. Distributional coding, however, represents rewards as a distribution of possible outcomes, which empirically accelerates learning by preserving richer internal representations. New research suggests dopamine, known for signaling reward prediction error, might employ such a distributional code in the brain, respecting the variety of future outcomes, not just collapsing them to a single number. (Timestamp: 6040)

Botvinick anticipates a convergence where neuroscience, with its new technologies and renewed interest in complex behavior, informs AI research. For AI, the key challenge is building flexible systems that are good at many things, can switch tasks easily, and quickly adapt to novel situations through abstraction and cognitive control, similar to the prefrontal cortex's function. (Timestamp: 6480)

Human-AI interaction is crucial for AI to truly 'make the world a better place,' moving beyond just philosophical discussion. AI agents need to learn from and adapt to human preferences, navigate conflicting human desires, and understand cultural values. This field opens up research into economics, social choice theory, and even political theory to shape AI development responsibly. (Timestamp: 5105)

Drawing on social psychology, Botvinick suggests that beyond 'ability,' AI needs to develop 'warmth' to be truly human-like. This would involve displaying caring, compassionate behavior that feels genuine. Engineering such intuitive and gratifying human interaction is an open scientific question, and achieving it would be the 'ultimate Turing test.' (Timestamp: 6791)