Jay McClelland: Neural Networks and the Emergence of Cognition | Lex Fridman Podcast #222

A book written by David Rumelhart and Don Norman, which McClelland describes as exploring interesting questions about cognition in an abstract, non-biological way.

A seminal book co-written by Jay McClelland and David Rumelhart in the 1980s that explored ideas of neural networks, paving the way for many concepts in the machine learning revolution.

A book published in 1967, which Jay McClelland mentions marked the emergence of cognitive psychology as a field, initially viewing the study of the nervous system as peripheral to understanding the mind.

An AI research division of Google, mentioned as a key player in scaling AI computations for advanced intelligence.

Where Jay McClelland was a student in 1968 during the Vietnam War protests and a major student revolution, which led him to reflect on human behavior and psychology.

A journal that accepted McClelland's experimental work but advised him to remove his theoretical section, pushing him to further develop his theories.

A theoretical journal where McClelland later submitted his theoretical work, despite initial rejection, encouraging his continued development as a theorist.

Where Francis Crick was based when he became involved with the PDP research group.

An academic honor society for which Jay McClelland was nominated despite his grades, based on his deep interest in ideas.

Where Jay McClelland became an assistant professor in 1974 and where David Rumelhart was one of the first assistant professors in the psychology department.

Jay McClelland was a member of its National Advisory Mental Health Council around 2000. He describes a shift in policy towards a purely biological and pharmacological approach to mental illness, which he criticizes.

Where Jay McClelland is a cognitive scientist and where David Rumelhart studied mathematical psychology and later retired from his professorship.

A linguist whose PhD dissertation discovered that the intuitions of formally trained linguists about language meaning and grammar differed significantly from those of ordinary people.

American poet whose phrase 'road less taken' is invoked to describe McClelland's tendency to pursue less obvious and more exciting scientific paths.

Linguist who argued that a genetic fluke led to language, separating humans from other animals. McClelland contrasts this with a broader evolutionary perspective involving sociality.

An engineer and psychology professor who used linear algebra to create neural network models of perception, categorization, and memory, inspiring Jay McClelland.

A well-known figure in neural network circles, one of the postdoctoral scholars who joined Rumelhart's group.

Co-discoverer of DNA, who, while at the Salk Institute, heard about the PDP research group and joined, highlighting the interdisciplinary appeal of the early neural network movement.

Co-authored the Parallel Distributed Processing book and the backpropagation paper. Jay McClelland's long-time collaborator and friend, who later succumbed to semantic dementia. Regarded as a brilliant and driven scientist.

Naturalist and biologist who developed the theory of evolution by natural selection. McClelland mentions Darwin's initial fear of his own ideas about the complexity of the eye and unguided processes.

Author and cognitive scientist who used the analogy of sand dunes to illustrate how higher-level cognitive entities emerge from lower-level elements, being real but fluid.

Physicist who suffered from a degenerative motor system condition, contrasted with David Rumelhart's semantic dementia, both illustrating the profound impact of neurological conditions.

A psychologist and economist known for his work on judgment and decision-making, mentioned in the context of great collaborations in science.

A philosopher whose ideas about the body as a machine and thought as a divine intervention shaped early views on the mind-body problem, contrasted with the biological perspective of neural networks.

An Italian film director known for appreciating the role of 'magic' and trickery in creating illusions that move people, paralleled with the emergent nature of thought.

A professor in the electrical engineering department at Stanford who, with a collaborator named Hoff, invented the Delta Rule for gradient descent in a single layer of connection weights.

A logician from whom Jeff Hinton is a descendant, influencing Hinton's desire to understand reasoning itself and connect the boolean tradition with probabilistic, constraint satisfaction realms through the Boltzmann machine.

Criticized a narrow formal view of mathematics, likening studying it as symbol manipulation to studying music without hearing a note, emphasizing the intuitive aspect of mathematics.

A chip designer engineer whose unique way of thinking, like Feynman's, is transformative to be around, illustrating the magic of cross-disciplinary collaboration.

Philosopher who believed systematic thought was an essential, innate characteristic of the human mind, contrasted with the idea of it being a derived and acquired cultural trait.

A physicist known for his unique visual and intuitive thinking style, brought up as an example of a brilliant mind similar to Jeff Hinton in combining clarity with intuition.

Jay McClelland's collaborator who developed the Pyramids and Palm Trees test to assess conceptual grounding in patients with semantic dementia.

A biologist who, in the late 19th century, performed surgery to demonstrate the presence of a hippocampus in chimpanzees, supporting the continuity of species.

Co-authored 'Explorations in Cognition' with David Rumelhart, and as the senior figure, fostered a spirit of playful exploration of ideas in the early cognitive science community.

Preceded Euclid in developing formal mathematical systems, indicating an early historical presence of formal thought.

Known for his stage theory of child development, which observes profound, discrete transitions in cognitive abilities at different ages.

A researcher who had been writing about neural networks since the 1960s, recognized as an early thinker in the field.

A specific form of progressive neurological condition involving the loss of meaning, where patients lose the ability to appreciate the meaning of experiences from various modalities, impacting conceptual understanding.

An alternative algorithm based on the concept Jeff Hinton was pursuing concurrently with backpropagation, published in 1985, which turned out to be less effective at the time.

A type of deep learning system described as a set of collections of neuron-like units in layers, performing massively parallel computation for tasks like image classification, abstracting raw pixel input to categories.

A class of neural network architectures, an idea that Jeff Hinton essentially introduced in one of his 1981 papers, highlighting his foresight in the field.

A test developed by Carolyn Patterson to assess the loss of conceptual grounding in semantic dementia patients, using words or pictures to determine their ability to relate concepts.

A cellular automaton, mentioned as an example where very rich, complex, organism-like behavior emerges from simple rules, illustrating the magic of emergence.

A concept in evolutionary biology suggesting long periods of stasis punctuated by sudden changes, which McClelland relates to stages of mental abilities and cognitive development.

A field involving the use of mathematics to model mental processes, similar to cognitive modeling, focusing on deriving behavioral predictions from fundamental principles and equations rather than computer simulations.

An early paper by Jeff Hinton in 1981 that inspired him, Rumelhart, and McClelland throughout the 1980s and continues to ground McClelland's thinking about the semantic aspects of cognition.

An optimization mechanism for training neural networks, co-authored by David Rumelhart and Jeff Hinton. It involves backpropagating error signals through layers to adjust connection weights, initially called the generalized delta rule.

An algorithm invented by Bernie Widrow and a collaborator named Hoff for gradient descent in single-layer neural networks, extended by Rumelhart into the generalized delta rule (backpropagation).

An optimization concept familiar to engineers, where connection weights are adjusted to minimize error or 'loss' in a system, introduced by Jeff Hinton as a way to approach neural network learning.

A term used to describe ideas where knowledge in a system resides in the connections between units, rather than explicit symbols or a separate dictionary, leading to implicit, emergent properties like visual recognition.

A term for AI approaches from the 1960s that focused on formal logic, knowledge bases, and symbolic representation, which David Rumelhart recognized as not fully providing answers to questions about understanding and inference.

A model created by Rumelhart and McClelland that uses neuron-like processing units with connection weights to process pixel-level inputs, construct features, letters, and words, demonstrating interactive bi-directional processing.

The setting for Descartes' inspiration on hydraulic systems and mechanistic theory of animal behavior, which he saw as separate from human thought.

An AI research and deployment company, mentioned as a key player in scaling AI computations for advanced intelligence, particularly in areas like large language models.

An AI research company mentioned as a place where the massive scale of computation is being played out to achieve superhuman intelligence, particularly with successes in games like Go and protein folding.

Miniature, self-assembling 'brains' grown in vitro, used to study embryonic development and the assembly of mechanisms capable of learning.