Leslie Kaelbling: Reinforcement Learning, Planning, and Robotics | Lex Fridman Podcast #15

After completing degrees in philosophy and computer science, Kaelbling took a job at SRI's AI lab where she was tasked with getting a new robot to perform tasks, which led to her direct involvement and interest in robotics.

Kaelbling suggests philosophy, particularly fields like logic, model theory, and formal semantics, provides excellent preparation for AI and computer science. She notes that concepts like belief and knowledge are formal and closely related to computer science work.

The major roadblock for expert systems was the assumption that humans could effectively articulate their knowledge into logical statements. Experts often lacked introspective access to their own reasoning processes and provided post-hoc explanations that weren't always accurate or comprehensive.

Abstractions are critical because it's impossible to reason at a completely fine-grained level about everything. Reducing the size of the state space and the reasoning horizon, through spatial, temporal, or goal-based abstractions, makes complex problems like planning for a PhD or making dinner tractable.

An MDP assumes complete knowledge of the current state, allowing perfect prediction of the future based on system dynamics. A POMDP acknowledges that the state is not fully observable, requiring agents to maintain a belief about the state based on past actions and observations to make decisions under uncertainty.

Kaelbling explains that optimal planning for complex problems, especially POMDPs in continuous spaces, can be computationally undecidable or intractable. She advocates for making reasoned approximations in both modeling and solution algorithms to achieve computable results.

Belief space refers to a control problem where the agent reasons not just about controlling the state of the world, but about controlling its own beliefs about the world. This involves making actions that gather information and update the agent's understanding, which is crucial for problems requiring deliberate information gathering.

Hierarchical planning involves creating abstractions in the state space and temporal dimensions, allowing high-level plans to be made without detailing every step. This enables robots to reason about dependencies and constraints among abstract sub-goals, with the expectation that details can be figured out as each sub-goal is approached.

Kaelbling is not overly concerned with consciousness, stating she would be content with robots that are behaviorally indistinguishable from humans, even if not conscious ('philosophical zombies'). However, she acknowledges that some form of system monitoring or 'self-awareness' for internal state observation is likely critical.

JMLR was founded to address issues with traditional academic publishing, offering open-access and no page charges. It demonstrated that prestige and high-quality research output are achievable without the financial barriers of conventional journals, leveraging the competence of computer scientists in managing infrastructure.

Kaelbling's primary concerns revolve around the 'utility problem' for autonomous systems, advising careful consideration of objective functions and value alignment. She emphasizes the need to communicate that AI systems optimize within a given class of solutions, not necessarily achieving human intent without explicit framing.