Markov Decision Processes
mathematical model for sequential decision making under uncertainty
Build a research pod on Markov Decision Processes.
6 expert discussions. Save them all to your own pod, ask any question, get cited answers.
Common Themes
Videos Mentioning Markov Decision Processes

Stanford AA274A Principles of Robotic Autonomy | Autumn 2019 | Decision Making Under Uncertainty
Stanford Online
The framework used for decision making under uncertainty discussed in the lecture, forming the basis for dynamic programming.

Leslie Kaelbling: Reinforcement Learning, Planning, and Robotics | Lex Fridman Podcast #15
Lex Fridman
A mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision-maker. It assumes the current state contains all necessary information about the future.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 1: Course Overview
Stanford Online
A formulation of problems in a stochastic setting, related to discrete-time optimal control.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 9: Stochastic Dyn. Program
Stanford Online
A mathematical framework used for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision maker. It's central to the lecture's discussion on optimal control with uncertainty.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 17: RL Value-Based Methods
Stanford Online
The mathematical formalism used for reinforcement learning, involving states, actions, rewards, and transitions.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 14: Intro to IL and RL
Stanford Online
A mathematical framework used in reinforcement learning to formally define sequential decision-making problems, consisting of states, actions, transition dynamics, reward functions, and a discount factor.