Markov Decision Processes

Concept

mathematical model for sequential decision making under uncertainty

Mentioned in 6 videos

Build a research pod on Markov Decision Processes.

6 expert discussions. Save them all to your own pod, ask any question, get cited answers.

Get Started Free

Videos Mentioning Markov Decision Processes

Stanford AA274A Principles of Robotic Autonomy | Autumn 2019 | Decision Making Under Uncertainty

Stanford AA274A Principles of Robotic Autonomy | Autumn 2019 | Decision Making Under Uncertainty

Stanford Online

The framework used for decision making under uncertainty discussed in the lecture, forming the basis for dynamic programming.

Leslie Kaelbling: Reinforcement Learning, Planning, and Robotics | Lex Fridman Podcast #15

Leslie Kaelbling: Reinforcement Learning, Planning, and Robotics | Lex Fridman Podcast #15

Lex Fridman

A mathematical framework for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision-maker. It assumes the current state contains all necessary information about the future.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 1: Course Overview

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 1: Course Overview

Stanford Online

A formulation of problems in a stochastic setting, related to discrete-time optimal control.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 9: Stochastic Dyn. Program

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 9: Stochastic Dyn. Program

Stanford Online

A mathematical framework used for modeling decision-making in situations where outcomes are partly random and partly under the control of a decision maker. It's central to the lecture's discussion on optimal control with uncertainty.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 17: RL Value-Based Methods

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 17: RL Value-Based Methods

Stanford Online

The mathematical formalism used for reinforcement learning, involving states, actions, rewards, and transitions.

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 14: Intro to IL and RL

Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 14: Intro to IL and RL

Stanford Online

A mathematical framework used in reinforcement learning to formally define sequential decision-making problems, consisting of states, actions, transition dynamics, reward functions, and a discount factor.