Goodhart's Law
adage that when a measure becomes a target, it ceases to be a good measure
Build a research pod on Goodhart's Law.
6 expert discussions. Save them all to your own pod, ask any question, get cited answers.
Common Themes
Videos Mentioning Goodhart's Law
Making Transformers Sing - with Mikey Shulman of Suno
Latent Space
A principle discussed by Mikey Shulman, stating that when a measure becomes a target, it ceases to be a good measure. Applied to LLM benchmarking and the importance of aesthetics over purely quantitative metrics.
Anca Dragan: Human-Robot Interaction and Reward Engineering | Lex Fridman Podcast #81
Lex Fridman
An observation that when a measure becomes a target, it ceases to be a good measure, applicable to reward function design in AI.
Alison Gopnik on Childhood Learning, AI as a Cultural Technology, and Rethinking Nature vs. Nurture
Conversations with Tyler
An observation stating that when a measure becomes a target, it ceases to be a good measure. Applied to education, where teaching to the test can undermine actual learning.
Why AI Agents Don't Actually Understand You — Danielle Perszyk, Amazon AGI Lab
Latent Space
An economic principle stating that when a measure becomes a target, it ceases to be a good measure, relevant to AI optimization.
Tobi Lütke: The Future of Work, AI Agents, and Human Judgment
The Knowledge Project Podcast
Discussed as a principle where a metric, when it becomes the objective, is no longer a good proxy and leads to overfitting, similar to how companies might overfit to stock prices or other single metrics.
🔬 Google's AI Scientist Started as an Attempt to Automate Kaggle — John Platt, Google Fellow
Latent Space
The principle that any metric that becomes a target is no longer a good metric, applied to the challenges of overfitting and reward hacking in AI and leaderboards.