Goodhart's Law
adage that when a measure becomes a target, it ceases to be a good measure
Build a research pod on Goodhart's Law.
7 expert discussions. Save them all to your own pod, ask any question, get cited answers.
Common Themes
Videos Mentioning Goodhart's Law
Making Transformers Sing - with Mikey Shulman of Suno
Latent Space
A principle discussed by Mikey Shulman, stating that when a measure becomes a target, it ceases to be a good measure. Applied to LLM benchmarking and the importance of aesthetics over purely quantitative metrics.
Anca Dragan: Human-Robot Interaction and Reward Engineering | Lex Fridman Podcast #81
Lex Fridman
An observation that when a measure becomes a target, it ceases to be a good measure, applicable to reward function design in AI.
Alison Gopnik on Childhood Learning, AI as a Cultural Technology, and Rethinking Nature vs. Nurture
Conversations with Tyler
An observation stating that when a measure becomes a target, it ceases to be a good measure. Applied to education, where teaching to the test can undermine actual learning.
AI Labs Warn Race Makes Breach ‘Inevitable’
AI Explained
Mentioned in the context of AI safety metrics, suggesting that as metrics become targets, they cease to be good measures.
Why AI Agents Don't Actually Understand You — Danielle Perszyk, Amazon AGI Lab
Latent Space
An economic principle stating that when a measure becomes a target, it ceases to be a good measure, relevant to AI optimization.
Tobi Lütke: The Future of Work, AI Agents, and Human Judgment
The Knowledge Project Podcast
Discussed as a principle where a metric, when it becomes the objective, is no longer a good proxy and leads to overfitting, similar to how companies might overfit to stock prices or other single metrics.
🔬 Google's AI Scientist Started as an Attempt to Automate Kaggle — John Platt, Google Fellow
Latent Space
The principle that any metric that becomes a target is no longer a good metric, applied to the challenges of overfitting and reward hacking in AI and leaderboards.