Key Moments
Michael Kearns: Algorithmic Fairness, Privacy & Ethics | Lex Fridman Podcast #50
Lex FridmanKey Moments
Michael Kearns on "Ethical Algorithms," fairness, privacy, and game theory.
Key Insights
Algorithmic fairness is a complex, multifaceted problem with inherent tensions between different fairness definitions.
Differential privacy offers a robust framework for data privacy, overcoming limitations of traditional anonymization techniques.
The "ethical algorithm" is not one that decides ethics itself, but one that can encode societal values agreed upon by stakeholders.
Group-based fairness definitions can be insufficient, and achieving individual fairness is a more challenging goal.
Algorithmic game theory provides tools to understand and design systems with interacting agents, relevant to platforms and markets.
The financial sector has seen a significant shift towards algorithmic trading, especially in execution and high-frequency trading.
THE CHALLENGE OF DEFINING AND IMPLEMENTING FAIRNESS
Michael Kearns discusses the complexities of algorithmic fairness, distinguishing it from privacy. Unlike privacy, for which differential privacy offers a relatively settled definition, algorithmic fairness is an area with inherent trade-offs. Kearns highlights theorems showing that desirable notions of fairness can be mutually exclusive, making it a significantly harder problem. The realization that algorithms inherently amplify existing societal biases, especially when trained on historical data, necessitates careful consideration. Furthermore, the definition of who or what constitutes 'harm' and which groups require protection is a crucial, non-technical decision that must precede algorithmic design.
GROUP VERSUS INDIVIDUAL FAIRNESS AND FAIRNESS GERRYMANDERING
Kearns elaborates on the spectrum between group and individual fairness. While group fairness metrics, like ensuring equal false rejection rates across racial groups, are a starting point, they can mask discrimination against intersectional subgroups. This phenomenon, termed 'fairness gerrymandering,' occurs when fairness is achieved at a broad group level but specific combinations of attributes lead to disparate outcomes. Achieving true individual fairness, where each person's unique attributes are considered, is a much more ambitious and complex algorithmic challenge that the field is still exploring, often requiring more refined subgroup analysis.
THE ROLE OF HUMAN VALUES AND THE PARETO FRONTIER
Kearns emphasizes that ethical algorithms do not self-determine ethical principles; rather, they are tools to encode human-defined values. The process of defining fairness requires societal agreement on who to protect and what constitutes harm, which can be politically charged and involve deeply unresolved debates like affirmative action. For understanding the trade-offs, Kearns introduces the concept of the Pareto curve, illustrating the numerical relationship between system error and unfairness. This curve serves as a crucial interface for stakeholders to make informed decisions about where to position a system on this spectrum, a decision that should not be made by computer scientists alone.
DIFFERENTIAL PRIVACY: A STRONGER FRAMEWORK FOR DATA PRIVACY
In contrast to fairness, algorithmic privacy has a more mature technical definition in differential privacy. This framework ensures that the outcome of an analysis is nearly identical whether an individual's data is included or excluded. The mechanism for achieving this is by carefully adding calibrated noise to computations. This approach overcomes the fundamental flaw of traditional anonymization techniques, which can be vulnerable to re-identification by combining datasets. Differential privacy allows for robust statistical and machine learning analyses while providing strong privacy guarantees for individuals.
ALGORITHMIC GAME THEORY AND COLLECTIVE OUTCOMES
Kearns discusses algorithmic game theory as a mathematical framework for analyzing systems of interacting individuals, particularly relevant for large-scale platforms and markets. Principles like Nash equilibrium help predict stability in systems where individuals act in their self-interest. The field explores how machine learning, especially 'no regret' learning, can drive these systems towards equilibria. Even when systems optimize for individual users (like navigation apps or social media feeds), the resulting 'competitive equilibrium' might not be the best for the collective, highlighting the need for careful design beyond simple engagement optimization.
THE EVOLUTION OF ALGORITHMIC TRADING AND FINANCIAL MARKETS
Reflecting on his work in algorithmic trading, Kearns notes a clear progression where computers first excel at tasks they are demonstrably better at than humans. This began with optimizing trade execution for large volumes and has extended to high-frequency trading, which exploits minute price discrepancies. While machines dominate these areas, Kearns posits that long-term, value-based investing, like that practiced by Warren Buffett, still requires human intuition, understanding of macroeconomic cycles, and broader contextual risks. Currently, machines are not well-equipped for the nuanced, multi-year predictions and risk assessments central to such investment strategies.
NAVIGATING THE ETHICAL LANDSCAPE IN A DIVISIVE CULTURE
Kearns acknowledges the difficulty of building ethical algorithms in a polarized society. He stresses that computer scientists' role is not to dictate societal values but to provide tools to implement agreed-upon principles. However, he also notes that technology has fundamentally altered society, sometimes in unforeseen negative ways. The challenge lies in balancing the economic models of platforms, which often optimize for engagement, with broader societal well-being. Solutions may involve user controls and experimenting with algorithms that expose users to diverse viewpoints, moving away from purely optimizing for user retention.
THE INTERPLAY OF DATA, PRIVACY, AND ECONOMIC MODELS
The discussion touches upon the economic underpinnings of the internet, where users' data is often the product, with advertisers as the customers. Kearns suggests that achieving greater individual control over data and privacy could necessitate a shift in these economic models, potentially leading to new market structures where individuals are compensated for their data. While this presents significant economic challenges, he remains optimistic about finding a better compromise between data utilization and robust privacy guarantees, likely requiring a combination of technological solutions and regulatory frameworks.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●Books
●Concepts
●People Referenced
Common Questions
'The Ethical Algorithm' is a book co-authored by Michael Kearns that explores algorithmic fairness, privacy, and ethics. It delves into how to encode societal norms into algorithms from a technical perspective, addressing issues like bias and data protection.
Topics
Mentioned in this video
His work in game theory established the existence of competitive equilibrium under general circumstances, providing a firm conceptual footing for the field.
A moral philosopher whose ideas on fairness are referenced when discussing algorithmic fairness definitions.
Professor at the University of Pennsylvania and co-author of 'The Ethical Algorithm,' a world-class researcher in several fields including machine learning, game theory, and quantitative finance.
Michael Kearns's co-author on 'The Ethical Algorithm' and a founder of the field of differential privacy.
The author of Michael Kearns's favorite novel, Infinite Jest.
Led an interesting project at NYU to build a browser plugin that obfuscates Google searches to protect user privacy.
Mentioned in the context of China's data collection and its potential to either protect human rights or violate them, depending on its application.
A professor with whom Michael Kearns studied at Harvard, describing the experience as wonderful.
A renowned investor whose long-term investment style is contrasted with high-frequency trading, suggesting that the human element remains crucial for such timescales.
A navigation app, similar to Google Maps, used to illustrate how algorithms can nudge users towards a competitive or Nash equilibrium in traffic.
A platform where many Netflix users publicly rate movies, which allowed for re-identification of individuals in the anonymized Netflix Prize dataset.
A navigation app used as an example of algorithms optimizing for individual self-interest (minimizing driving time), leading to a competitive equilibrium.
A tech giant whose search engine is used as an example to discuss privacy concerns and the obfuscation of user data.
An e-commerce platform mentioned alongside social media platforms as using algorithms to optimize for individual user preferences, driving towards competitive equilibrium.
Mentioned as an example of a company whose shares an investor might buy or sell in the context of institutional trading strategies.
A social media platform mentioned as an example where user 'likes' can predict sensitive personal attributes, illustrating challenges in data privacy.
Used as an example of a stock in which a large hedge fund might want to buy a significant stake over time, highlighting complex execution problems in algorithmic trading.
The institution where Michael Kearns is a professor.
A think tank where game theory was taken seriously in the 1960s as a tool for reasoning about US-Soviet nuclear armament and disarmament.
The Massachusetts Institute of Technology, mentioned as a location relevant to 'Infinite Jest' and where the interview takes place, implicitly.
New York University, where a project developed a browser plugin to obfuscate Google searches and enhance user privacy.
Where Michael Kearns attended as an undergraduate, remembering it as a large school with diverse academic opportunities.
Co-sponsored a workshop on machine learning for macroeconomic prediction, exploring the application of ML to longer financial timescales.
Where Michael Kearns pursued his graduate studies, finding the computer science department smaller and more specialized than his undergraduate experience.
A new book co-authored by Michael Kearns, which focuses on algorithmic fairness, bias, privacy, and ethics.
A novel Michael Kearns mentioned reading in high school, indicating his early interest in literature.
Michael Kearns's favorite novel, by David Foster Wallace, which significantly influenced him and is set near MIT.
A field that forms the theoretical foundation for much of Michael Kearns's work and the subject of his co-authored book.
A stable state in game theory where no player can benefit by unilaterally changing their strategy, given the strategies of others.
A field of study, particularly algorithmic game theory, that reveals structure in competitive and cooperative human interactions, on which Michael Kearns has done extensive work.
A strong and widely accepted definition of privacy that algorithms can be designed to achieve, which avoids the weaknesses of traditional anonymization methods.
A classic example in game theory illustrating how individuals acting in their self-interest can lead to a collectively worse outcome than if they cooperated.
The use of algorithms in financial markets to optimize execution problems and make directional predictions in asset prices, with varying success rates across different time scales.
More from Lex Fridman
View all 505 summaries
154 minRick Beato: Greatest Guitarists of All Time, History & Future of Music | Lex Fridman Podcast #492
23 minKhabib vs Lex: Training with Khabib | FULL EXCLUSIVE FOOTAGE
196 minOpenClaw: The Viral AI Agent that Broke the Internet - Peter Steinberger | Lex Fridman Podcast #491
266 minState of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490
Found this useful? Build your knowledge library
Get AI-powered summaries of any YouTube video, podcast, or article in seconds. Save them to your personal pods and access them anytime.
Try Summify free