What is Longtermism and why is it important?

Longtermism is the philosophical view that ensuring the long-term future of humanity goes well is a key priority. It emphasizes the immense scale of potential future generations and the profound, long-lasting impact our actions today can have on them, for better or worse.

What are existential risks according to William MacAskill?

Existential risks are events that could lead to human extinction or permanent, drastic loss of future value. Examples include next-generation bioweapons causing a collapse of civilization, unaligned artificial intelligence disempowering or eliminating humanity, or a perpetual totalitarian dystopia locking in fascist values.

How does Effective Altruism handle the 'Pascal's Mugging' problem with expected value?

Effective Altruism generally views 'Pascal's Mugging' as a reductio ad absurdum rather than an implication to be endorsed. The standard response in decision theory to avoid absurd conclusions from tiny probabilities of infinite value is to use a bounded value function, suggesting there's an upper bound to how much good or well-being can be achieved.

Why is it psychologically difficult to care about future generations?

Human psychology evolved for an ancestral environment where our impact was limited to dozens of people, not billions across the globe or future generations. We mistakenly equate distance in time with unreality, making it hard to vividly imagine or feel a strong moral connection to hypothetical future people, even though their existence often depends on avoiding present catastrophes.

How does the future give meaning to the present?

The prospect of a long-term human future provides meaning to our present projects and aspirations. We plant trees for future generations, build enduring structures, and strive for moral, political, and scientific progress, seeing ourselves as part of a grand relay race. The knowledge of a foreclosed future would devastatingly sap meaning from our lives.

Is humanity currently at a 'hinge moment' in history?

Yes, it is argued that we are living in an unusually impactful time because technological progress is accelerating at an unprecedented rate. This rapid advancement creates new powers (like nuclear energy, biotechnology, AI) that carry both immense benefits and the potential for exceptionally long-lasting impacts, including the power to destroy civilization or lock in ideologies.

What is 'value lock-in' and why is it a concern for the future?

Value lock-in is the risk that a particular, narrow ideology or moral view could become globally dominant and persist for an extremely long time, potentially leading to a perpetual dystopia. This could happen through technological and political developments, such as extended lifespans for dictators or immortal digital rulers, undermining the natural drivers of moral change.

What is the 'lock-in paradox' regarding moral progress?

The lock-in paradox recognizes that while we don't want to permanently lock in our current, imperfect values, creating a world conducive to sustained moral reflection and progress might require some form of 'locking in' foundational principles. These principles would include commitments to open debate, tolerance of opposing views, and restrictions on power gained through means other than sound arguments.

Why should we worry about Artificial General Intelligence (AGI) if it's far off?

The exact timeline for AGI is uncertain, but assuming continued progress in intelligent machines and the substrate independence of intelligence, it's virtually inevitable. The concern isn't just about a distant future; AI experts estimate a significant chance of human-level AI within decades and a non-trivial chance of extremely bad outcomes (like extinction or disempowerment) within our lifetime. This urgency necessitates immediate focus on AI safety and governance, akin to how we address car crash risks.

How quickly is machine learning progressing, and what does it imply for AI timelines?

Machine learning is progressing much faster than even recent forecasts predicted. Examples include language models capable of solving college-level math proofs with 50% accuracy or producing philosophy essays indistinguishable from bottom-tier undergraduates. This rapid pace suggests that human-level AI might arrive sooner than many expect, potentially within 20-37 years, and highlights the urgency of addressing related risks, especially if growth is extremely fast.

Will AI passing the Turing Test be comprehensive AGI, and what are the implications?

AI passing the Turing Test does not necessarily mean comprehensive AGI, but it marks a significant milestone. Even short of full AGI, the experience of having conversational AI indistinguishable from humans (like an oracle-like Siri) will be transformative. It highlights the distinction between intelligence and consciousness, and the need to prioritize developing beneficial, 'oracle AI' that aids in alignment, rather than autonomous agents.

Key Moments

How Much Does the Future Matter?: A Conversation with William MacAskill (Episode #292)

Sam Harris

Science & Technology4 min read121 min video

Aug 15, 2022|88,757 views|1,292|2,000

sam harris sam harris podcast waking up podcast waking up waking up with sam harris sam harris waking up waking up sam harris sam harris jordan peterson sam harris joe rogan author neuroscientist philosopher

Save to Pod

Key Moments

TL;DR

William MacAskill discusses effective altruism, longtermism, existential risks, and the importance of safeguarding the future.

Key Insights

Effective altruism (EA) is a philosophy and community focused on maximizing good done with resources, emphasizing evidence and impact.

Longtermism posits that making the long-term future of humanity go well is a key ethical priority, considering the vast potential of future generations.

Existential risks (x-risks) are threats that could cause extinction or civilizational collapse, with a growing consensus that these risks are significant.

Expected value reasoning, while useful, faces challenges like the 'Pascal's mugging' problem, suggesting a need for bounded value functions or alternative decision theories.

Psychological misalignment exists between our evolved moral sentiments and the scale of modern challenges, making it hard to care about distant or future people.

The future provides meaning to the present, and the current historical moment is seen as a 'hinge moment' due to accelerating technological growth and its dual-use nature.

Value lock-in is a serious concern, where a narrow or negative ideology could become permanently dominant, hindering future moral progress.

Artificial intelligence (AI) development presents significant existential risks, particularly concerning misaligned AI, and requires proactive safety and governance measures.

Though predicting technological timelines is difficult, a rapid pace of progress in AI is plausible, necessitating urgent attention to safety alongside development.

Political and economic systems often incentivize short-term thinking, highlighting the need for institutional reforms to better consider long-term interests.

DEFINING EFFECTIVE ALTRUISM, LONGTERMISM, AND EXISTENTIAL RISKS

William MacAskill defines Effective Altruism (EA) as a philosophy and community dedicated to maximizing positive impact with available resources, driven by a rational and evidence-based approach to doing good. Longtermism, a core tenet of much EA work, emphasizes prioritizing the long-term future of humanity, recognizing the potentially immense scale of future lives and the significant impact present actions can have on their well-being. Existential risks (x-risks) are identified as threats that could lead to human extinction or civilizational collapse, with recent assessments suggesting these risks are more probable than commonly perceived, akin to significant personal risks.

ADDRESSING CRITICISMS AND THE CHALLENGES OF EXPECTED VALUE REASONING

MacAskill addresses criticisms, including the idea that market-based economic growth is the sole or primary driver of good, arguing that market failures like externalities necessitate intervention. He also tackles the 'Pascal's mugging' problem, where improbable scenarios with massive expected value can lead to absurd conclusions. This suggests that simple expected value calculations might not fully capture our moral intuitions, perhaps due to bounded value functions or the need for more nuanced decision theories, implying that our ethical frameworks need refinement when dealing with extremely large or small probabilities.

PSYCHOLOGICAL MISALIGNMENT AND THE DIFFICULTY OF CARING FOR THE FUTURE

A significant challenge lies in our evolved psychology, which is not well-suited to caring about distant or future people. Our moral sentiments evolved for small, immediate communities, making abstract concepts like future generations feel less real and urgent. This 'psychological misalignment' means that even well-intentioned individuals may find it easier to feel empathy for immediate suffering than for the potential well-being or suffering of billions yet to be born, highlighting a disconnect between our moral intuitions and the reality of global and temporal scale.

THE FUTURE'S MEANING AND OUR HINGE MOMENT IN HISTORY

The future is presented not just as a recipient of our actions but as a source of meaning for our present endeavors. Projects like building cathedrals or pursuing scientific progress gain significance from their potential to contribute to an ongoing human story. MacAskill argues that we are currently at a critical 'hinge moment' in history. Accelerating technological development, a hallmark of the modern era, brings both immense opportunities and profound risks, suggesting that decisions made now could have an outsized and lasting impact on the entirety of humanity's future.

VALUE LOCK-IN AND THE NEED FOR CONTINUED MORAL PROGRESS

A major concern is 'value lock-in,' where a single, potentially flawed ideology or social system could become permanently entrenched, halting future moral and societal progress. This could arise from powerful technologies like advanced AI, enabling a totalitarian regime to maintain control indefinitely. To prevent this, MacAskill suggests that locking in certain foundational principles, like commitment to debate, tolerance, and restrictions on unchecked power, might be necessary to ensure a space for continued reflection, empathy, and moral advancement, even if it means accepting some limitations on immediate ideological freedom.

THE IMPLICATIONS AND RISKS OF ADVANCED ARTIFICIAL INTELLIGENCE

The development of artificial intelligence (AI) is highlighted as a central existential risk. The potential for AI to surpass human intelligence raises concerns about alignment – ensuring AI goals match human values. MacAskill stresses that the precise timeline for advanced AI is uncertain, but the possibility of rapid progress, combined with the substrate independence of intelligence, makes proactive safety measures crucial. He likens the situation to receiving a future warning, urging a serious, integrated approach to AI safety, akin to how bridge engineers prioritize structural integrity, rather than treating it as an afterthought.

POLITICAL SHORT-TERMISM AND THE OPPORTUNITY FOR CHANGE

The inherent short-termism of current political and economic systems poses a significant obstacle to addressing long-term issues. Politicians are incentivized by immediate electoral cycles, making it difficult to prioritize the interests of future generations who cannot vote or contribute financially today. MacAskill explores potential institutional reforms, such as future-generation ombudspersons or citizen assemblies, but emphasizes that a fundamental cultural shift towards valuing the long-term is needed to align political incentives with the well-being of future populations and create conditions for sustained positive change.

Mentioned in This Episode

●Software & Apps

●Organizations

●Books

●Concepts

●People Referenced

Common Questions

Effective Altruism is a philosophy and community focused on using evidence and reason to find the most impactful ways to do good. Unlike traditional charity which might focus on salient causes, EA encourages a step back to consider all global problems and prioritize those where one's efforts can have the biggest impact.

Topics

Mindset & Self-Improvement AI & Machine Learning Society & Philosophy Moral Philosophy Ai Safety Human Progress Political Incentives Technological Growth

Mentioned in this video

Organizations

FBI

The US federal agency that raided Donald Trump's home, noted to be run by a Trump appointee, Christopher Wray.

Center for Effective Altruism

Co-founded by William MacAskill, this organization has raised over a billion dollars for charities, focusing on doing as much good as possible.

People

William MacAskill

Associate Professor in Philosophy at the University of Oxford, TED speaker, co-founder of the Center for Effective Altruism, and author of 'What We Owe The Future.'

Nina Bajakajian

Author of the Time Magazine piece on the Effective Altruism movement.

Stuart Russell

AI researcher whose analogy about machines knowing when they will arrive is used to illustrate the urgency of AI safety, and who advocates for integrating safety into AI development.

Donald Trump

Former US President whose Mar-a-Lago home was raided by the FBI for misappropriated documents, leading to concerns about political stature and partisanship.

Christopher Wray

The Trump-appointed head of the FBI, mentioned in the context of allegations of anti-Trump partisanship.

Paul Bloom

Friend mentioned by Sam Harris as someone who blurbed William MacAskill's book.

Elon Musk

Mentioned as an example of someone who, through business, might do a different species of good than environmental activism by building electric cars.

Toby Ord

William MacAskill's colleague and author of 'The Precipice,' who suggests technology is advancing faster than human wisdom.

Sam Harris

Host of the Making Sense podcast and author, who interviews William MacAskill and shares his own ethical framework.

Ayatollah Ruhollah Khomeini

Iranian religious leader who pronounced a fatwa against Salman Rushdie.

Nick Bostrom

Philosopher who sketched the implications of 'expectational total utilitarianism' and the 'fanaticism problem' (Pascal's Mugging).

Samuel Scheffler

Philosopher and author of 'Death in the Afterlife,' whose work discusses the impact on meaning if humanity's future were foreclosed.

Salman Rushdie

Author attacked on stage, whose plight highlights the issue of religious fanaticism and the cowardice of secularists to defend free speech.

Sam Bankman-Fried

Businessman whose politics were referenced in a Wall Street Journal article criticizing effective altruism.

University of Oxford

Academic institution where William MacAskill is an Associate Professor in Philosophy.

Ray Kurzweil

Inventor and futurist known for his claims about accelerating technological change.

John Stuart Mill

19th-century philosopher who argued that posterity gives meaning to present projects.

Tyler John

Philosopher with whom William MacAskill co-authored an article on long-termist political institutions.

Concepts

Pascal's Mugging

A thought experiment illustrating how small probabilities of extremely large gains can lead to counter-intuitive or absurd decisions if expected value theory is applied without bounds.

Existential Risk

Risks that could cause human extinction or permanent, drastic collapse of civilization, such as advanced bioweapons, unaligned AI, or totalitarian dystopias.

Mohism

An ancient Chinese philosophy with intellectual roots resembling consequentialism, mentioned by MacAskill as a historical antecedent to effective altruism.

Forbes 30 Under 30

An award William MacAskill received as a social entrepreneur.

Longtermism

A philosophical perspective emphasizing the importance of shaping the long-term future of humanity, recognizing the vast potential value of future generations.

Effective Altruism

A philosophy and community focused on using evidence and reason to determine the most effective ways to improve the world.

Pascal's Wager

A philosophical argument for believing in God based on expected value, mentioned in the context of the 'fanaticism problem'.

Turing Test

A test of a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human, expected to fall for chatbots within a decade.

Software & Apps

GPT-3

A language model that William MacAskill used to mark undergraduate philosophy exams, demonstrating its surprising capability.

Siri

Apple's virtual assistant, used as an example of current AI that could become indistinguishable from humans in conversation.

Alexa

Amazon's virtual assistant, used as an example of current AI that could become indistinguishable from humans in conversation.

Media

Wall Street Journal

A newspaper that published an article critical of effective altruism, which Sam Harris found to lack substance.

Time

Magazine that featured William MacAskill and the Effective Altruism movement on its cover.

Charlie Hebdo

French satirical magazine that suffered a massacre, mentioned as another example of atrocities related to free speech and religious extremism.

Books

What We Owe the Future

William MacAskill's book, which is the main topic of discussion for this podcast, focused on longtermism and existential risk.

The Precipice

A book by Toby Ord which posits that technology is advancing faster than human wisdom, leading to increased existential risk.

Legislation & Policy

United States Constitution

Mentioned as an aspirational document aiming to create a liberal society with diverse worldviews and moral progress, with restraints on individual power.

Companies

Twitter

Social media platform where Sam Harris discussed the attack on Salman Rushdie and was subsequently criticized for his phrasing.