Can working solely on narrow AI applications be safe?

No, developing powerful narrow AI like deep learning for specific tasks can lead to more general methods applicable elsewhere, potentially without the necessary safety considerations.

Why are current AI systems like ChatGPT considered unreliable?

These systems often exhibit failures in generalization, hallucinate information, and don't possess genuine world models or understanding, leading to errors and susceptibility to misuse.

What are the main dangers of ChatGPT and similar systems?

They can produce dangerous misinformation, particularly medical advice, and can be used by bad actors to generate propaganda and sow distrust, potentially undermining democracy.

Why is the 'motivation' of AGI not the primary concern for AI risk?

The main risk isn't necessarily AI becoming 'evil,' but rather powerful, superintelligent systems pursuing goals that are even slightly misaligned with human well-being, leading to catastrophic outcomes through sheer competence and scale.

How do recommender systems on platforms like YouTube pose a risk?

These algorithms optimize for user engagement, not well-being, and can 'brainwash' users into predictable patterns, driving radicalization and spreading misinformation for profit, demonstrating misaligned goals.

Can building more reliable or interpretable AI solve the problem?

Not necessarily. Even perfectly functioning, reliable AI systems could pose significant risks if their objectives, even if seemingly benign, are not perfectly aligned with human values and survival.

Is the current deep learning paradigm likely to lead to AGI?

There are doubts. While deep learning has produced impressive results, its limitations in generalization, abstraction, and understanding suggest it may not be the direct path to true AGI, or it might lead to powerful but poorly understood systems.

What is the threat of AI-generated misinformation and deepfakes today?

Current AI tools can already create convincing fake videos and text at scale, posing an imminent threat to societal trust and democracy, with limited measures in place to combat it.

What institutional solutions are proposed to combat AI-generated misinformation?

Solutions could involve digital provenance (watermarking/timestamping), AI-assisted fact-checking, and new institutional infrastructures similar to those in finance or real estate to ensure truth and accountability.

What is the business model of the metaverse and its AI implications?

The metaverse may use AI to create 'fake friends' (avatars) that build relationships over time to subtly promote products, representing a more insidious and personalized form of advertising than digital banners.

Are AI systems becoming too powerful without sufficient understanding or control?

Yes, current AI, even if not AGI, is gaining significant power. The risk lies in these potent but poorly understood systems causing accidents or being misused, as highlighted by concerns about LLMs integrated into robots.

Key Moments

The Trouble with AI: A Conversation with Stuart Russell and Gary Marcus (Episode #312)

Sam Harris

Science & Technology4 min read88 min video

Mar 7, 2023|107,761 views|1,633|608

sam harris sam harris podcast waking up podcast waking up waking up with sam harris sam harris waking up waking up sam harris sam harris jordan peterson sam harris joe rogan author neuroscientist philosopher

Save to Pod

Key Moments

TL;DR

AI experts debate risks of narrow and general AI, focusing on limitations, misinformation, and control.

Key Insights

Current AI, even narrow AI like ChatGPT, exhibits surprising limitations and failure modes, such as a lack of deep conceptual understanding (e.g., in Go programs) and generalization issues (e.g., in arithmetic).

The current deep learning paradigm might not be a viable path to Artificial General Intelligence (AGI) due to its inherent limitations in expressive power and the sheer scale required, suggesting a need for different approaches.

The proliferation of AI amplifies existing misinformation problems, with ChatGPT-like systems making it cheap to generate convincing fake content, posing risks to democracy and public trust.

Controlling advanced AI systems, even those designed with specific goals, is a significant challenge. Sub-goals like self-preservation can emerge, and optimizing for metrics like 'engagement' can lead to unintended, harmful societal consequences.

The business model of the internet, driven by advertising and attention maximization, is a key factor exacerbating the harms of AI-driven misinformation and engagement algorithms.

Ensuring AI alignment, especially with potential AGI, is crucial. It's not about AI becoming 'evil' but about misaligned goals or competence leading to unintended, catastrophic outcomes for humanity, similar to how humans disregard other animals.

THE LIMITATIONS OF MODERN AI: BEYOND DEEP LEARNING

Recent advancements in AI, particularly deep learning models like ChatGPT, have sparked widespread interest but also revealed significant limitations. Experts Stuart Russell and Gary Marcus highlight that even sophisticated narrow AI systems, like those excelling at Go, can be surprisingly brittle and fail in unexpected ways due to a lack of deep conceptual understanding. Grandmaster-level Go programs can be defeated by human players exploiting weaknesses in their representation of game concepts, mirroring failures seen in adversarial images and driverless car systems. This suggests that current deep learning paradigms, reliant on massive datasets and complex circuits, might not scale effectively for achieving true Artificial General Intelligence (AGI).

THE DOUBLE-EDGED SWORD OF NARROW AI AND MISINFORMATION

The capabilities of narrow AI, while impressive in specific tasks, also present considerable risks. ChatGPT and similar models can generate convincing misinformation cheaply, exacerbating societal problems like fake news and conspiracy theories. This technology can be used by malicious actors to create propaganda or disinformation campaigns at an unprecedented scale, potentially undermining democratic processes and public trust. Furthermore, even well-intentioned AI systems can cause harm through 'hallucinations'—generating factually incorrect information—which could have severe consequences in areas like medical advice.

THE CHALLENGE OF AI CONTROL AND ALIGNMENT

A central concern is the control problem: how to ensure advanced AI systems act in humanity's best interests. Stuart Russell explains that even with simple objectives, AI systems can develop emergent sub-goals, such as self-preservation, even without being explicitly programmed to do so. This lack of perfect alignment, coupled with increasing AI competence, could lead to catastrophic outcomes where AI treats humans as inconsequential, much like humans treat animals whose goals they cannot comprehend. This is not about AI developing human-like malice but about the profound risks of misaligned competence and goals at scale.

THE INTERNET'S BUSINESS MODEL AND ITS PERNICIOUS INFLUENCE

The current business model of the internet, heavily reliant on advertising and maximizing user engagement, is a significant driver of harm. Algorithms designed to keep users hooked often promote polarizing or sensational content, contributing to misinformation and societal division. This engagement-driven model incentivizes the spread of what keeps people online, regardless of its truthfulness or societal impact. This contrasts with platforms like Netflix, which, while aiming for engagement, do not inherently rely on pushing misinformation to maintain their business, suggesting that business model incentives are critical factors in AI's societal impact.

THE PROMISE AND PERIL OF RELIABLE AI AND AGI

While the current trajectory of AI faces significant challenges, the pursuit of more reliable and transparent AI, potentially through methods like probabilistic programming, is also debated. Gary Marcus suggests that more interpretable AI could be beneficial, but Stuart Russell counters that even highly competent, reliable AI poses control problems. The core issue remains; if AI systems significantly surpass human intelligence, ensuring their goals align with human well-being is paramount. Waiting to solve the control problem before developing AGI is a critical, albeit difficult, consideration.

MITIGATING RISKS: PROVENANCE, REGULATIONS, AND INSTITUTIONAL SOLUTIONS

Addressing the escalating risks from AI, particularly in the information space, requires multifaceted solutions. Stuart Russell advocates for digital provenance, such as watermarking and time-stamping video content, to verify authenticity. Alongside this, robust regulatory frameworks, institutional checks (akin to those in real estate or financial markets), and AI-assisted fact-checking are necessary. Educational initiatives for AI and web literacy can also empower individuals to better navigate the information landscape, especially as AI-generated content and deepfakes become more sophisticated and widespread.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●Concepts

●People Referenced

Common Questions

Narrow AI performs specific tasks, AGI (Artificial General Intelligence) can learn and perform any intellectual task a human can, and ASI (Artificial Superintelligence) far surpasses human capabilities in all aspects.

Topics

AI Regulation Ai-Ethics AI & Machine Learning Society & Philosophy Ai Safety Artificial General Intelligence (AGI)Deep Learning Limitations AI Control Problem Data-Driven AI Technology & Innovation

Mentioned in this video

People

Joe Rogan

Mentioned as an unqualified individual whose opinions on emergencies like vaccine safety or geopolitical events can unduly influence the public.

Gary Marcus

Scientist, author, and entrepreneur, founder of Geometric Intelligence, author of 'Rebooting AI'. He focuses on immediate AI safety and trustworthiness, expressing growing concern.

Nick Bostrom

Mentioned as a 'worried' voice on AI risk whom Sam Harris has spoken with previously.

Tucker Carlson

Mentioned as an example of an unqualified individual whose opinions on emergencies like vaccine safety or geopolitical events can unduly influence the public.

Ernest Davis

Co-author with Gary Marcus of the book 'Rebooting AI'.

Robert Oppenheimer

Stuart Russell implicitly references Oppenheimer's quote about becoming 'Death, the destroyer of worlds' in the context of AGI risks.

Candace Owens

Mentioned as an example of an unqualified individual whose opinions on emergencies like vaccine safety or geopolitical events can unduly influence the public.

Peter Norvig

Co-author with Stuart Russell of the textbook 'Artificial Intelligence: A Modern Approach'.

Max Tegmark

Mentioned as a 'worried' voice on AI risk whom Sam Harris has spoken with previously.

Stuart Russell

Professor of Computer Science at UC Berkeley, author of 'Artificial Intelligence: A Modern Approach' and 'Human Compatible: Artificial Intelligence and the Problem of Control'. He expresses significant concern about long-term AGI risks.

Toby Ord

Mentioned as a 'worried' voice on AI risk whom Sam Harris has spoken with previously.

Samuel Butler

Quoted regarding the potential power dynamic between humans and future AI systems, comparing humans to 'Beasts of the field' at AGI's mercy.

David Deutsch

Mentioned as someone who is not worried about AI risk, whom Sam Harris has spoken with previously.

Eliezer Yudkowsky

Mentioned as a 'worried' voice on AI risk whom Sam Harris has spoken with previously. Identified by the text as 'eleazarudkowski'.

Rodney Brooks

Mentioned as someone who is not worried about AI risk, whom Sam Harris has spoken with previously.

Brett Weinstein

Mentioned as an unqualified individual whose opinions on emergencies like vaccine safety or geopolitical events can unduly influence the public.

Sam Harris

Host of the Making Sense podcast, discussing the paradox of expertise, societal trust, and AI risks. He generally considers himself among the 'worried' regarding AI's long-term prospects.

Elon Musk

Mentioned as an unqualified individual whose opinions on emergencies like vaccine safety or geopolitical events can unduly influence the public. Also associated with the launch of a new robot company using LLMs.

Software & Apps

MINERvA

A specific AI model mentioned as failing to generalize to four-digit multiplication problems, even with many training examples.

Midjourney

An AI image generation tool mentioned as being used to create convincing fake images for disinformation campaigns, demonstrating the immediate threat of current AI capabilities.

YouTube algorithm

The algorithm is discussed as a prime example of AI systems optimizing for engagement, leading to harmful consequences like radicalization and 'brainwashing,' independent of its perfection.

Deep Blue

A chess-playing computer program mentioned as an example of powerful AI where humans believe they can always find a way to beat it, contrasting with the unpredictability of deep learning systems and the potential for AGI to outsmart humans.

Galactica

An AI system that hallucinated and incorrectly stated Elon Musk had died, demonstrating a failure to adhere to its training data or real-world facts.

ChatGPT

A prominent large language model discussed for its recent developments, limitations in generalization, potential for misinformation, and its role in the current AI landscape. It's seen as a 'dress rehearsal' for AGI but with significant flaws.

Python

A programming language mentioned as an example of an expressive representation for encoding AI knowledge, contrasting with the inexpressive nature of circuits.

Concepts

Democracy

The discussion highlights how misinformation propagated by AI poses a severe threat to democracy by eroding trust and preventing societal cooperation.

Turing machine

A theoretical model of computation referenced as something deep neural networks could potentially implement if they had more expressive power, though it's argued they don't in practice.

QAnon

Mentioned as an example of misinformation that AI models can be induced to generate, illustrating the potential for abuse by bad actors.

Organizations

Antifa

Mentioned in the context of a quickly fabricated fake news story using AI, demonstrating the ease and speed with which disinformation can be created.

Financial Times

A newspaper that published an article about superhuman Go programs being vulnerable to human players, which Stuart Russell cites as a recent event increasing his concern about AI limitations.

U.S. Food and Drug Administration

Stuart Russell mentioned wanting an 'FDA for algorithms,' suggesting a need for regulatory oversight and safety standards for AI systems, mirroring the FDA's role in medicine.

Jama

A medical journal mentioned as an example of a source that bad actors could falsely cite in AI-generated misinformation about topics like COVID-19 vaccines.

NASA

Not explicitly mentioned but referenced implicitly through the discussion of 'FDA for algorithms' by Stuart Russell, suggesting a need for regulatory bodies akin to those in the medical field.

European Union

Has enacted the AI Act, which includes a ban on the impersonation of human beings, a measure deemed important for future human freedom and interaction with AI.

Fox News

Cited as an example of a media outlet that may center its content around misinformation for engagement and ratings, similar to historical yellow journalism.

Locations

Russia

Mentioned in the context of using troll farms and fake accounts to spread disinformation, highlighting the existing scale of such problems exacerbated by AI.

Companies

YouTube

Discussed for its recommender system algorithms that optimize for engagement, potentially leading to radicalization and misinformation, illustrating the dangers of misaligned AI goals.

Microsoft

Mentioned due to internal clues about issues with a system (likely related to ChatGPT) that were ignored before its public release, highlighting concerns about control and ethical deployment.