Why is reasoning considered the biggest blocker for AI agents?

Reasoning is crucial for agents to handle complex, long-term goals, similar to how humans iteratively write essays or plan trips. Today's models are not optimized for this, and there's a lack of explicit reasoning data on the internet.

What is Imbue's approach to improving AI reasoning?

Imbue focuses on optimizing their foundation models for reasoning through specific pre-training data and by generating data that explicitly captures reasoning traces. They also leverage code, which is seen as highly structured and explicit reasoning data.

Why did Imbue build its own simulation environment, Avalon?

Imbue built Avalon because existing environments like Minecraft were too slow for the extensive research needed to understand how agents can handle diverse tasks and learn without heavily handcrafted rewards. Avalon is 100 times faster than Minecraft.

What are the main limitations of current AI agents?

Current agents often lack reliability, have 'leaky' abstractions, and struggle with the remaining 20% of complexity needed for robust performance, similar to early challenges in self-driving cars. Their reasoning and planning capabilities are also limited.

How does Imbue plan to help others deploy AI agents?

Imbue is developing tools and abstractions that they predict will be helpful for other agent builders, potentially forming an 'operating system' for agents. They aim to create a platform that allows for rapid and reliable agent deployment, moving beyond the current 'bare metal' phase of agent building.

What is the role of code in Imbue's AI research?

Code is viewed as the most explicit form of reasoning data available online. Imbue uses code to help understand agent limitations, improve reasoning capabilities, and enable models to act on computers, ultimately serving as a natural language programming interface.

Why is chat limited as an interface for AI agents?

Chat interfaces are sequential and don't easily allow for inspecting or modifying an agent's state mid-execution. Imbue believes more robust interfaces are needed that enable users to collaborate with agents by modifying plans, forking processes, and debugging.

What are the biggest challenges for AI agents to achieve long-running tasks?

Memory and the ability to permanently learn new abstractions and processes are key challenges. While RAG is useful for preferences, it's not sufficient for complex scenarios like an AI scientist accumulating nuanced knowledge, suggesting a need for breakthroughs in agent memory.

What is 'scenius' and why is it important for innovation?

'Scenius' refers to the 'scene' or environment that fosters genius-level ideas. Imbue believes that fostering such environments, like Xerox PARC or Bell Labs, is crucial for generating groundbreaking innovations by bringing exceptional people together.

What are the key takeaways about the future of AI?

AI is a powerful tool acting as a better computer, offering 'free intellectual energy' to solve problems. While risks exist, Imbue focuses on engineering safety, theoretical understanding, and targeted regulation to ensure AI is protective and not harmful.

Key Moments

Why AI Agents Don't Work (yet) - with Kanjun Qiu of Imbue

Latent Space Podcast

Science & Technology5 min read73 min video

Oct 21, 2023|2,927 views|87|11

Save to Pod

Key Moments

TL;DR

AI agents face challenges in reasoning and reliability, with a focus on developing inspectable models and better interfaces.

Key Insights

Current AI agents are not yet fully functional due to limitations in reasoning and reliability, hindering their widespread deployment.

Imbue is focusing on developing foundation models optimized for reasoning, as this is seen as the primary blocker for advanced agents.

The company emphasizes the importance of inspectability and user control in agent design, moving away from purely black-box systems.

Code is considered a critical form of reasoning data, acting as a curriculum for models to learn explicit structures and logic.

Reliability and robustness in agents are key challenges; Imbue is building tools and abstractions to address these 'leaky' aspects.

The future of personal computing may involve agents acting as natural language programming interfaces, with Imbue aiming to build the foundational 'operating system' for these agents.

PERSONAL JOURNEY AND THE FOUNDING OF IMBUE

Kanjun Qiu's background includes a BS and MS from MIT, time at the MIT Media Lab focusing on electronic textiles and STEM education for young women, and experience in bizops at Dropbox. This led to a passion for human agency and effectiveness, which translated into founding multiple companies with her co-founder Josh. Their journey through a VR headset company (Ember) and an AI recruiting platform (Sorceress) provided crucial insights. Sorceress, while functional, faced market challenges similar to the dating industry due to high churn, teaching them about the need for user trust in AI systems and informing their current approach at Imbue.

THE SHIFT TO BUILDING CAPABLE AI AGENTS

The transition to Imbue (formerly Generally Intelligent) was driven by observations in late 2019 and early 2020 about the accelerating progress in self-supervised learning, particularly in language and vision models. Recognizing that scale was starting to yield powerful results, Qiu and her co-founder envisioned a future where computers could handle complex intellectual tasks, freeing humans for more creative pursuits, much like the Industrial Revolution freed people from physical labor. This vision led them from exploring startup ideas leveraging self-supervised learning to establishing Imbue as a research lab focused on building capable AI agents that could perform larger goals autonomously.

ADDRESSING THE REASONING BLOCKER IN AI AGENTS

Imbue's core mission is to develop foundation models optimized for reasoning, which they identify as the biggest obstacle to creating AI agents capable of handling complex, long-term goals. Unlike simple task completion, writing an essay or booking a flight often involves iterative refinement, research, and decision-making under uncertainty. Current models are not inherently optimized for this, as explicit reasoning data is scarce online. Imbue addresses this by focusing on specific data generation and pre-training strategies to bolster reasoning capabilities, aiming to create agents that can plan, adapt, and exhibit judgment.

THE ROLE OF DATA, CODE, AND INTERFACES

The emphasis at Imbue is shifting from solely relying on compute to prioritizing high-quality data, with generated data showing promise comparable to human-labeled data for specific tasks like reasoning traces. Code is viewed as a highly explicit and structured form of reasoning data, serving as a crucial curriculum for models. Furthermore, Imbue recognizes that for AI agents to be usable, they need to be inspectable and controllable. They are developing interfaces that allow users to collaborate with agents, modify their plans, and understand their decision-making processes, rejecting the notion of entirely black-box, end-to-end AI systems.

BUILDING RELIABLE AGENTS VERSUS EXTENSIVE HACKS

A significant challenge in AI agent development is moving beyond a 'pile of hacks' (like RAG, context stuffing, prompt engineering) to achieve genuine reliability and user trust. While these techniques extend current model capabilities substantially, they often result in 'leaky abstractions' where the agent's behavior is unpredictable. Imbue believes that deep improvements in underlying models, particularly in reasoning, combined with better interfaces and abstractions, are necessary to create agents that users can depend on for critical tasks, similar to how software engineering evolved from low-level programming to high-level abstractions.

THE VISION FOR THE NEXT GENERATION OF COMPUTING

Imbue aims to 'rekindle the dream of the personal computer' by enabling agents that can act as a natural language programming interface. They envision a future where computers are highly malleable, allowing users to direct them using natural language, much like a modern IDE for natural language programming. This involves creating robust agentic systems and the necessary tooling and abstractions, potentially forming an 'operating system for agents.' The company emphasizes a philosophy of 'dogfooding' – using their own agents daily – to drive rigorous development and ensure the agents are truly reliable and useful.

COMMUNITY, CULTURE, AND THE IMPORTANCE OF 'SCENIUS'

Qiu highlights Imbue's unique culture, focusing on social process design and viewing team members as 'creative agents' rather than assets. This philosophy extends to fostering a safe and inspiring environment that encourages risk-taking and mutual growth. Her experience with co-living hacker houses like 'Archive' underscored the power of 'scenius' – genius derived from one's environment – where a concentrated group of exceptional individuals can collectively achieve more. This principle informs Imbue's team-building and community-oriented approach, aiming to create an environment where breakthroughs are more likely.

NAVIGATING THE FUTURE OF AI DEVELOPMENT AND REGULATION

The current era in AI is compared to historic technological inflection points like the dawn of the digital computer or the early days of personal computing. Imbue believes we are witnessing a rapid wave of innovation, with agents becoming increasingly sophisticated. While memory and long-running tasks remain critical challenges, breakthroughs in areas like retrieval augmentation are incrementally improving capabilities. Imbue is also actively engaged in AI safety and policy, developing tools to analyze regulatory proposals and advocating for thoughtful deployment, recognizing that new technologies bring both immense potential and inherent risks.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●Concepts

●People Referenced

Compute vs. Data Labeling Costs for AI Models

Data extracted from this episode

Model	Estimated Compute Cost	Estimated Data Labeling Cost
LLaMA 2	$3-4 million	$20-25 million

Common Questions

Imbue's mission is to rekindle the dream of the personal computer by developing AI agents that can perform larger tasks and offer greater leverage to human users, essentially creating a more powerful computing platform.