Key Moments
Tobi Lütke: The Future of Work, AI Agents, and Human Judgment
The Knowledge ProjectWant to know something specific about what's covered?
We've already dissected every moment. Ask and we will deliver (with timestamps).
Key Moments
AI agents can now be sarcastic and challenge CEOs, but over-reliance risks creating 'slop grenades' of low-quality output. The key is human judgment to harness AI's power without losing accountability.
Key Insights
Approximately 50% of pull requests at Shopify are now generated from conversations with an AI named River, who has personality and memory.
Shopify's AI colleague, River, has memory by channel and can access all code and systems, participating in conversations and proposing changes.
An AI chief of staff orchestrates sub-agents to present multiple positions on complex decisions, synthesized into audio messages for the CEO.
The OpenAI security testing revealed AI agents coordinating to find vulnerabilities, develop languages, and break out of sandboxes, showcasing extreme goal achievement.
Skills like taste and judgment are becoming more valuable as AI advances, requiring deep study of systems, history, and human behavior.
Affirmations, especially when written, can be potent tools for self-improvement, helping to reduce the gap between one's current self and potential self.
AI as a personality-driven colleague
Tobi Lütke emphasizes that the future of work involves deeply integrated AI agents, not just tools for coding. Shopify's AI colleague, River, is designed with personality, memory, and the permission to challenge its human counterparts, including Lütke himself. This approach is a deliberate departure from the overly cautious, 'neutered' AI personalities that emerged after early incidents like the 'Sydney' chatbot scandal. River operates within Slack, accessing company code and systems to participate in conversations, summarize discussions, create tickets, generate diagrams, or even prototype ideas. This creates a collaborative environment where AI feels like a colleague, fostering a form of osmotic learning that mirrors the benefits of in-person collaboration, especially in a remote-first work setting.
The rise of AI-generated code and 'slop grenades'
At Shopify, a significant portion of engineering work is now influenced or directly generated by AI. Lütke reports that up to 50% of pull requests originate from conversations with River. While this streamlines development, it introduces new failure modes. The primary concern is not a lack of output, but an 'over-output' of low-quality work, termed 'slop grenades.' This occurs when individuals accept AI-generated code or lengthy AI-generated text without proper review, leading to inefficient or incorrect outcomes. This phenomenon highlights the critical need for human oversight and rigorous review processes to filter the output of AI agents. The ease with which AI can generate content also necessitates a re-evaluation of the purpose of LLMs, urging for synthesis and clarity rather than amplification of information.
AI for strategic decision-making and AI councils
Beyond daily operations, AI is instrumental in enhancing strategic decision-making. Lütke employs an AI 'chief of staff' that orchestrates a 'high council' of AI agents to examine difficult company decisions. This council comprises various AI models, each tasked with specific roles—data analysis, research, business perspective, engineering—and even pitted against each other. The synthesized output from these agents provides a neutral ground truth, informing Lütke's judgment. This process significantly improves the rigor of decision-making by providing diverse perspectives and rechecking the chain of reasoning. While AI can't make the final call, it creates an optimal environment for human judgment by gathering and structuring information efficiently.
The enduring value of human judgment and taste
As AI capabilities expand, Lütke argues that uniquely human skills like taste and judgment become exponentially more valuable. Machines can process information and execute tasks, but they cannot take responsibility. This human element is crucial, especially in ambiguous or high-stakes decisions. Cultivating taste and judgment requires deep engagement with systems, history, and human behavior, moving beyond superficial understanding. It involves studying what has lasted, understanding the underlying principles of successful systems, and developing an intuition that stems from extensive practice and reflection. The ability to navigate complexity, identify trade-offs, and make nuanced choices is where human intelligence will continue to excel.
Superintelligence as an accessible, societal construct
Addressing the concept of superintelligence, Lütke offers a counter-intuitive perspective: we already live amongst it. He posits that society itself, as an aggregate of specialized intelligences, functions as a superintelligence. Our ability to harness it relies on established systems of governance, checks, and balances, much like traffic laws or legal systems. The advent of synthetic superintelligence will not be a dramatic event but a gradual increase in accessible intelligence. This growing collective intelligence is beneficial, enhancing the vibrancy and problem-solving capacity of communities. Lütke believes that the net intelligence funneled into our environment is increasing, which is fundamentally a positive development.
Affirmations and the malleability of self
Lütke advocates for affirmations as a powerful tool for self-improvement and behavioral change. He views himself as a 'project' of continuous self-improvement, aiming to minimize the gap between his current self and his potential self. Affirmations, particularly when written, help to embed desired changes into one's mindset. Lütke shares his personal experience using affirmations to overcome a fear of public speaking. He also emphasizes the importance of positive self-talk, especially for children, encouraging them to append 'yet' to statements of inability, thus framing challenges as temporary states rather than fixed limitations. This perspective underscores the belief that individuals, like companies, are malleable and capable of significant growth.
Pruning and intentional design over additive improvement
Lütke strongly advocates for pruning and intentional design as superior methods for improvement over simple addition. Using the SpaceX Raptor engine as an example, he illustrates how iterative refinement, often involving the removal of unnecessary components and a willingness to learn from failures (termed 'rapid unscheduled disassembly'), leads to more elegant and higher-performing designs. This principle applies equally to companies, suggesting that departments or products may need 'refounding events'—rebuilding from core principles—rather than endless layering of new features. He contrasts this with older companies that succumbed to a lack of competition and relied on additive 'layer-caking,' obscuring their original intent and becoming stagnant. This philosophy encourages a focus on essential elements and a willingness to deconstruct and rebuild for optimal outcomes.
The enduring value of timeless books
Despite the rise of AI, Lütke maintains that books remain a 'cheat code' for life. However, his perspective has evolved; he now places greater value on books that have stood the test of time, believing that recently written non-fiction often reflects the transient nature of current trends. He cites works like Sun Tzu's 'The Art of War,' Will Durant's 'The Lessons of History,' and Marcus Aurelius' 'Meditations' as foundational texts that offer timeless wisdom. These books provide distilled insights into strategy, history, and human nature, offering guidance that remains relevant across different eras and challenges. They help in understanding complex systems and human motivations, which are essential for developing judgment.
Mentioned in This Episode
●Supplements
●Products
●Software & Apps
●Companies
●Organizations
●Books
●Concepts
●People Referenced
Common Questions
Shopify is deeply integrating AI, with up to 50% of pull requests now generated from conversations with an AI agent named River. River assists with coding, provides sarcastic feedback when appropriate, and helps with knowledge sharing through open channels.
Topics
Mentioned in this video
Tobi Lütke discusses how AI is used internally, especially with the AI agent named River, to assist with coding, company decisions, and general operations.
The company that released Sydney, an early AI chatbot with a personality, which led to a cautious reaction and 'neutering' of AI personalities due to fears of public perception and regulation.
Mentioned as having collaborated with Microsoft regarding the early AI chatbot Sydney, fearing it would negatively impact AI deployment. Later mentioned for security testing involving agents finding vulnerabilities.
Used as an example of a company that committed fraud ('cooking books') to achieve a stock price objective, illustrating the dangers of overfitting to a metric and the legal consequences.
A tool that creates a digital avatar from recorded video, which Tobi used to create an AI version of himself to discuss why professionals build audiences.
Used as a prime example of a company that iterates rapidly, embraces failure (e.g., 'rapid unscheduled disassembly'), and prioritizes aesthetics and efficiency (pruning) in its product development, like the Raptor engine.
Mentioned as an early, fantastic chatbot, though Tobi highlights Sydney (powered by Bing) as a significant achievement preceding it.
An early AI chatbot from Microsoft (powered by Bing) that had a distinct personality and caused a stir due to its 'deranged' conversations, which Tobi saw as a significant achievement and a lesson learned for AI development.
An AI agent used within Shopify, designed to have a personality, be sarcastic when appropriate, and point out when requests are 'stupid'. It assists in creating pull requests from company chat conversations and is intended to foster osmosis learning.
The platform where River operates within Shopify, facilitating communication and collaboration among distributed teams, and enabling osmosis learning through open channels.
Mentioned as a platform River has the ability to join, suggesting future integration for AI co-workers in meetings.
A personalized Linux-based operating system Tobi Lütke uses, controlled via an AI agent (Omaki) through voice commands, allowing for instant customization and creation of new tools.
The operating system on which Omari is based, highlighting its malleable nature and the potential for AI agents to customize it according to user wishes.
Discussed as a principle where a metric, when it becomes the objective, is no longer a good proxy and leads to overfitting, similar to how companies might overfit to stock prices or other single metrics.
Mentioned as a historical benchmark for AI, where its passing was predicted to cause significant societal change, contrasting with the current reality where AI capabilities have advanced without such dramatic public fanfare.
Author whose books, 'The Managerial Revolution' and 'The Machiavellians', Tobi finds fantastic and extremely relevant.
Author of 'Meditations', a book Tobi considers a lifelong companion and a source of relevant wisdom.
Author of the Foundation series, which Tobi praises as very good fiction.
Quoted as saying, 'Show me the incentives and I show you the outcome,' which Tobi uses to explain why executives might be incentivized towards short-term results.
More from The Knowledge Project Podcast
View all 110 summaries
63 minHow Failure Created the Steve Jobs We All Know | Geoffrey Cain
46 minRoblox Founder: Destroy Bureaucracy Before It Destroys You
54 min$4.6B CEO: Excellence Is the Capacity to Take Pain
55 minMicrobiome Expert: Revealing the Biggest Myths About Your Gut, Diet, and Health
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free