Key Moments
Stop planning for 2027: how OpenAI builds product 90 days at a time | Tara Sesha and Nan Yu (OpenAI)
Key Moments
OpenAI ships imperfect AI features rapidly, prioritizing user feedback over perfection, as the AI landscape shifts every 90 days.
Key Insights
OpenAI prioritizes shipping imperfect features to gather empirical user evidence rather than aiming for upfront perfection, a significant mindset shift.
Product development at OpenAI is guided by aiming for capabilities 2-3 months ahead of current model capacity, balancing present utility with future potential.
The debate between single-identity vs. multi-identity agents hinges on mapping to human nature for managing complexity, with bundles of activity being more intuitive than managing numerous individual agents.
Enterprises are experiencing a 'beyond breakneck' pace of AI change, yet shipping revolutionary changes like agents is crucial to prevent them from being leapfrogged by competitors.
The 'last mile' problem is critical; AI tools that complete 99% of a task but fail at the end can be more frustrating than tools that don't start at all.
Voice interfaces are predicted to be a major future development, offering a more natural and intuitive interaction method that reduces the need for complex tech support.
Embracing imperfection in AI product development
OpenAI's approach to product development has shifted significantly, moving away from the 'perfect before shipping' mindset ingrained from experiences at companies like Stripe. Tara Seshan emphasizes the urgency of getting functionality into users' hands, stating that empirical evidence from actual usage is invaluable and superior to theoretical planning. This iterative approach means shipping imperfect solutions, like the 'toggle' feature, to gather feedback and iterate quickly. The rationale behind this is to get powerful capabilities, such as agentic harnesses, into the hands of over a billion ChatGPT users without overly disrupting their existing workflows. This philosophy acknowledges that build-and-throw-away cycles are necessary, but stresses the importance of a coherent narrative to guide users through product evolution and maintain transparency, fostering acceptance of change.
Key constraints for building high-quality AI products
Several core principles guide OpenAI's product development, even at a rapid pace. Nan Yu highlights that a primary constraint is users' understanding and ability to absorb new capabilities, a concept referred to as 'capability overhang.' Tara Seshan adds that products must be additive, unlocking genuine value and delight for users, and ideally, offering novel use cases for the models. A crucial internal bar involves shipping features internally, testing them rigorously, and assessing uptake, delight, and new use cases unlocked. Most importantly, product development aims for a horizon of 2-3 months into the future, aligning with anticipated model capabilities without being overly anchored to the present or too futuristic to be usable. This zone aims for a 'sufficiently AGI' state, ensuring the product is valuable, user-centric, and enables the model to perform optimally.
Navigating the pace of change for enterprises
The rapid evolution of AI presents a unique challenge for enterprises, often perceived as slow to adopt change. However, OpenAI's experience shows that even B2B customers who struggle with the daily pace of change need revolutionary advancements pushed to them. The 'agent revolution' necessitated shipping agent capabilities to enterprises quickly to prevent them from considering alternative products. This principle of 'do what your users need, not what they say they need' is paramount. While enterprises might have their own pace, delaying fundamental shifts can lead to them being leapfrogged. OpenAI's strategy involves pushing these giant, process-breaking changes to unlock greater value, even if it disrupts existing workflows, underscoring the need for businesses to adapt rapidly to remain competitive in the AI era.
Designing AI agents: single vs. multi-identity
The design of AI agents presents a divergence: either centralizing on a single, generic agent identity capable of anything, or crafting multiple, specialized micro-agents. Tara Seshan suggests that successful agent design maps onto human nature, which struggles to manage an excessive number of distinct threads. She likens managing 40 agents to managing 40 direct employees, which is overwhelming for most. Therefore, grouping activities into bundles or hierarchical structures, such as a 'chief of staff' agent managing others, appears more natural and manageable. This aligns with how users naturally organize tasks, finding a single point of management more intuitive than juggling numerous independent agents. This approach emphasizes simplifying user interaction by mirroring human cognitive load limits.
Tactical considerations in agent design
Beyond the philosophical debate of single versus multi-identity agents, practical product considerations are critical. Nan Yu points out that questions around data access, permissions, and how an agent manifests to the user are paramount. For instance, with a single agent, how does it switch between using a service account versus the user's account? How does memory behave, especially in segmented contexts like private channels? Should it use universal or user-specific credentials? These tactical, use-case-specific edge cases heavily influence the design of agentic experiences. The interplay between user empathy and systems thinking is essential, but the ability to manifest these designs realistically requires deep consideration of platform components and potential edge cases to ensure a sensible and comprehensive delivery to the user.
The last mile problem in AI product completion
A critical aspect of AI product development is ensuring completion, particularly the 'last mile' of a task. Tara Seshan argues that getting 99% of a job done but failing at the very end is often worse than a non-starter because it creates a sense of unfulfilled promise. Tools like 'computer use' are praised because they reliably complete the task, even if they are slow or token-intensive. This ensures that the user's goal is met, providing a fallback when ecosystem tools or plugins are unavailable or don't interface well. The ideal user experience, drawing inspiration from Charles Eames, is for the user to feel like a welcomed guest in a home where all their needs have been anticipated and gently provided for. This focus on full task completion, especially the final steps, is crucial for user satisfaction and trust in AI products.
The evolving role of product managers and user empathy
While classic product management skills like user empathy and systems thinking remain fundamental, the AI era demands new emphasis and a relentless drive for iteration. Tara Seshan stresses the importance of learning from feedback loops at an accelerated clock speed, metaphorically 'throwing a lot of spaghetti against the walls.' Collaboration with research is also highlighted as a new, critical skill. Product managers need to bring specific use cases, understand user goals, analyze session data, and learn to write effective evaluations to drive the iteration loop. Nan Yu notes that for B2B and developer tools, engineers are highly detail-oriented, often requiring multiple follow-up questions to elicit useful feedback. For AI agents, where issues can be subtle, a direct, deeper relationship with users to understand context and nuances is becoming table stakes for product leaders, akin to the 'DMable PM' concept.
Future of AI interaction: Voice and self-driving experiences
Looking ahead, voice interfaces are predicted to be a major transformative element in AI. Tara Seshan is highly optimistic about voice, describing it as a more natural, intuitive, and less error-prone way to interact with computers, significantly reducing the need for complex instructions and tech support. Nan Yu anticipates a future where AI experiences resemble 'self-driving' systems, addressing the common 'empty input box' problem. These intelligent products will guide users, offering a gentle on-ramp and using their own capabilities to assist users in completing tasks, making the interaction feel inherently natural and pre-emptive. This move towards proactive assistance and intuitive interfaces like voice signals a significant shift in how users will engage with AI in the coming years, making AI products more accessible and user-friendly.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●People Referenced
OpenAI Product Development Principles
Practical takeaways from this episode
Do This
Avoid This
Common Questions
OpenAI prioritizes shipping imperfect products quickly to gather user feedback and iterate. They focus on getting functionality into users' hands and learning from real-world usage, rather than achieving theoretical perfection upfront.
Topics
Mentioned in this video
Mentioned as an example of someone who might manage a large number of direct employees, drawing a parallel to managing many agents.
Mentioned in relation to the 'yappers' API and the impact of voice technology.
Mentioned as an expert who might be able to predict OpenAI's business dynamics, contrasted with the speaker's own difficulty in doing so.
Quoted regarding the ideal user experience being like a guest in one's home, anticipating needs gently.
A conversational AI model from OpenAI, mentioned as a platform with over a billion users, to which agentic capabilities were introduced.
Mentioned as an example of a feature or output that a user might dislike, prompting feedback to product leaders.
Mentioned as a tool for business users and as a potential hardware prediction for the future.
Mentioned as an example of multi-identity agents, contrasted with a single, centralized agent identity.
A platform or feature mentioned for trying out GPT, where voice interaction is highlighted as a positive experience.
More from Lenny's Podcast
View all 56 summaries
29 minWhy Claude can’t be your PM (yet) | Anthropic CPO Panel
22 minRoles aren't converging—they're expanding | Tamar Yehoshua (Atlassian CPO)
95 minMolly Graham: The grief, burnout, and opportunity hiding inside the AI transition
23 minMarty Cagan: Strong Opinions, loosely held
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free