Key Moments
Why PMs are suddenly valuable again | Tulsee Doshi (Head of Product, Gemini)
Lenny's PodcastKey Moments
Google's Head of Product for AI, Tulsee Doshi, reveals that the company's 60-20-20 framework encourages building features that are only 60% functional now, implying a need for products that can evolve with AI's rapid progress.
Key Insights
Google's frontier AI strategy spans from hardware to end-user products, serving billions of users across applications like Search, engineering, and cybersecurity.
The 60-20-20 framework suggests building products where 60% works now, 20% is nearly working, and 20% is experimental, acknowledging that AI models are not fully predictable.
AI safety is addressed through monitoring, ensuring model alignment with directives, and leveraging AI to enhance cybersecurity systems, as seen with Gemini 4's partnership with security experts.
Product managers in frontier AI labs are sought for their 'flair' for models, daily interaction, and personal use cases to assess model capabilities.
To combat AI-generated content overload, Doshi suggests being intentional about where AI assists in content creation (e.g., data analysis) versus human insight (e.g., strategic vision).
Tulsee Doshi's personal philosophy emphasizes actively seeking 'serendipity' by stepping outside comfort zones and embracing new experiences to foster luck and innovation.
The unpredictable nature of AI model development
Tulsee Doshi, Head of Product for Google's frontier AI models like Gemini, likens AI development to "Snape's potions class," where adding ingredients doesn't guarantee a predictable outcome. While researchers set goals for models, the final capabilities can be surprising and unique. Product managers must not only shape the input but also identify and amplify emergent strengths. This unpredictability means that the path to advancing AI is not purely scientific but also requires an element of intuition and adaptation. While there's ongoing research to make aspects predictable, understanding the 'character' and 'flair' of a model as it emerges is crucial for product development. This involves observing metrics, understanding how capabilities manifest in final products, and assessing the model's perceived personality. This iterative learning process allows for continuous improvement and discovery of new capabilities.
Google's full-stack AI strategy
Doshi outlines Google's comprehensive strategy in the AI race, emphasizing a full-stack approach that covers hardware, foundational models, and end-user products. With 14 products already serving over a billion users, Google aims to build leading models across the cost-quality Pareto curve. This means developing models that serve diverse use cases, from quick information retrieval for search users to complex tasks for cybersecurity professionals. The goal is not just to have the most advanced model but to create a broad range of experiences that improve users' lives. This approach allows Google to innovate across different levels, leveraging its vast ecosystem to deploy AI effectively and meet the needs of billions.
The 60-20-20 framework for product development
Challenging traditional startup wisdom of building a Minimum Viable Product (MVP), Doshi introduces the 60-20-20 framework for AI product development. This approach advocates for building features where 60% works well today, 20% is 'almost working' or functional enough, and 20% is experimental or aspirational. The rationale is that AI models are constantly evolving, so building features that are only partially functional now prepares products for future advancements. By incorporating elements that are not yet perfect but show promise, teams can create products that are functional today and can deliver a 'magical experience' as models improve. This forward-looking strategy helps products grow alongside AI capabilities, rather than becoming obsolete.
Balancing innovation with user experience
The rapid pace of AI development presents a challenge in balancing the release of cutting-edge features with maintaining a user-friendly experience. Doshi highlights the need for new interaction models and user experiences for AI. For example, products like Genie and Veo (now Omni) are being designed with specific use cases like video and music editing in mind, exploring diverse form factors. However, for mass-market products like AI in Search, accessibility for billions of users is paramount, requiring a tailored user experience. Conversely, professional tools may need to support complex commands and integrations. Ultimately, the ideal user experience depends on the target audience, ranging from broadly accessible interfaces to highly technical environments.
The rising value of product managers in AI
Doshi observes a resurgence in the value of product managers (PMs), especially in frontier AI. As building becomes easier, the skill of identifying *what* is worth building, *why* it's worth solving, and *how* to unify teams behind a vision becomes critical. Good PMs can discern the 'essence' from the 'noise,' turning AI's potential into valuable user experiences. In fast-paced environments like Google, PMs are essential for structuring chaos and bringing clarity. Doshi specifically seeks PMs who understand the 'flair' of AI models, engage with them daily, and develop personal use cases to rigorously test their capabilities. Technical aptitude, the ability to dive into data, and a flexible, adaptive mindset are crucial given the field's rapid evolution. Resilience and the capacity to constantly re-evaluate roadmaps are key.
AI safety and Google's approach
Doshi emphasizes the critical role of AI safety, drawing on her six years leading Google's Responsible AI product team. She highlights the importance of monitoring systems that can track a model's 'chain of thought' to detect and address misalignments. This provides confidence in deployment and helps refine models. Furthermore, AI can actively enhance safety by identifying vulnerabilities in global systems. Gemini 4, for instance, partners with cybersecurity experts to find and fix security flaws, as demonstrated by a case where AI helped secure a hospital system. Doshi believes optimism for AI safety stems from continuous, cautious, and intentional thought processes, acknowledging the need for diligence and questioning in model development.
Developing and evaluating AI models
The process of creating and evaluating AI models is complex, involving both scientific rigor and an understanding of emergent properties. Doshi stresses the importance of building intuition for models, suggesting PMs test them daily with personal use cases. She advises starting AI evaluations with a small set of prompts (around 20) to test various capabilities and understand differences between models. Beyond structured evaluations, developing 'impressions' of a model's improvement involves testing them in areas of personal interest, whether it's creative writing, visual aesthetics, or complex analysis. Doshi also shares her experience improving skin tone representation by collaborating with Professor Monk, moving from a six-point Fitzpatrick scale to a more nuanced ten-point scale, enhancing accuracy in products like Pixel cameras and image generation.
The role of AI in enhancing productivity and preventing burnout
Doshi acknowledges the feeling of 'doing more' with AI, but also the risk of burnout. She believes burnout stems from chaos and unenjoyable work, not just long hours. AI tools, if implemented well, can reduce drudgery and free up time for strategic thinking and creativity. The key is to focus on tasks that are meaningful and engaging. Doshi highlights the importance of building trust in AI tools and fostering a culture where team members share skills and knowledge. This includes dedicating time for experimentation, such as internal workshops. She also touches on 'AI waste,' advocating for intentional use of AI, leveraging it for data analysis and reporting while reserving human insight for strategy and creative work. The goal is to use AI to augment, not overwhelm, human capabilities.
Mentioned in This Episode
●Software & Apps
●Companies
●Books
●Concepts
●People Referenced
AI Product Development and Management Guide
Practical takeaways from this episode
Do This
Avoid This
Common Questions
Google takes a holistic approach, covering everything from hardware to end-user products, aiming to build models across the Pareto curve for cost-vs-quality. They focus on creating a diverse range of experiences and ensuring their models serve various user needs, from search to cybersecurity.
Topics
Mentioned in this video
A generative media model from Google focused on music generation.
A Google service that collects user data, leveraged by AI assistants.
A frontier model from Google that the guest works on, mentioned as an example of Google's advanced AI capabilities.
Mentioned as a successful company using WorkOS.
A Google service that collects user data, often used by AI assistants.
An AI assistant that reportedly leverages user data, mentioned in the context of Google's lack of a comparable product.
A generative media model from Google focused on video and image editing/generation.
An AI product mentioned as potentially aiming to leverage user data, in the context of Google's offerings.
A feature within Gipboard on Pixel phones that is highly recommended for mobile typing.
Google's set of open models, part of the generative media model suite.
An example of a different form factor for interacting with AI models.
An AI assistant that reportedly leverages user data, mentioned in the context of Google's lack of a comparable product.
A Google tool that some users find difficult to locate, indicating a challenge in product discoverability.
Mentioned as an example of a product becoming increasingly complex, similar to Codex ChatGPT.
An AI assistant mentioned as being used by the speaker to automate tasks and build workflows.
The podcast's newsletter, offering a free year of WhisperFlow to subscribers.
A keyboard feature on Pixel phones that offers excellent mobile typing, including a feature called 'Rampler'.
An AI product that the guest finds transformative for writing due to wrist issues, functioning as a voice-to-text tool.
Mentioned as a successful company using WorkOS.
Mentioned as an example of AI product complexity, with its interface becoming overwhelming with tabs, dropdowns, and points.
Mentioned as a successful company using WorkOS.
A Google service that collects user data, which is often leveraged by AI assistants.
A Google tool or service that some users find difficult to locate.
A product being built by Google under the name Omni, related to generative media capabilities.
An AI assistant that reportedly leverages user data, mentioned in the context of Google's lack of a comparable product.
A sponsor of the podcast season, providing enterprise-ready APIs for B2B software integration, including features for AI agents.
Mentioned as a successful company using WorkOS.
The company where the guest, Tulsee Doshi, works as Head of Product for Google's frontier models.
An AI assistant that reportedly leverages user data, mentioned in the context of Google's lack of a comparable product.
A cybersecurity company, now a subsidiary of Google, partnering with Gemini 4 to identify and address security vulnerabilities.
Mentioned as a leading AI lab and competitor to Google, and its researcher was previously a guest on the podcast discussing model interpretability.
Mentioned as a leading AI lab and competitor to Google.
The company featured in 'Creativity, Inc.', used as an example for building a culture that encourages innovation and resilience.
Mentioned as a company with a strong distribution advantage, similar to Google, allowing for rapid progress.
A Google product designed for video and music editing and creation.
A previous podcast guest who was asked a similar question about AI assistants leveraging Google data.
Author of 'Exhalation', a book of sci-fi short stories recommended by the guest.
Mentioned for his advice to periodically discard all your tools (skills) to see what happens, as models may have evolved to make them redundant.
A romantic fantasy series mentioned by the guest as a recent interest for light reading.
Mentioned for a recent discussion about AI becoming proficient in cybersecurity and the resulting investment in cyber defense.
A TV series on Apple TV+ praised for its masterful style and content, and is the guest's favorite recent watch.
A romantic fantasy book series recommended by the guest.
A book about Pixar's culture, recommended for its insights on building a supportive and innovative work environment, relevant to fostering creativity and resilience.
A collection of sci-fi short stories by Ted Chiang, specifically a story about digital avatars kept alive after a platform shutdown, resonating with themes of human-AI interaction and consciousness.
A romantic fantasy book series recommended by the guest.
More from Lenny's Podcast
View all 62 summaries
88 min86 minutes of paid-UGC alpha | Hector Serrano (Wispr Flow)
86 minAn 86-minute guide to finding PMF | Rob Snyder (Author, The Power of Pull)
73 minA mind-expanding conversation on how models actually work | Emmanuel Ameisen (Anthropic)
63 minWhy every company now needs to think and operate like a lab team | Josh Woodward (VP Google Labs)
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free