How can I tell if people understand my product?

The simplest indicator is whether users bother to sign up. You can also measure the ratio of visitors who take any action after landing on your page, rather than immediately leaving. For B2B, clicks on the pricing page can also be a decent indicator of interest.

What is the shark fin effect and how can I avoid it?

The shark fin effect describes viral growth that leads to a rapid influx of users who then churn quickly because the product lacks retention. To avoid this, focus on building genuine user value and retention from the start, rather than just optimizing for acquisition.

Is Monthly Active Users (MAU) a good metric?

The speaker suggests that MAU is becoming a 'BS metric' similar to registered users, as it can be misleading. Daily Active Users (DAU) is considered a harsher but more meaningful metric for engagement and retention, showing a strong correlation with reduced churn.

How should early-stage startups with zero users measure success?

When you have very few users, direct conversation is crucial. Badger your handful of users for feedback. You can run simple A/B tests by asking them which version of a feature they prefer or by observing their behavior, as direct qualitative feedback is more valuable than early data points.

What is a North Star metric?

A North Star metric is a single number that represents the core value your product delivers and on which you're willing to bet the company. It should guide the team's focus, and you should commit to it for a period (e.g., six months) before re-evaluating.

How do you measure product-market fit?

Product-market fit can be measured both qualitatively (users spreading the word) and quantitatively. Key quantitative measures include retention rates (DAU, weekly/monthly retention) and how often users engage with the product, aiming for benchmarks above average for your industry.

Should I prioritize 'quality of life' features or user acquisition features?

Initially, prioritize features that differentiate your product and drive acquisition and retention. Features like 'forgot password' can wait unless they become a significant barrier to retention, indicated by increasing requests or repeated login failures, which then makes them critical for the user flywheel.

Key Moments

Suhail Doshi - How to Measure Your Product

Y Combinator

Science & Technology4 min read60 min video

Sep 12, 2018|148,473 views|2,686|61

YC Y Combinator Startup School Suhail Doshi Mixpanel

Save to Pod

Key Moments

TL;DR

Product measurement is key. Focus on understanding, ease of use, and retention to drive growth.

Key Insights

Prioritize three core areas: product understanding, ease of use on day one, and user retention.

Growth can be broken down into a formula: Visits -> Signups -> Valuable Actions -> Returning Users -> Spreading the Product.

Start simple with 1-3 core metrics (Northstar Metric) and avoid overcomplicating analytics.

The 'enemy' is the back button; ensure your product's value proposition is clear from the first interaction.

Ease of onboarding is critical; initially, focus on getting users to their 'aha!' moment, not on features like password recovery.

Retention is paramount. Viral growth without retention can lead to company failure (the 'shark fin effect').

FOUNDATIONAL PRINCIPLES OF PRODUCT MEASUREMENT

Suhail Doshi emphasizes a top-down approach to product analytics, focusing on the problems startups aim to solve rather than complex data analysis. He identifies three critical areas: ensuring users understand what the product does, making it easy to use from day one, and retaining users by providing ongoing value. This framework is crucial for building a product people truly want and will continue to use, forming the bedrock of sustainable growth.

THE GROWTH FORMULA AND THE DANGER OF OVERCOMPLICATION

Doshi presents a simple growth formula: Visits -> Signups -> Valuable Actions -> Returning Users -> Spreading the Product. He stresses that even large companies often overcomplicate their metrics, leading to analysis paralysis. Instead, startups should identify 1-3 'Northstar' metrics to guide their efforts, avoiding the temptation to track dozens of sub-metrics which can obscure clarity and hinder decision-making.

MAKING YOUR PRODUCT UNDERSTANDABLE AND ACCESSIBLE

The initial hurdle for any product is making its value proposition clear. Doshi highlights the 'back button' as the primary enemy, indicating user confusion or lack of perceived value. He shares Mixpanel's early struggles with a confusing tagline and broad targeting. The key takeaway is to have a clear, benefit-oriented message, often refined through iteration and customer feedback, focusing on what differentiates your product, like Mixpanel's shift to 'actions speak louder than pages'.

STREAMLINING THE GETTING STARTED EXPERIENCE

Ease of initial use is paramount. Doshi advises against optimizing for features like 'forgot password' if users aren't even getting to the core functionality. A simplified onboarding process, even if it means skipping less critical steps initially (like ZenDesk's decision not to have a save button upfront), can significantly improve conversion. The goal is to enable users to experience the product's core value quickly, as exemplified by Google's minimalist search or Airbnb's streamlined booking initiation.

THE CRITICAL IMPORTANCE OF USER RETENTION

Doshi profoundly emphasizes retention, warning against the 'shark fin effect' where viral growth can mask underlying issues, leading to eventual company demise. He explains how rapid user acquisition without corresponding retention means losing users faster than they are acquired. Measuring daily active users (DAU) is presented as a more robust metric than monthly active users (MAU), as it reflects a deeper level of engagement and correlates strongly with reduced churn.

MONITORING REVENUE CHURN AND GAUGING PRODUCT-MARKET FIT

For subscription-based or B2B companies, monthly revenue churn is a critical metric that can be deceptively damaging. Doshi illustrates how a seemingly small monthly churn rate can translate to significant annual revenue loss. Product-market fit, he suggests, is quantitatively measured by strong retention rates (e.g., DAU) and users who actively spread the word. He advises against premature complexity, advocating for focused effort on core metrics and user feedback.

STRATEGIES FOR EARLY-STAGE COMPANIES AND USER FEEDBACK

For startups with zero or very few users, direct customer interaction is indispensable. Doshi recounts his personal experience badgering early users for feedback and conducting 'crude' A/B tests to inform product decisions. Even without sophisticated tools, talking to customers provides depth of insight unmatched by raw data. He also highlights the importance of documenting this feedback for future team alignment and product development, stressing simplicity in analytical approaches.

ADVANCED CONSIDERATIONS AND COMMON PITFALLS

Doshi touches upon advanced topics like the 'next feature fallacy' and the tendency for founders to overcomplicate analytics, leading to confusion and distraction. He reiterates that focusing on a single Northstar metric for a defined period (e.g., six months) is more effective than overwhelming dashboards. Finally, he advises against optimizing for problems a startup doesn't yet have, like fraud, and encourages a deliberate approach to building quality-of-life features versus those that directly drive acquisition or retention.

Mentioned in This Episode

●Software & Apps

●Companies

●Organizations

●People Referenced

Product Measurement Cheat Sheet

Practical takeaways from this episode

Do This

Focus on 3-5 core metrics (North Star Metric)

Measure understanding: Do people sign up?

Measure ease of use: Can users get started quickly?

Measure retention: Are users coming back consistently?

Talk to your users, especially in the early stages

Iterate on your initial user experience continuously

Consider letting users access the product before full sign-up if possible

Be harsh with your metrics to accurately gauge product-market fit

Simplify analytics; avoid overcomplication and too many dashboards

Focus on building value for customers over minor conveniences initially

Avoid This

Don't overcomplicate metrics; avoid paralysis from too many numbers

Don't optimize for fraud or spam until you actually experience it

Don't rely solely on monthly active users (MAU); prioritize daily active users (DAU)

Don't ignore revenue churn; understand its long-term impact

Don't chase vanity metrics that don't reflect true product value

Don't focus on 'the next feature' as the sole solution to growth

Don't get distracted by new products or ideas before mastering the current one

Key Product Measurement Areas

Data extracted from this episode

Area	Key Questions	How to Measure
Product Understandability	Do people understand what you've built?	Sign-up rate, Clicks on key features, Pricing page clicks (B2B)
Ease of Getting Started	Is it easy to use the product on day one?	Funnel conversion rates (landing page to sign-up to first valuable action), Speed of initial task completion
User Retention	Are people coming back and using the product?	New user retention rate (e.g., 30-day), Daily Active Users (DAU), Monthly Churn Rate

Common Questions

Focus on a few core metrics that represent growth and user engagement. Key areas include understandability (sign-ups), ease of use (onboarding completion), and retention (users returning). Avoid overwhelming your team with too many metrics, and prioritize a North Star metric.

Topics

Technology & Innovation Business & Entrepreneurship Product Management Customer Feedback Product Analytics Growth Hacking Startup Metrics North Star Metric User Retention Onboarding Optimization

Mentioned in this video

Software & Apps

Google Slides

A presentation software, formerly known as Zen Tur, used as an example of a company that prioritized core functionality.

Twitter Bootstrap

A CSS framework mentioned as a tool for easily building user interfaces.

Google Analytics

A web analytics service offered by Google that tracks and reports website traffic, used as a point of comparison for Mixpanel's differentiation.

Slack

Mentioned regarding conversion rates on landing pages and retention benchmarks for chat applications.

Facebook Connect

A social login method discussed as an alternative to standard sign-up processes.

Craigslist

Used as an example of a website with a complex and seemingly chaotic interface that still functions effectively.

People

Suhail Doshi

Co-founder of Mixpanel.

Andrew Chen

Author of 'The Next Feature Fallacy,' an article recommended for advanced reading on product strategy.

Peter Thiel

Mentioned in relation to a question about conventional wisdom in startups.

Companies

Mentioned as an example of a company whose registered user count was deemed less important than active usage.

Mixpanel

A company that provides product analytics tools to help businesses measure user activity and product performance.

Airbnb

A company whose user onboarding and landing page strategies are discussed as examples of optimizing user experience and simplifying complex processes.

Zen Tur

A YC company that later became Google Slides, used as an example of prioritizing essential features over minor conveniences like a save button.

LivingSocial

A company that experienced viral growth but lacked retention, forcing a pivot to compete with Groupon.

Groupon

A competitor that LivingSocial had to pivot to compete with due to its own product's lack of retention.

Amazon

Used as an example of a site with a complex interface that is highly optimized.

YouTube

Mentioned as an example to illustrate how guiding principles and optimization goals shape a platform's interface.

Jawbone

A hardware company that used Mixpanel's services, indicating the applicability of these metrics to hardware with software components.