Key Moments
Things That Don't Scale, The Software Edition – Dalton Caldwell and Michael Seibel
Key Moments
Early-stage software companies use "hacks" that don't scale to validate products and gain users.
Key Insights
The "90/10 solution" (90% benefit for 10% work) is a core principle for early-stage software development.
Gmail's initial development involved "dirty hacks" like manual server fixes and an invite-only system due to limited resources.
Facebook scaled by creating separate infrastructure (databases, code instances) for each university, a strategy that didn't scale globally.
Twitch employed "static page" hacks and delayed stream propagation to manage massive traffic spikes from popular streamers.
Imeem repurposed video-playing technology for audio files, demonstrating creative problem-solving under resource constraints.
Google's MapReduce was developed out of necessity when the original indexing system failed under increasing web scale.
THE POWER OF THE 90/10 SOLUTION
The core concept discussed is the "90/10 solution," coined by Paul Buchheit, which advocates for achieving 90% of the benefit with only 10% of the work. This principle is vital for early-stage software companies that often face constraints in time and resources. Founders are encouraged to prioritize quick solutions that deliver core value, even if they are not perfectly engineered or scalable from the outset. This pragmatic approach allows for rapid product testing and validation in the real market.
GMAIL: HACKS FOR AN EMAIL REVOLUTION
Paul Buchheit's creation of Gmail exemplifies doing things that don't scale. Initially, he integrated his email into a Google Groups UI, manually building features as he needed them. When a server issue arose, he physically fixed it with a screwdriver, highlighting the hands-on, non-scalable approach. The invite-only system, often perceived as a growth hack, was actually a necessity due to insufficient server space, demonstrating that even seemingly strategic decisions can stem from resource limitations.
FACEBOOK'S UNIVERSITY-SPECIFIC ARCHITECTURE
In its early days, Facebook scaled by implementing a highly unscalable architecture: maintaining separate infrastructure for each university. This involved distinct PHP instances, MySQL databases, and Memcache servers for every school. The URL structure, like 'harvard.thefacebook.com,' reflected this localized setup. This method avoided the immediate challenge of scaling a single global database, allowing rapid expansion across campuses by simply copying and pasting code and provisioning new, often inexpensive, servers.
TWITCH: MANAGING UNPREDICTABLE PEAK TRAFFIC
Twitch faced the challenge of handling traffic spikes up to 20 times the average for live video, far exceeding typical website fluctuations. To manage this, they implemented a 'static page' hack. This feature allowed them to temporarily disable dynamic elements like usernames and real-time view counts to reduce server load during peak events, ensuring the core video stream remained accessible. Another hack involved pre-propagating video streams to multiple servers before a popular streamer went live, masking initial loading delays.
FRIENDSTER VS. MYSPACE: SCALABILITY DIFFERENCES
The contrast between Friendster and MySpace illustrates different scaling approaches. Friendster attempted to calculate complex social network connections in real-time, leading to database and server thread issues. In contrast, MySpace offered a simpler solution: denoting users as 'in your friends list' or 'in your extended network,' a feature that required minimal computation. This simpler approach allowed MySpace to scale more effectively by avoiding the performance bottlenecks Friendster encountered with its more ambitious, less scalable feature.
IMEEM'S AUDIO-AS-VIDEO WORKAROUND
Imeem creatively bypassed limitations by repurposing video-playing technology for audio content. They used open-source video tools and hacked them to play music files, essentially treating every audio upload as a video file with zero video data. This enabled them to leverage existing browser-based video playback innovations without developing entirely new systems specifically for audio streaming, demonstrating a resourceful solution under duress.
GOOGLE'S MAPREDUCE ORIGINS
The creation of Google's MapReduce framework, a foundational technology for distributed computing, stemmed from a critical failure in their original web indexing system. As the internet grew exponentially, the batch process for re-indexing the web began to fail, leading to months of stale search results. This crisis necessitated the development of MapReduce to parallelize and manage the immense task of crawling and indexing, illustrating how core scalable technologies can emerge from periods of significant system stress.
THE PHILOSOPHY OF TURNING ON THE WATER
The overarching advice for startups is akin to 'turning on the water' in a house with potentially leaky pipes. Instead of exhaustively testing every component beforehand, the practical approach is to launch, observe what breaks, and then fix it. This iterative process of releasing, identifying problems, and fixing them is fundamental to building large, successful companies. Early-stage efforts focus on proving user demand, earning the privilege to invest in scalable solutions later.
EARNING THE PRIVILEGE TO SCALE
Building scalable systems is not the starting point but rather a privilege earned after demonstrating product-market fit. Companies like Apple, with Wozniak's hand-soldered computers, and Google, with its initial search algorithm, focused on creating something users wanted first. Only after proving their core value did they invest in the sophisticated, scalable technologies like AirPods or MapReduce that define them today. This earned privilege allows companies to tackle complex engineering challenges once user demand is validated.
REAL-WORLD AND SOFTWARE APPLICATION OF NON-SCALABLE METHODS
The principle of 'doing things that don't scale' extends beyond software, seen in examples like Airbnb founders taking photos or DoorDash employees doing deliveries. In software, it means prioritizing delivering value over perfection. The advice is to find any means to provide something users genuinely want and then address the subsequent problems. This approach, while sometimes 'dirty,' is effective and leads to building successful products by solving challenges as they arise, iteratively improving along the way.
Mentioned in This Episode
●Software & Apps
●Companies
●Concepts
●People Referenced
Doing Things That Don't Scale: A Startup's Guide
Practical takeaways from this episode
Do This
Avoid This
Common Questions
It means prioritizing getting a product that users want out the door quickly, even if the methods used are manual, inefficient, or not designed for mass adoption. The focus is on validation and iteration, with scalability addressed later.
Topics
Mentioned in this video
Coined the term '90/10 solution' and invented Gmail. He built features iteratively by addressing his own needs and fixing issues as they arose, a prime example of doing things that don't scale.
An email service developed by Google, which Paul Buchheit created by first putting his own emails into an existing Google Groups UI and building features iteratively based on his needs.
The technology company where Paul Buchheit developed Gmail and where early search technology faced scaling issues leading to the creation of MapReduce.
An operating system Paul Buchheit preferred to run on his desktop, influencing his decision not to use Outlook on Windows.
The social media giant whose early scaling method involved creating separate PHP instances and MySQL servers for each university, a 'copy-paste' approach to manage growth.
A server-side scripting language used by Facebook in its early days, where a separate instance was run for each university to manage scaling.
A database management system used by Facebook in its early stages, with separate instances created for each university to handle scaling challenges.
A live streaming platform that employed various 'things that don't scale' hacks, such as using static pages to handle traffic spikes and implementing free peering relationships.
The precursor to Twitch, which used a button to turn dynamic pages into static ones to handle massive traffic surges from live streams.
An early social networking site that struggled with scalability issues, particularly with a feature that calculated the user's extended network, leading to performance problems.
A multimedia software platform used by YouTube and IMeans to play video and audio content directly within a web browser without external applications.
An open-source framework that was created based on the MapReduce programming model, used widely by large internet companies.
A social networking site that offered a simplified solution for displaying connections ('so-and-so is in your extended network') as a contrast to Friendster's complex, resource-intensive approach.
A music service that cleverly used open-source video technology by treating music files as video files with zero bits in the video field, effectively creating a video site with no video to play music.
A social news aggregation, web content rating, and discussion website, from which Twitch borrowed the idea of a community-driven translation system.
The video-sharing platform that innovated by playing video directly in the browser using Flash, a technique adopted by IMeans for its music service. Also, a former employer of a network ops guy hired by Twitch for peering expertise.
The original algorithm developed by Google for its search engine, based on a paper, which was effective but faced scaling challenges as the internet grew.
A programming model developed by Google to process large data sets in parallel, created out of necessity when their original indexing process failed due to the web's rapid growth.
A company whose founders took photos of rental properties themselves in the early days, illustrating the 'things that don't scale' principle in the real world.
A technology company whose early, non-scalable methods (like Wozniak's hand-soldering) enabled its later success and the development of scalable products like AirPods.
A food delivery service whose founders reportedly did deliveries themselves, serving as another real-world example of tackling initial scaling challenges.
Co-founder of Apple, known for hand-soldering the original Apple computers, representing an early, non-scalable approach that earned the company the ability to create modern products like AirPods.
More from Y Combinator
View all 127 summaries
54 minThe Future Of Brain-Computer Interfaces
38 minCommon Mistakes With Vibe Coded Websites
20 minThe Powerful Alternative To Fine-Tuning
24 minThe AI Agent Economy Is Here
Found this useful? Build your knowledge library
Get AI-powered summaries of any YouTube video, podcast, or article in seconds. Save them to your personal pods and access them anytime.
Try Summify free