Key Moments
SF Compute: Commoditizing Compute
Latent Space PodcastKey Moments
SF Compute is building a liquid GPU market, focusing on reducing risk and financialization.
Key Insights
The GPU market differs significantly from the CPU market due to customer price sensitivity and scaling laws.
CoreWeave's success stems from long-term contracts and treating GPU clusters as real estate, not traditional cloud services.
SF Compute functions as a GPU marketplace and broker, creating liquidity by matching buyers and sellers of compute.
A significant portion of the current GPU market's challenges and prices are driven by credit risk and the need for financial instruments like futures.
SF Compute emphasizes auditing, standardization, and building trust to de-risk both technical and financial aspects of GPU compute procurement.
The company aims to reduce industry hype by setting low expectations and over-delivering, offering a calm, reliable alternative.
UNDERSTANDING THE GPU MARKET DYNAMIC
The conversation highlights a fundamental difference between CPU and GPU markets. Unlike CPUs, where customers often buy based on capacity needs and are less price-sensitive if software margins are high, GPU customers are extremely price-sensitive due to the sheer cost of hardware and the direct correlation between more GPUs and increased revenue/performance. This price sensitivity means that software and services, which drive high margins in CPU clouds, are less effective in the GPU space. CoreWeave's success is attributed to recognizing this by focusing on long-term contracts and avoiding short-term, on-demand sales which are precarious in the GPU market.
COREWEAVE'S INNOVATIVE BUSINESS MODEL
CoreWeave's financial performance is likened to a real estate business rather than a traditional cloud provider or software company. They achieve high margins by securing long-term, locked-in contracts with creditworthy clients, effectively mitigating the risk of GPU depreciation and market fluctuations. This strategy allows them to secure lower costs of capital from lenders. The analysis suggests that hyperscalers like AWS, Azure, and Google may struggle to achieve similar margins by simply reselling NVIDIA GPUs, as it competes with their core, higher-margin businesses and doesn't leverage their primary strengths.
SF COMPUTE: BUILDING A LIQUID GPU MARKET
SF Compute was initially an AI lab that pivoted to brokering GPU clusters to survive bankruptcy. They evolved into a compute marketplace designed to create liquidity in the GPU market. Unlike traditional providers, SF Compute enables short-term rentals, even down to hourly purchases, by aggregating supply and demand. This provides flexibility for customers who need burst capacity or cannot commit to year-long contracts, a model that addresses the needs of startups and researchers often overlooked by larger providers.
FINANCIALIZATION AND RISK REDUCTION
A core tenet of SF Compute's strategy is the financialization of compute, aiming to reduce both technical and financial risk. They believe that creating a liquid spot market for GPUs is essential for developing futures contracts. These financial instruments, similar to those in agriculture or commodities, would allow data centers to lock in prices, reduce their own borrowing costs, and de-risk their operations. This, in turn, would prevent inflated venture capital valuations and create a more stable economic system for AI development.
AUDITING AND STANDARDIZATION FOR RELIABILITY
Technical reliability is paramount, especially in a marketplace. SF Compute invests heavily in auditing GPU clusters through burn-in processes, performance testing, and continuous active/passive checks. This rigorous approach ensures hardware integrity and identifies faulty components, a critical step often missed by resellers. By managing clusters from bare metal up and standardizing contracts, SF Compute aims to provide a dependable and predictable service, mitigating the inherent hardware risks associated with high-performance computing.
A CALM BRAND IN A HYPED INDUSTRY
In contrast to the often hyped and high-expectation atmosphere of the AI industry, SF Compute cultivates a brand focused on being calm and nature-inspired. This deliberate choice sets low expectations, allowing them to exceed them with reliable service and competitive pricing. This approach is rooted in their own origin story of near-bankruptcy, fostering a desire to de-escalate the industry's intensity. Their branding, including minimalist website design and imagery of San Francisco's natural beauty, reflects this ethos of grounded reliability.
Mentioned in This Episode
●Products
●Software & Apps
●Companies
●Organizations
●People Referenced
Common Questions
GPUs are fundamentally different due to their high cost and the scaling laws that apply to AI workloads. Unlike CPUs, where demand plateaus, adding GPUs directly translates to increased revenue for model training and inference. This makes customers highly price-sensitive and focused on maximizing compute within their budget.
Topics
Mentioned in this video
Founder of SF Compute, discussing the economics of GPU compute, market dynamics, and the company's unique approach.
Mentioned as someone who advised Evan Conrad with the mantra 'don't die' during his startup journey.
Head of Design at SF Compute, praised for his attention to detail and craft.
Credited with coining or naming the concept of price sensitivity in GPU markets, extending it to the cost implications for AI training runs.
CTO of SF Compute and co-founder of Voltage Park, described as a kind and earnest leader.
COO of SF Compute, with a design background, contributing to the company's brand 'vibes'.
Google Cloud Platform, mentioned alongside AWS as a provider of commodity CPU cloud services.
A major cloud provider mentioned in the context of CPU cloud models and how companies like Gusto use their services.
A company mentioned as an example of a successful entity that does not own underlying compute hardware, highlighting the benefit of splitting compute and real estate businesses.
The primary programming language used at SF Compute, though they are open to engineers with strong Linux backgrounds.
The operating system expertise that SF Compute is looking for in systems engineers.
A GPU cloud provider mentioned as a place SF Compute initially expected to rent GPUs but found was not feasible for short-term needs.
A dual-entry accounting database mentioned as a tool for building systems that don't lose money.
An early language model discussed in the context of Evan Conrad's past excitement about building applications.
Evan Conrad's previous startup, a mental health app that faced retention issues.
A large language model whose training cost is estimated and used to illustrate the potential for companies to consider building their own chips.
A GPU cloud company co-founded by Eric Park, the current CTO of SF Compute.
A cloud provider that was successful by locking in long-term contracts and focusing on customers with low credit risk, differentiating itself from traditional CPU cloud models.
An example of a company that uses CPUs for web servers, where demand plateaus after a certain capacity is met, contrasting with GPU demand.
A startup customer of SF Compute.
A company aiming to create the most liquid GPU market, focusing on a brokerage model and aiming to reduce technical and financial risk in compute procurement.
An HR service company used as an example of a CPU-dependent business with a flat demand curve, contrasting with GPU-intensive workloads.
A hyperscaler that could potentially compete in the GPU cloud space but maintains a complex relationship with NVIDIA, its GPU supplier.
A cloud provider mentioned as an example which might lose money by attempting to bundle hardware and software services.
A major AI research company that is a significant customer for GPU compute and is discussed in relation to its potential to build its own chips.
A startup customer of SF Compute, mentioned by its acronym PHND.
A customer communication platform considered a successful model for handling support use cases related to writing emails.
A distributed systems company founded by Evan Conrad that pivoted from Quirk, facing competition from 'people building a house'.
Manufacturer of GPUs, whose hardware is central to the discussion around GPU cloud economics and the challenges of reselling their products.
A cloud computing provider discussed in the context of potentially losing money by coupling software and hardware services.
A startup accelerator program where Dalton Caldwell is mentioned, who advised Evan Conrad.
A high-end NVIDIA GPU model frequently discussed in the context of supply, demand, and pricing in the GPU cloud market.
A specific model of NVIDIA GPU that SF Compute initially sought to rent in large quantities.
Google's Tensor Processing Units, mentioned as an alternative to NVIDIA GPUs that Google also offers.
Application-Specific Integrated Circuit, mentioned in the context of companies like OpenAI potentially designing their own chips for AI workloads.
An organization whose grantees have worked with SF Compute, highlighting the company's support for researchers.
An organization that set up the Andromeda cluster, discussed in the context of VCs providing compute resources.
A $100 million GPU cluster set up by AI Grants, discussed as an example of VCs providing compute resources and its arbitrage on credit risk.
More from Latent Space
View all 101 summaries
86 minNVIDIA's AI Engineers: Brev, Dynamo and Agent Inference at Planetary Scale and "Speed of Light"
72 minCursor's Third Era: Cloud Agents — ft. Sam Whitmore, Jonas Nelle, Cursor
77 minWhy Every Agent Needs a Box — Aaron Levie, Box
42 min⚡️ Polsia: Solo Founder Tiny Team from 0 to 1m ARR in 1 month & the future of Self-Running Companies
Found this useful? Build your knowledge library
Get AI-powered summaries of any YouTube video, podcast, or article in seconds. Save them to your personal pods and access them anytime.
Try Summify free