SGLang
An inference engine and framework for large language models, developed for high performance, better usability compared to other frameworks, and optimized for specific models and techniques like continguous decoding and radix attention.
Save the 4 videos on SGLang to your own pod.
Sign up free to keep building your knowledge base on SGLang as more episodes are added.
Videos Mentioning SGLang
DeepSeek V3, SGLang, and the state of Open Model Inference in 2025 (Quantization, MoEs, Pricing)
Latent Space
An inference engine and framework for large language models, developed for high performance, better usability compared to other frameworks, and optimized for specific models and techniques like continguous decoding and radix attention.
Why Inference Matters 12x More Than Training | YC Paper Club
Y Combinator
An inference engine that Luminal's compiler works within, leveraging its solutions for speculative decoding.
The Inference Frontier: 10x Faster Models to Self-Optimizing AI — Philip Kiely & Ali Taha, Baseten
Latent Space
An open-source inference engine, often receives weights ahead of time for new models.
Why The Harness Matters More Than The Model | YC Paper Club
Y Combinator
An inference engine for running language models.