SGLang
Software / App
An inference engine and framework for large language models, developed for high performance, better usability compared to other frameworks, and optimized for specific models and techniques like continguous decoding and radix attention.
Mentioned in 2 videos
Save the 2 videos on SGLang to your own pod.
Sign up free to keep building your knowledge base on SGLang as more episodes are added.
Videos Mentioning SGLang
DeepSeek V3, SGLang, and the state of Open Model Inference in 2025 (Quantization, MoEs, Pricing)
Latent Space
An inference engine and framework for large language models, developed for high performance, better usability compared to other frameworks, and optimized for specific models and techniques like continguous decoding and radix attention.
The Inference Frontier: 10x Faster Models to Self-Optimizing AI — Philip Kiely & Ali Taha, Baseten
Latent Space
An open-source inference engine, often receives weights ahead of time for new models.