The Shape of Compute (Chris Lattner of Modular)

A significant challenge was proving that a startup could achieve state-of-the-art performance without relying on existing proprietary systems like CUDA. Many in the industry believed this was impossible for a new player.

Mojo is designed to be a high-performance language in the Python family, offering significantly faster execution speeds, often surpassing even Rust. It aims to make it easier to write performant code, especially for GPUs, and to seamlessly extend existing Python codebases without complex bindings.

Max is an AI inference-focused framework built by Modular that is designed to work directly with Mojo. It emphasizes performance, control, and low latency, aiming to keep Python out of the critical model logic loop and offering features like automatic kernel fusion.

Modular's approach, inspired by LLVM, allows for clean separation of concerns among teams specializing in different areas like parsing, optimization, and code generation. This modularity makes the system more adaptable to rapidly changing AI requirements and easier to evolve compared to monolithic frameworks.

Modular focuses on a tightly integrated, co-designed stack for performance and control. VLM, while ambitious, is described as a 'hot mess' with broad but sometimes unreliable hardware support. SGLang is seen as a more focused and goal-oriented project.

The core Max framework and Mojo language are free to use on NVIDIA and CPUs at any scale. Modular charges for cluster management, enterprise support, and services that help customers scale their AI workloads and manage compute.

Key lessons include avoiding a 'hot start' by launching at an early version (Mojo started as 0.1), and ensuring internal adoption by using the technology within the company (Modular is Mojo's first customer), unlike Swift which had limited internal awareness at launch.

Lattner believes inference scales with the customer base and application of AI, whereas training scales with research teams. By focusing on production use cases and the gap between research and deployment, Modular positions itself for broader growth as AI adoption increases.

AI coding tools are seen as valuable assistants that can significantly scale the 'on-ramp' for learning new languages or technologies. They handle much of the mechanical work and boilerplate, allowing developers to focus on core logic and problem-solving, acting like hiring another engineer for the team.

Code's primary purpose is for humans to understand what it does, not just to instruct the computer. It needs to be readable, expressive, and allow reasoning about constraints like latency, memory, and cost, even as AI tools become more prevalent in code generation.

Lattner finds that leading his own startup is more personal than working at established companies, making team departures more impactful. He also emphasizes the learning curve in building infrastructure from scratch and managing team dynamics during periods of intense development and uncertainty.