CUDA

The spelled-out intro to neural networks and backpropagation: building micrograd

Andrej Karpathy

Mentioned when tracing PyTorch internals (there are separate CPU and GPU kernels for operations such as tanh backward).

Answer.ai & AI Magic with Jeremy Howard

Latent Space

A parallel computing platform and programming model created by Nvidia, essential for GPU acceleration in AI, with community efforts around it mentioned.

E167: Google's Woke AI disaster, Nvidia smashes earnings (again), Groq's LPU breakthrough & more

All-In Podcast

Nvidia's proprietary parallel computing platform and API, mentioned as a key factor in Nvidia's dominance and a challenge for competitors.

All-In Summit: In conversation with Vinod Khosla

All-In Podcast

Nvidia's parallel computing platform, mentioned as a potential source of lock-in if alternative AI development methods are not explored.

llm.c's Origin and the Future of LLM Compilers - Andrej Karpathy at CUDA MODE

Latent Space

A parallel computing platform and programming model developed by Nvidia for general processing on graphics processing units (GPUs). Essential for accelerating AI workloads.

Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI

Latent Space

A parallel computing platform and API model created by NVIDIA, mentioned in the context of generating GPU code for PyTorch operators.

Building an open AI company - with Ce and Vipul of Together AI

Latent Space

NVIDIA's parallel computing platform and API, mentioned as an area of expertise sought by Together AI.

The $3 Trillion AI Coding Opportunity

a16z Deep Dives

Jim Keller: The Future of Computing, AI, Life, and Consciousness | Lex Fridman Podcast #162

Lex Fridman

An NVIDIA computing platform and API model that enables parallel computation on GPUs, initially for HPC and later benefiting from the AI trend.

The State of Silicon and the GPU Poors - with Dylan Patel of SemiAnalysis

Latent Space

Mentioned as low-level programming that ideally users wouldn't have to write directly.

The End of Finetuning — with Jeremy Howard of Fast.ai

Latent Space

A parallel computing platform and API model created by Nvidia, mentioned in the context of writing low-level GPU code for optimizations.

FlashAttention-2: Making Transformers 800% faster AND exact

Latent Space

NVIDIA's parallel computing platform and API model, used for writing code that runs on NVIDIA GPUs.

Ep 18: Petaflops to the People — with George Hotz of tinycorp

Latent Space

CUDA is described as a C-like language that compiles through multiple stages, all of which are term-complete, unlike the approach favored by George Hotz.

Chris Lattner: The Future of Computing and Programming Languages | Lex Fridman Podcast #131

Lex Fridman

NVIDIA's parallel computing platform and programming model for GPUs. It allows programmers to write scalar-like code that is implicitly parallelized across thousands of GPU cores.

"Founder Mode," DOJ alleges Russian podcast op, Kamala flips proposals, Tech loses Section 230?

All-In Podcast

NVIDIA's parallel computing platform, cited as a dependency for PyTorch which makes it problematic for long-term hardware independence in AI development.

Chris Lattner: Compilers, LLVM, Swift, TPU, and ML Accelerators | Lex Fridman Podcast #21

Lex Fridman

Nvidia's parallel computing platform and programming model, which utilizes Clang and LLVM for graphics and GPGPU.

The Shape of Compute (Chris Lattner of Modular)

Latent Space

NVIDIA's parallel computing platform and API, which Modular aims to replace or abstract away with its own stack.

Jensen Huang on GPUs - Computerphile

Computerphile

Information Theory for Language Models: Jack Morris

Latent Space

A parallel computing platform and programming model mentioned as a valuable skill to learn, though not the only path to high employability.

DeepWiki: The GitHub Encyclopedia

Latent Space

A parallel computing platform and programming model used for comparison with Python implementations in research projects.

DOJ targets Nvidia, Meme stock comeback, Trump fundraiser in SF, Apple/OpenAI, Texas stock market

All-In Podcast

Mentioned as a potential area of antitrust investigation for NVIDIA, regarding its use to lock customers into NVIDIA GPUs.

Scarlett Johansson vs OpenAI, Nvidia's trillion-dollar problem, a vibecession, plastic in our balls

All-In Podcast

NVIDIA's parallel computing platform and programming model, described as the software interface everybody uses, giving NVIDIA a strategic advantage.

Theano Tutorial (Pascal Lamblin, MILA)

Lex Fridman

A parallel computing platform and API model created by Nvidia, used by Theano for GPU computations and code generation.

Torch Tutorial (Alex Wiltschko, Twitter)

Lex Fridman

NVIDIA's parallel computing platform and programming model that allows software developers to use a CUDA-enabled graphics processing unit (GPU) for general purpose processing.