The Pile
Software / App
A diverse dataset created by Eleuther AI from various sources including Common Crawl, PubMed, Arxiv, GitHub, and books, aiming for open-source accessibility.
Mentioned in 2 videos
Save the 2 videos on The Pile to your own pod.
Sign up free to keep building your knowledge base on The Pile as more episodes are added.
Videos Mentioning The Pile

LLMs for Agent Builders: Context Engineering | Stanford CS329Z Lecture 2 with Michael Ryan
Stanford Online
A large, diverse dataset released publicly for training language models, including sources beyond the internet.

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 13: Data (Sources, Datasets)
Stanford Online
A diverse dataset created by Eleuther AI from various sources including Common Crawl, PubMed, Arxiv, GitHub, and books, aiming for open-source accessibility.