Common Crawl / C4

Software / App

Large web-crawl data sources often used in data mixtures for LLM pretraining; discussed in the training-data section.

Mentioned in 1 video

Videos Mentioning Common Crawl / C4

Let's reproduce GPT-2 (124M)

Let's reproduce GPT-2 (124M)

Andrej Karpathy

Large web-crawl data sources often used in data mixtures for LLM pretraining; discussed in the training-data section.