Common Crawl / C4

Software / AppMentioned in 1 video

Large web-crawl data sources often used in data mixtures for LLM pretraining; discussed in the training-data section.

Videos Mentioning Common Crawl / C4

Let's reproduce GPT-2 (124M)

Let's reproduce GPT-2 (124M)

Andrej Karpathy

Large web-crawl data sources often used in data mixtures for LLM pretraining; discussed in the training-data section.