Byte Pair Encoding

Concept

A data compression technique adapted for language models, used to construct vocabularies tailored to data by merging frequent byte pairs.

Mentioned in 1 video

Videos Mentioning Byte Pair Encoding

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 1: Overview, Tokenization

Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 1: Overview, Tokenization

Stanford Online

A data compression technique adapted for language models, used to construct vocabularies tailored to data by merging frequent byte pairs.