BPE

Also written: byte-pair encoding

Byte-pair encoding: a way to build a token vocabulary by repeatedly merging the most common pairs of characters.

Other terms from the same lesson

• byte-level BPE: BPE that starts from the 256 possible bytes instead of letters, so it can spell out any text at all, emoji included.
• pre-tokenization: Splitting text into rough chunks, such as words and spaces, before the tokenizer merges pieces inside each chunk.