Positional encoding

Also written: positional encodings, position embedding, position embeddings

Extra numbers that tell the model where each token sits in the sequence.

Other terms from the same lesson

• RoPE: Rotary position embeddings: marking word order by turning each query and key by an angle that grows with position, so attention scores depend on how far apart two tokens are. Most modern open models use it.