Causal mask

Also written: causal masking

A rule that stops each token looking at the tokens after it, so the model can't peek at the answer.

Other terms from the same lesson

• lower-triangular: A grid with numbers on and below the diagonal and zeros above it. The causal mask has this shape: each token sees only itself and earlier ones.
• teacher forcing: Training on real text by predicting every next token at once from the true earlier tokens, not the model's own guesses.