Residual stream

The running total that flows through a transformer: each layer reads it and adds its result back in, instead of replacing it.

Other terms from the same lesson

• residual block: A layer that adds its result to its own input (x + f(x)), so the input always has a clean path through.
• residual connection: A shortcut that adds a layer's input to its output, so information and gradients flow easily through deep networks.