Residual block

Also written: residual blocks

A layer that adds its result to its own input (x + f(x)), so the input always has a clean path through.

Other terms from the same lesson

• residual connection: A shortcut that adds a layer's input to its output, so information and gradients flow easily through deep networks.
• residual stream: The running total that flows through a transformer: each layer reads it and adds its result back in, instead of replacing it.