Phase 2 · Math, when you need it · Linear algebra
Tensors
Arrays with more axes: batches, sequences, heads.
Open this lesson Create a free account
Why it matters
Real AI models never look at one example at a time. A language model works on a batch of sentences at once. Each sentence is a row of words, and each word is a list of numbers. That's three layers of grouping. Arrays with layers like this are called tensors (the main data type in PyTorch is literally called Tensor), and most of the code inside a model is reshaping them and multiplying them. If you can follow the shapes, you can follow the model.
What you'll be able to do
- Describe scalars, vectors, matrices and tensors by their number of axes
- Read shapes like (batch, tokens, features) and say what each axis means
- Reduce, reshape and reorder axes with NumPy
- Use batched matrix multiplication the way transformers do
Before this lesson
Next in this phase: Eigenvectors (intuition)