Phase 4 · Deep learning · Architectures
Embedding layers
Learned lookup tables that turn tokens into vectors.
Open this lesson Create a free account
Why it matters
Neural networks need numbers, but much of the world comes as IDs: words, tokens, products, customers. An embedding layer gives every ID its own learned vector, and training arranges those vectors so that things used in similar ways end up close together. It's the first layer of every language model, and the same idea powers recommendations and lookalike audiences.
What you'll be able to do
- Explain an embedding layer as a learned lookup table, equivalent to one-hot × matrix
- Use nn.Embedding and see which rows receive gradients
- Learn product embeddings from co-purchases and find similar items
- Relate embedding tables to LLM tokens and to the embedding APIs from Phase 1
Before this lesson
Next in this phase: PyTorch tensors