Phase 2 · Math, when you need it · Linear algebra
SVD and PCA
Compress data to its most important directions.
Open this lesson Create a free account
Why it matters
Real data has many correlated features, and embeddings have hundreds or thousands of dimensions. PCA finds the few directions that carry most of the variation, so you can compress data, remove noise, or plot 1,536-dimensional embeddings on a 2D chart and actually see clusters. The tool underneath, the singular value decomposition, also explains LoRA, the cheap fine-tuning method you'll meet in Phase 6.
What you'll be able to do
- Explain PCA as finding the directions in which data varies most
- Run PCA with NumPy's SVD and read the explained variance
- Project high-dimensional data to 2D to look at it
- Use a low-rank approximation to compress a matrix, and connect it to LoRA
Before this lesson
Next in this phase: Functions and graphs