Glossary

AI terms in plain English

203 words you'll meet in AI and machine learning, each explained in everyday language, with a link to the lesson that teaches it. In lessons, these words have a dotted underline: hover or tap one to see its meaning.

A

accuracy: The share of predictions that were right. Misleading when one answer is much more common than the others. Lesson: Precision, recall and F1
activation function: The simple function a neuron applies after its weighted sum. Without it, stacking layers would add nothing. Lesson: Activation functions
agent: A model in a loop that decides what to do next, uses tools, looks at the results, and repeats until the job is done. Lesson: The agent loop
agent loop: The repeating cycle of an agent: think, call a tool, read the result, decide again. Lesson: The agent loop
alignment: Making AI systems reliably do what people intend, safely and honestly. Lesson: Alignment and safety
API: A way for one program to ask another for data or actions, usually over the internet. Lesson: How APIs work
API key: A secret password-like string that identifies you to an API. Never share it or put it in code. Lesson: API keys and secrets
argument: A value you hand to a function when you call it, like the number in round(3.7). Lesson: Functions
array: A grid of numbers (a row, a table, or more) that NumPy can do maths on all at once. Lesson: NumPy arrays
attention: How a transformer lets each token look at the other tokens and pull in the information it needs. Lesson: Attention: the intuition
AUC: Area under the ROC curve: the chance the model scores a random true case above a random false one. 1 is perfect, 0.5 is guessing. Lesson: ROC curves and AUC
autograd: Software that works out gradients automatically from the computation graph, so you never do it by hand. Lesson: Build an autograd engine

B

backpropagation: Working backwards from the error to find how much each weight was to blame, so each can be adjusted. Lesson: Backpropagation
base model: A model straight out of pretraining: it continues text but doesn't yet follow instructions. Lesson: Base models vs assistants
batch: A small group of examples processed together in one training step. Lesson: Mini-batches and SGD
Bayes' rule: A way to update how likely something is when new evidence arrives. Lesson: Conditional probability and Bayes
benchmark: A standard test set used to compare models. Useful, but models can be tuned to the test. Lesson: Benchmarks and their limits
bias: In a model, a starting value added before anything else (the intercept). It can also mean a systematic error; the context tells you which. Lesson: Linear regression
bigram: A pair of neighbouring tokens. A bigram model predicts the next token from only the one before. Lesson: A bigram model
boolean: A value that is either True or False. Lesson: Decisions with if/else
BPE: Byte-pair encoding: a way to build a token vocabulary by repeatedly merging the most common pairs of characters. Lesson: Tokenization and BPE

C

causal mask: A rule that stops each token looking at the tokens after it, so the model can't peek at the answer. Lesson: Causal masking
chain rule: To find how a chain of steps responds to a nudge, multiply how much each step responds. Lesson: The chain rule
chunking: Cutting long documents into smaller pieces so the most relevant pieces can be found and sent to a model. Lesson: Chunking documents
class: A template for making objects that bundle data with the functions that work on it. Lesson: Classes and objects
classification: Predicting which category something belongs to, such as spam or not spam. Lesson: Features and labels
clustering: Grouping similar examples together without being told the groups in advance. Lesson: Clustering
CNN: A network that slides small pattern-detectors across an image to find edges, shapes and objects. Lesson: Convolutional networks
commit: A saved snapshot of your project in git, with a message saying what changed. Lesson: Version control with git
comprehension: A one-line way to build a list from another, like [x * 2 for x in prices]. Lesson: Comprehensions
computation graph: A map of every calculation a model did, which lets the computer trace each result back to its inputs. Lesson: Computation graphs
conditional probability: How likely something is once you know something else is true, like rain given dark clouds. Lesson: Conditional probability and Bayes
confidence interval: A range that probably contains the true value, given the data you sampled. Lesson: Samples and confidence
confusion matrix: A table counting right and wrong predictions by type: true and false positives and negatives. Lesson: Precision, recall and F1
constitutional AI: Training a model to critique and improve its own answers against a written set of principles. Lesson: Constitutional AI
context window: The most text a model can take in at once, measured in tokens: its short-term memory. Lesson: Tokens and the context window
cosine similarity: How alike two vectors' directions are, from 1 (same direction) through 0 (unrelated) to -1 (opposite). Lesson: Dot product and cosine similarity
cross-entropy: A score for how surprised a model is by the right answer. Lower is better; it's the usual training loss for classifiers and language models. Lesson: Cross-entropy and KL divergence
cross-validation: Scoring a model several times on different held-back slices of the data, for a steadier estimate. Lesson: Cross-validation
CSV: A plain-text spreadsheet: one row per line, values separated by commas. Lesson: Files and CSV data

D

decision tree: A model that asks a series of yes/no questions about the inputs to reach a prediction. Lesson: Decision trees
derivative: How fast something changes when you nudge its input: the slope at a point. Lesson: Derivatives
dictionary: A collection of key–value pairs, like a contacts list that maps a name to a phone number. Lesson: Dictionaries
dimension: How many numbers a vector has. A vector with 3 numbers is 3-dimensional. Lesson: Vectors
distillation: Training a small model to imitate a big one, keeping much of its skill at lower cost. Lesson: Distillation
distribution: A description of all the possible outcomes and how likely each one is. Lesson: Distributions
dot product: Multiply two vectors number by number and add up the results. Big when they point the same way. Lesson: Dot product and cosine similarity
DPO: Direct preference optimization: training straight from preference pairs, without a separate reward model. Lesson: DPO and direct preference methods
dropout: Randomly switching off some neurons during training so the network can't lean on any single one. Lesson: Dropout

E

eigenvector: A direction that a matrix only stretches or shrinks, without turning it. Lesson: Eigenvectors (intuition)
embedding: A list of numbers that captures what a piece of text means, so similar meanings get similar numbers. Lesson: Embeddings
embedding layer: A lookup table that gives each word or token its own learned list of numbers. Lesson: Embedding layers
encoder: A transformer that reads the whole text at once to understand it, used for embeddings and classification. Lesson: Encoders and embedding models
endpoint: One specific web address of an API that does one job, like /v1/messages. Lesson: How APIs work
entropy: How unpredictable something is. A fair coin has more entropy than a coin that nearly always lands heads. Lesson: Entropy and surprise
environment variable: A setting stored outside your code, often used to keep secrets such as API keys out of files. Lesson: API keys and secrets
epoch: One full pass of training over every example in the training set. Lesson: Training with gradient descent
eval: A set of test cases with expected results, used to measure how well an AI system does its job. Lesson: Evals
exception: An error Python raises when something goes wrong while the program runs. Lesson: Errors and exceptions
exponential: Growth by repeated multiplying, like doubling every step. Lesson: Exponentials and logarithms

F

f-string: A string with an f in front that can slot values in, like f"Hi {name}". Lesson: Strings and f-strings
F1 score: One number that balances precision and recall. Lesson: Precision, recall and F1
feature: One input column a model learns from, such as a customer's age or number of orders. Lesson: Features and labels
feature engineering: Turning raw data into inputs that make a pattern easier to learn, like days since last purchase. Lesson: Feature engineering
few-shot: Putting a few worked examples in a prompt so the model copies the pattern. Lesson: Examples in prompts
fine-tuning: Further training a pretrained model on a smaller, focused set of examples to change its behaviour. Lesson: Supervised fine-tuning
float: A number with a decimal point, such as 0.5 or 3.14. Lesson: Numbers and arithmetic
forward pass: Running inputs through the network, layer by layer, to get a prediction. Lesson: The forward pass
function: A named, reusable piece of code that takes inputs and gives back a result. Lesson: Functions

G

git: A tool that records versions of your files so you can see changes and undo mistakes. Lesson: Version control with git
GPT: Generative pre-trained transformer: a stack of transformer blocks trained to predict the next token. Lesson: The GPT architecture
GPU: A graphics chip that does thousands of multiplications at once, which is why it trains neural networks fast. Lesson: GPUs and compute
gradient: The list of all the partial derivatives: it points in the direction that makes a result rise fastest. Lesson: Gradients
gradient boosting: Adding small decision trees one at a time, each fixing the mistakes left by the ones before. Lesson: Random forests and boosting
gradient descent: Improving a model by taking small steps in the direction that lowers the error the most, over and over. Lesson: Optimization by walking downhill
grounding: Making a model answer only from sources you give it, and say so when they don't cover the question. Lesson: Hallucinations and grounding

H

hallucination: When a model states something false with confidence, because it predicts plausible text rather than checking facts. Lesson: Hallucinations and grounding
HTTP: The rules web browsers and APIs use to send requests and get responses. Lesson: How APIs work

I

import: The Python command that loads code from a module so you can use it. Lesson: Modules and imports
index: The position of an item in a list. Python counts from 0, so the first item is at index 0. Lesson: Lists
inference: Using a trained model to make predictions, as opposed to training it. Lesson: Inference and the KV cache
integer: A whole number with no decimal point, such as 3 or -12. Lesson: Numbers and arithmetic
interpretability: Research into what is actually happening inside a model's numbers. Lesson: Interpretability

J

JSON: A common text format for data, made of lists and key–value pairs. Most APIs send and receive it. Lesson: JSON

K

key: In attention, what a token offers to be matched on, like a page's index entry. Lesson: Queries, keys and values
KL divergence: A measure of how different one set of probabilities is from another. Lesson: Cross-entropy and KL divergence
KV cache: Saving the keys and values of earlier tokens so the model doesn't recompute them for every new token. Lesson: Inference and the KV cache

L

label: The right answer for an example, which the model learns to predict. Lesson: Features and labels
language model: A model that predicts the next token from the ones before it. Lesson: Language modeling
latency: How long you wait for a response. Lesson: Cost and latency
layer: One stage of a neural network that transforms numbers before passing them on. Lesson: Multi-layer networks
learning rate: How big a step training takes each time. Too big overshoots; too small crawls. Lesson: Learning rate
likelihood: How well some settings explain the data you saw. Training often picks the settings that make the data most likely. Lesson: Maximum likelihood
linear regression: Predicting a number with a straight-line formula: a weight for each input, added up, plus a starting value. Lesson: Linear regression
list: An ordered collection of values in square brackets, like [3, 5, 8]. Lesson: Lists
LLM: A large language model: a program trained on huge amounts of text to predict the next word, which lets it write, answer and reason. Lesson: What an LLM actually does
LLM-as-judge: Using one model to grade another model's answers against a rubric. Lesson: LLM-as-judge
logarithm: The reverse of an exponential: how many times you multiply to get a number. log₁₀(1000) is 3. Lesson: Exponentials and logarithms
logistic regression: A simple classifier: a weighted sum of inputs squeezed into a probability between 0 and 1. Lesson: Logistic regression
logit: A raw score a model gives each option before softmax turns the scores into probabilities. Lesson: Softmax
loop: Code that repeats: for each item in a list, or while something stays true. Lesson: Loops
LoRA: Low-rank adaptation: fine-tuning by training a small add-on instead of all the weights, which is far cheaper. Lesson: LoRA
loss: A single number saying how wrong a model's predictions are. Training tries to make it small. Lesson: Loss functions

M

machine learning: Getting a computer to learn a pattern from examples instead of writing the rules by hand. Lesson: What machine learning is
matrix: A table of numbers with rows and columns. Lesson: Matrices
matrix multiplication: Combining two tables of numbers by taking dot products of rows with columns. It is most of the arithmetic inside a neural network. Lesson: Matrix multiplication
MCP: Model Context Protocol: a standard way to plug tools and data sources into AI apps. Lesson: Model Context Protocol
mean: The average: add the values up and divide by how many there are. Lesson: Mean, variance and spread
mean squared error: A loss: square each prediction's error and average them, so big mistakes count extra. Lesson: Loss functions
method: A function that belongs to an object, called with a dot, like name.upper(). Lesson: Classes and objects
mixture of experts: A model with many specialist sub-networks, where only a few are switched on for each token, saving compute. Lesson: Mixture of experts
MLP: Multi-layer perceptron: the basic neural network, with layers of neurons fully connected to the next layer. Lesson: Multi-layer networks
model: The thing that learns: a formula with adjustable numbers that turns inputs into predictions. Lesson: What machine learning is
module: A file of Python code you can import and reuse. Lesson: Modules and imports
multi-agent: Several AI agents working together, each with its own role, passing work between them. Lesson: Multi-agent systems
multi-head attention: Running several attention patterns side by side, so the model can track different relationships at once. Lesson: Multi-head attention
multimodal: Able to work with more than text, such as images, documents or audio. Lesson: Images and documents

N

nearest neighbors: Predicting from the most similar past examples. Lesson: Nearest neighbors
neural network: A model made of layers of simple units; each layer turns its inputs into new numbers for the next layer. Lesson: Multi-layer networks
neuron: One unit in a neural network: a weighted sum of its inputs, then a simple function. Lesson: The artificial neuron
normal distribution: The bell-shaped curve: most values sit near the average, and fewer the further you go. Lesson: The normal distribution
normalization: Rescaling the numbers inside a network so they stay in a steady range, which makes training smoother. Lesson: Normalization layers
NumPy: A Python library for fast maths on whole arrays of numbers at once. Lesson: NumPy arrays

O

object: One thing made from a class, with its own data, like one customer record. Lesson: Classes and objects
observability: Recording what your AI app did on each request (inputs, outputs, timing, cost) so you can find problems. Lesson: Logging and monitoring
one-hot encoding: Turning a category into columns of 0s and 1s, with a 1 in the column of the matching category. Lesson: Preparing data
open-weight model: A model whose weights are published, so you can download and run it yourself. Lesson: Running open models locally
optimizer: The rule that decides how to change the weights each step, given the gradients. Adam is the usual choice. Lesson: Optimizers
overfitting: When a model memorises its training examples, including their noise, and does worse on new data. Lesson: Overfitting and underfitting

P

package: Code someone else wrote and published, which you install and import, like NumPy. Lesson: Packages and environments
parameter count: How many adjustable numbers (weights) a model has. Large language models have billions. Lesson: Scaling laws
partial derivative: How much a result changes when you nudge just one of several inputs and hold the rest still. Lesson: Partial derivatives
PCA: A way to boil many columns of data down to the few directions where the data varies most. Lesson: SVD and PCA
positional encoding: Extra numbers that tell the model where each token sits in the sequence. Lesson: Positional encodings
precision: Of the things a model flagged, the share that were right. Lesson: Precision, recall and F1
preference data: Pairs of answers to the same prompt where people (or a model) marked which one is better. Lesson: Instruction and preference data
pretraining: The first, huge training run where a model learns language by predicting the next token across vast amounts of text. Lesson: Pretraining at scale
probability: How likely something is, from 0 (never) to 1 (certain). Lesson: Probability basics
prompt: The instructions and text you send to a model. Lesson: Prompt fundamentals
prompt caching: Reusing the work a model already did on a repeated prompt start, which makes calls cheaper and faster. Lesson: Cost and latency
prompt chaining: Splitting a job into steps, where each model call's answer feeds the next call. Lesson: Prompt chaining
prompt injection: An attack where text the model reads (a web page, an email) contains instructions that try to hijack it. Lesson: Prompt injection and security
PyTorch: The most widely used Python library for building and training neural networks. Lesson: PyTorch tensors

Q

quantization: Storing a model's numbers with fewer digits so it uses less memory and runs faster, at a small cost in quality. Lesson: Quantization
query: In attention, what a token is looking for, like a search you type. Lesson: Queries, keys and values

R

RAG: Retrieval-augmented generation: look up relevant documents first, then give them to the model so it answers from them. Lesson: Retrieval-augmented generation
random forest: Many decision trees trained on different samples, voting together. Lesson: Random forests and boosting
reasoning model: A model that works through a problem step by step before answering, trading time for accuracy. Lesson: Reasoning models and effort
recall: Of the things that should have been flagged, the share the model caught. Lesson: Precision, recall and F1
regression: Predicting a number, such as next month's sales. Lesson: Features and labels
regularization: Gently penalising a model for being too complicated, to stop it overfitting. Lesson: Regularization
reinforcement learning: Learning by trial and error from a reward signal, rather than from right answers. Lesson: RLHF
ReLU: The most common activation function: keep positive numbers, turn negative ones into 0. Lesson: Activation functions
residual connection: A shortcut that adds a layer's input to its output, so information and gradients flow easily through deep networks. Lesson: Residual connections
return value: What a function hands back when it finishes. Lesson: Functions
reward model: A model that scores answers by how much people would prefer them. Lesson: RLHF
RLHF: Reinforcement learning from human feedback: training a model towards answers people prefer, using a learned scorer. Lesson: RLHF
RNN: A network that reads a sequence one step at a time, carrying a memory forward. Lesson: Recurrent networks
ROC curve: A chart of how a classifier trades catching more true cases against raising more false alarms. Lesson: ROC curves and AUC

S

sampling: Picking the next token at random according to the model's probabilities, instead of always taking the top one. Lesson: Sampling strategies
scaling laws: The finding that models improve predictably as you add data, size and compute. Lesson: Scaling laws
scikit-learn: A popular Python library of ready-made machine-learning models and tools. Lesson: The scikit-learn workflow
SFT: Supervised fine-tuning: training on example prompts with good answers, so a model learns to respond helpfully. Lesson: Supervised fine-tuning
SGD: Gradient descent using a small random batch each step instead of all the data: noisier but much faster. Lesson: Mini-batches and SGD
shape: The size of an array along each direction, like (3, 4) for 3 rows and 4 columns. Lesson: Tensors
sigmoid: An S-shaped function that squeezes any number into the range 0 to 1. Lesson: Logistic regression
similarity search: Finding the texts whose meaning is closest to a question, by comparing their embeddings. Lesson: Similarity search
slope: How steep a line or curve is: how much it goes up for each step across. Lesson: Derivatives
softmax: A function that turns a list of scores into probabilities that add up to 1, keeping the biggest score biggest. Lesson: Softmax
speculative decoding: Speeding up generation by letting a small model draft several tokens and a big model check them at once. Lesson: Speculative decoding
standard deviation: The typical distance of a value from the average. It is the square root of the variance. Lesson: Mean, variance and spread
statistical significance: Evidence that a difference you measured (say, in an A/B test) is probably real, not luck. Lesson: A/B tests and significance
status code: A number in an API's reply saying how it went: 200 means OK, 404 means not found. Lesson: How APIs work
streaming: Showing a model's reply as it is written, word by word, instead of waiting for all of it. Lesson: Streaming responses
string: A piece of text in a program, written inside quotes, such as "hello". Lesson: Strings and f-strings
structured output: Asking a model to answer in an exact format, such as JSON with set fields, so code can use it. Lesson: Structured output
supervised learning: Learning from examples that come with the right answers. Lesson: Features and labels
system prompt: Standing instructions that set how a model should behave for a whole conversation. Lesson: System prompts and roles

T

temperature: A setting for how adventurous a model's word choices are: low is predictable, high is varied. Lesson: Sampling strategies
tensor: A grid of numbers with any number of directions: a vector is 1-D, a matrix is 2-D, a stack of matrices is 3-D. Lesson: Tensors
terminal: A text window where you type commands to your computer instead of clicking. Lesson: The terminal
test set: Examples kept aside until the very end to give an honest score on unseen data. Lesson: Train, validation and test sets
token: A chunk of text a model reads and writes, often a word or part of a word. Models count and charge by tokens. Lesson: Tokens and the context window
tokenization: Splitting text into tokens, and turning each into a number the model can use. Lesson: Tokenization and BPE
tool use: Letting a model ask your code to run a function, such as a search or a calculator, and use the result. Lesson: Tool use
top-p: Sampling only from the most likely tokens, cutting off the unlikely tail. Lesson: Sampling strategies
traceback: The error report Python prints. Read it from the last line up: that line says what went wrong. Lesson: Errors and exceptions
training loop: The repeated cycle of training: predict, measure the loss, backpropagate, update the weights. Lesson: The training loop
training set: The examples a model learns from. Lesson: Train, validation and test sets
transformer: The design behind today's language models: layers of attention plus small neural networks, repeated many times. Lesson: The transformer block
transformer block: One repeating unit of a transformer: attention, then a small neural network, each with a shortcut and normalization. Lesson: The transformer block
transpose: Flipping a matrix so its rows become columns. Lesson: Transpose, identity and inverse

U

underfitting: When a model is too simple to capture the pattern, so it does badly even on training data. Lesson: Overfitting and underfitting

V

validation set: Held-back examples used to compare and tune models while you work. Lesson: Train, validation and test sets
value: In attention, the information a token passes along when it is matched, like the page you get back. Lesson: Queries, keys and values
vanishing gradients: When the error signal shrinks to almost nothing (or blows up) as it passes back through many layers, so learning stalls. Lesson: Vanishing and exploding gradients
variable: A name that holds a value so your program can use it later, like a labelled box. Lesson: Variables
variance: How spread out values are around their average. Lesson: Mean, variance and spread
vector: An ordered list of numbers, like [3, 1, 4]. It can stand for a point, a direction, or a description of something. Lesson: Vectors
vector database: A database built to store embeddings and quickly find the closest ones to a query. Lesson: Vector databases
virtual environment: A separate folder of installed packages for one project, so projects don't clash. Lesson: Packages and environments
vocabulary: The full list of tokens a model knows. Lesson: Tokenization and BPE

W

weight: An adjustable number inside a model that says how much an input matters. Training sets the weights. Lesson: Linear regression