Glossary · Phase 1 · Building with LLMs
Tail latency
How long the slowest requests take, such as the slowest 5% (p95). Users notice these waits even when the average is fine.
Learn it properly
Logging and monitoring · See what your LLM app actually did in production. About 25 minutes.
Other terms from the same lesson
- observability: Recording what your AI app did on each request (inputs, outputs, timing, cost) so you can find problems.
- percentile: The value below which a given share of results fall. The 95th percentile beats 95% of the others.
- redact: To hide sensitive parts of text before storing it, such as replacing an email address with [email].
- span: One step inside a trace, such as a single model call or search, with its own start and end time.
- trace: A record of every step in one multi-step run, such as an agent's model calls and tool calls, grouped under one id.