Glossary · Phase 1 · Building with LLMs
Latency
How long you wait for a response.
Learn it properly
Cost and latency · Prompt caching, batching and routing to cheaper models. About 30 minutes.
Other terms from the same lesson
- prompt caching: Reusing the work a model already did on a repeated prompt start, which makes calls cheaper and faster.