Phase 1 · Building with LLMs · Quality & production
Cost and latency
Prompt caching, batching and routing to cheaper models.
Open this lesson Create a free account
Why it matters
An LLM feature that costs $0.02 a request is a rounding error in a demo and $6,000 a month at 10,000 requests a day. The same few levers decide both the bill and the wait: how much text you send, how much you get back, which model reads it, and whether repeated text is cached. Teams routinely cut costs by more than half with no loss in quality by applying them in order.
What you'll be able to do
- Work out what a request costs from its usage fields, including cache reads and writes
- Structure prompts so prompt caching works, and know when it pays off
- Rank the other cost levers: model choice, output length, effort, batching, trimming input
- Explain what makes a reply slow and how to make it feel fast
Before this lesson
Next in this phase: Logging and monitoring