Glossary · Phase 1 · Building with LLMs
Token bucket
A way to rate-limit: each user has a bucket of request tokens that slowly refills. Each request takes one, and an empty bucket means wait.
Learn it properly
Shipping an LLM app · Deploy, secure keys, rate-limit and handle failures. About 30 minutes.
Other terms from the same lesson
- backend: The part of an app that runs on your own server, out of users' sight. It's where secrets like API keys belong.
- front end: The part of an app that runs in the user's browser or phone. Anyone can read its code.
- health check: A simple web address your host calls now and then to check your app is still running.
- staging: A private copy of your app where you try changes before real users see them.