Glossary · Phase 1 · Building with LLMs
Time to first token
How long you wait before the first word of a reply appears. Most of it is the model reading the prompt.
Learn it properly
Streaming responses · Show output as it's generated instead of waiting. About 25 minutes.
Other terms from the same lesson
- server-sent events: A simple way for a server to keep a connection open and push a stream of small updates, used to send a reply token by token.
- streaming: Showing a model's reply as it is written, word by word, instead of waiting for all of it.