Glossary · Phase 2 · Math, when you need it
Information content
How surprising one outcome is, in bits: rare events carry more information than common ones.
Learn it properly
Entropy and surprise · Measure uncertainty in bits. About 25 minutes.
Other terms from the same lesson
- bit: The unit of information: the answer to one fair yes-or-no question, like one coin flip.
- entropy: How unpredictable something is. A fair coin has more entropy than a coin that nearly always lands heads.
- nat: A unit of information that uses natural logs instead of base 2. One nat is about 1.44 bits.