Token
A chunk of text (roughly ¾ of a word) that AI models read and generate one piece at a time.
01 Definition
AI language models don't see whole words — they break text into tokens, small chunks that might be a whole word, part of a word, or punctuation. A rough rule of thumb: 100 tokens ≈ 75 words in English.
02 Why It Matters
Tokens are how AI companies measure usage and set pricing — 'pay per token' is the actual billing unit behind most AI APIs, and context limits are measured in tokens, not words or pages.
03 Example
The sentence 'AI tools are useful' is about 5 tokens.