Token
The fundamental unit of data processed by an LLM, roughly equivalent to a word or part of a word in English.
What it is
AI models do not read human text; they read tokens. A token is a chunk of text, typically consisting of about four characters or three-fourths of a standard English word. API costs, context limits, and processing speeds are all calculated based on the number of input and output tokens.
When you would use it
You track tokens when managing the budget and performance constraints of an AI application, ensuring you don't exceed rate limits or exhaust the context window.
Common operations
- Calculating the financial cost of a specific API request.
- Determining if a document will fit inside a model's context window.