Token (AI)
A token in AI is a chunk of text the model works with instead of letters or words. Usually it is part of a word, about four characters in English. Tokens are the unit for both context length and price.
How it works
Text is split into tokens by frequency. Common words become a single token; rare ones are broken into pieces. So "the" is one token while an invented word might take five.
Cyrillic splits more finely than Latin script: the same meaning in Russian usually takes one and a half to two times the tokens it takes in English. That feeds straight into both price and how much fits in the context.
The model sees tokens, not the letters inside them. Hence its long-standing weakness at questions like "how many r's are in this word": it works with fragments, not characters.
What that is in words
Rough ratios, enough to estimate size and cost.
| 1 token (English) | ≈ 4 characters |
| 1,000 tokens | ≈ 750 words |
| An A4 page | ≈ 500–700 tokens |
| The same text in Russian | ×1.5–2 |
A 128,000-token context holds roughly a mid-length novel in English, and noticeably less in Russian. A request is priced on input plus output tokens together, so a long conversation gets more expensive with every message.