Developer tools

Exact text token counter

Count plain text with the pinned cl100k_base encoding.

Processed locally

Exact for this encoding and literal text, not a full API request or every model. Processing stays on your device.

Your answer will appear here.

Help and method

Uses gpt-tokenizer 4.0.0 and cl100k_base. Spaces, line breaks and Unicode remain unchanged; special-token markers count as literal text. Maximum 20,000 UTF-16 units. Messages, tools, images and output tokens are excluded.

OpenAI gpt-tokenizer 4.0.0 Methods and reliability

Worked example

An example you can try or adapt to your own values.

Text: hello world

Result

Tokens: 2; Token IDs: 15339, 1917

Worked example

Result

Useful next