chaitanya.dev / Tools / LLM Token Estimator

LLM Token Estimator

Quick character-based token estimates for GPT-4 / 4o, Claude, and Llama. Paste text and get a ballpark count without shipping it to an API.

Estimate LLM tokens without hitting an API

Shipping text to a tokenizer endpoint just to count tokens is expensive, slow, and — if the text is sensitive — a leak waiting to happen. This tool estimates tokens from character counts using published chars-per-token ratios. For English prose it’s within ±10% of the real tokenizer; for code and non-Latin scripts it’ll underestimate.

Characters
0
Words
0
GPT-4 / 4o
~0
Claude
~0
Llama 3
~0
Ratios used: GPT-4/4o ≈ 4.0 chars/token, Claude ≈ 3.8, Llama 3 ≈ 3.5. These are English-prose averages. Code typically runs denser (fewer chars/token); highly tokenized content like CJK or code-heavy payloads can be 2–2.5 chars/token, so treat these as lower-bound estimates for non-prose.

When to actually trust this

  • Cost estimation up to ±10% for English prose. Good enough for budgeting a feature or sanity-checking a prompt cost before a batch run.
  • Context-window planning. If the estimator says “fits 32k” you’ll likely stay within a 32k limit with headroom. If it says “fits 128k” on a 131k model, tokenise properly before shipping.
  • Don’t trust it for billing reconciliation. Use the provider’s tokenizer or the usage field returned by the API.

Code and non-Latin scripts will come in denser than this estimator suggests. If you’re pasting a big JSON blob or Chinese/Japanese text, assume the real count is 20–50% higher than what you see here.