Speaking French to an AI Costs 31% More Tokens

The number checks out

The GDPR text in English: 65,755 tokens. The same text in French: 85,998. That is 30.8% more to say the exact same thing, a figure that rounds very neatly to 31%.

The number has been making the rounds since October 1st, after Next.ink took that same regulation and ran that same count. We went and pulled the text ourselves, from the EU publications server, in French, English, and German, and ran it through ten different tokenizers. The scripts and the source texts are kept on file: anyone can rerun the measurement.

Here is why it matters: providers bill by the token, both for what you send in and what you get back out. The checkout doesn't weigh your groceries, it scans barcodes. Same basket, more items scanned, bigger bill. The percentage isn't an academic curiosity: it shows up directly on an API invoice at the end of the month.

So the number holds up. What doesn't hold up is what people conclude from it.

The GDPR is the worst case, not the typical one

See 31% and the instinct is to assume all French text costs a third more to process. So we reran the measurement on four other kinds of text, using the tokenizer behind OpenAI's current models. Journalism: +24.7%. Conference speech transcripts: +20.7%. A Jules Verne novel: +19.5%. Commented code: +10.7%.

The gap shrinks by a factor of three between an EU regulation and a code file. Dense legislative text is the worst-case fuel-economy rating of language, not the daily commute: it captures the extreme, not the typical trip. If your own usage looks more like notes, code, or everyday conversation, the real surcharge is two to three times smaller than the number making the rounds. Treat the GDPR figure the way you'd treat a lab-rated fuel economy sticker: a ceiling worth knowing, not a prediction of your own trip.

Half the gap has nothing to do with AI

Here is the part nobody mentions. Before tokens even enter the picture: the French GDPR text runs 414,756 characters, against 360,047 in English. French is 15.2% longer in characters, just to say the exact same thing.

No tokenizer will ever shave that half off. It doesn't come from the technology, it comes from the language itself: the French original reads "de la mise en œuvre du présent règlement," against the English "implementing this Regulation." Same clause, more letters, and no amount of clever engineering on the tokenizer side will ever close that part of the gap.

So out of the 30.8% measured on the GDPR, the machine itself only adds 13.5% on top of that extra length. "French costs 31% more" is true, but it pins on AI a gap that is, by half, the dictionary's fault, not the tokenizer's.

German works as a useful control, and it complicates the picture further. The same regulation runs to fewer characters than French does, and to 51,324 words against 62,909. Yet it costs 28.2% more tokens than English, nearly as much as French. German compounding builds long words the tokenizer has never seen, and the tokenizer chops them into fragments. Counting characters alone doesn't settle the question either.

And the remaining 13.5% is already shifting

This isn't some constant of nature. It's a dial somebody turns.

With the tokenizer behind GPT-3.5 and GPT-4, the French penalty was 34.4%. With the one behind the models that followed, it drops to 13.5%. Same text, same language, same company: what changed was the tokenizer's vocabulary, not French itself.

Across our ten tokenizers, the spread runs from +11% to +67%. Fifty-six points of range, where a single figure tends to circulate. And the gentlest tokenizer of the lot isn't a commercial model at all: it's CroissantLLM, an academic model trained equally on French and English, whose penalty on French is flat-out negative. Its vocabulary turns out to be more efficient in French than in English.

You might assume a French lab would nail this by default. It doesn't: on the same text, Mistral's most recent public tokenizer comes in at 28.7%, against 30.8% for OpenAI. Better, but only by two points. A model trained in France doesn't ship with a built-in discount on French.

Then there's the control that closes the loop. That same CroissantLLM, so forgiving toward French, slaps a +112% penalty on German, the worst figure in the entire table. A tokenizer doesn't understand languages, and it plays no favorites among them. It simply knows whatever it was shown during training, and stumbles on everything it wasn't. Swap the training data and the ranking of languages swaps right along with it.

What the number actually tells you

Anthropic's own documentation adds the line that sums up the whole story: starting with Claude 4.7, the same text produces "about 30% more tokens" than it used to, because the tokenizer changed underneath it. Same text, same provider, six months apart.

A text's token count, then, isn't a fixed property of that text. "The cost of French" doesn't exist as some standalone fact: there's the cost of French at a given provider, with a given generation of its tokenizer, applied to a given kind of text.

And if the goal is simply to spend less, there's an easier lever than switching your working language entirely. The author of the original measurement says as much himself: asking for shorter answers saves far more.