Nine detectors analyzed the same pages of Orélien's novel

On September 21, an X account called Balance ton Claude posted screenshots of the first two chapters of C'était ça ou mourir, the novel by Thélyson Orélien. Splashed across them: Pangram's verdict, "100% AI."

AFP ran the same extracts through eight other detectors. With InVID-WeVerify, the tool the agency co-built, the passages come back "very probably written by a human." The other seven, including Copyleaks and Winston AI, disagree.

Nine machines, the same pages, no agreement. That's the least-repeated fact in this whole story, and the only one that actually says anything about what a detector proves.

Four days, and a book pulled from a prize list

The account had 92 followers. Its post has racked up more than three million views. That same Monday, the novel won the Prix du Roman Fnac.

On Wednesday the 23rd, Radio-Canada's Décrypteurs published an investigation that had nothing to do with AI: at least five opinion pieces the author signed in 2012 and 2013 lift passages from other writers. On Thursday, Éditions du Boréal suspended the book's promotional campaign. On Friday the 25th, the Académie Goncourt pulled it from its selection.

The author cut his French book tour short and went back to Quebec. By late September, his personal website had gone dark.

How you actually test a detector

A text-generation detector doesn't read, it compares. It scores how closely a passage resembles what it has learned to flag as model output. The number it spits out is a statistical bet, and the percentage makes it sound far more certain than it is.

The right way to stress-test one is to feed it texts whose answer you already know. The idea came from the accuser: Balance ton Claude had already run four of the author's 2014 pieces through Pangram, all coming back at 0% AI, before franceinfo picked up the same protocol.

franceinfo replicated it on two other pieces by the same man, a 2015 book and a 2014 article, both predating generative AI. Pangram called them "100% human." The newsroom also ran four novels from this fall's literary season, by Ananda Devi, Philippe Jaenada, Amélie Nothomb, and Lucie Rico: all human, scoring between 99 and 100%.

So the negative control was set up twice, by two parties who weren't working together. A tool that says "human" when the text is human, and "machine" on the disputed one, isn't proof, but it isn't nothing either.

A thousand novels, and not one in French

That leaves the author's own objection, and it isn't just a defense. Speaking to Libération, he floats a technical hypothesis: "I think the software is tuned to French literary traditions and classic authors. My traditions are Haitian and Caribbean. The imagery, the repetition, the rhythms, that's a living, spoken language, and it's mine."

In other words: a tool calibrated on one kind of prose might read an oral aesthetic as an anomaly. That's a claim about training data, which makes it testable. His Quebec publisher puts it differently in its own statement: the analyses rely on a tool "whose reliability is questionable and has not, to our knowledge, ever been established on French-language fiction."

We went and read the study everyone cites to back up Pangram's reliability. It's solid work, by Brian Jabarian and Alex Imas at the University of Chicago, from August 2025. Its corpus: 1,992 human-written passages across six genres. The literary section is a thousand extracts from English novels published before 2000, pulled from the public domain on Project Gutenberg. On that genre, Pangram's false-positive rate is zero.

Zero out of a thousand extracts is excellent. But it's not a result on French, let alone a contemporary novel: none of the six datasets is in French.

What the maker publishes, and what it keeps to itself

Pangram states a false-positive rate of roughly 1 in 10,000, or 0.01%, and 0.003% for books. The company even breaks that number down by genre, and by language where it draws a distinction: English, Spanish, Japanese, Arabic. French isn't on the list, and no line covers novels.

One number stands out: poetry, at 0.05%, five times the stated average. That's the genre closest to chanted, spoken language, with its repetitions and refrains. Pangram itself acknowledges its tool performs worse on poems, recipes, and short texts, wherever the form repeats itself.

A Stanford study, in 2023, showed that detectors of that generation classified text by authors whose native language isn't English as AI-generated more often. Pangram didn't exist yet at the time, and it does answer on that point — leaving it out would make this piece dishonest. Its February 2024 technical report, signed by its two founders Bradley Emi and Max Spero, claims a ten-domain benchmark that explicitly includes "creative writing" and "books," and states that the classifier is not biased against non-native English speakers. That's the rebuttal to the "never measured on fiction" charge. It's published in English, by the company itself.

Three false-positive rates, and only one measured from outside

The 0.01% on the website isn't the number the company publishes in its latest report. The "Pangram 4 Technical Report," posted to arXiv in late July 2026 and co-signed by both founders, puts the false-positive rate at 0.0041% — roughly 1 in 24,300 — with an AUROC of 0.9916 and a 0.3396% false-negative rate. Two numbers, a factor of two and a half between them, and the same source: the company.

The only large-scale independent measurement on French text came out on September 29, from Next, a French tech outlet. It pushed 460 human-written articles through Pangram: 450 came back fully human, 9 mostly human with some passages flagged, and one single false positive. That's 1 in 460, about 0.2%. On 350 AI-generated texts, the tool caught roughly 95%, leaving something like 5% false negatives.

Next states its limit before giving its result, and the limit matters: "Keep one important thing in mind about our analysis: it rests on a corpus of technical articles." So it's no more a measurement on fiction than the others. But it's the only one that doesn't come from the vendor, and it runs twenty times higher than the rate on the vendor's site, fifty times higher than the one in its report.

Twenty times 1 in 10,000 is still 1 in 460: the order of magnitude isn't damning. It just says that between a vendor's number and an outside check, there's the usual gap. And that what's riding on it, here, is a book pulled from a prize list.

Two accusations, two different kinds of proof

This is where precision matters, because the Académie Goncourt wasn't precise. Its statement lumps together, in one sentence, the "plagiarism its author has committed over time and that has just come to light," and the claim that the novel is, "in all likelihood, very largely the product of artificial intelligence."

These two accusations don't get proven the same way. Plagiarism is established by placing two texts side by side: Radio-Canada tracked down the source op-ed, published five days before the author's own piece, in a Philippine newspaper, and any reader can check it themselves. The author hasn't responded on that front.

The AI accusation rests on a score, and eight other tools run on the same pages produced conflicting results. Folding both into one sentence lends the second the solidity of the first.

The threshold is a choice, not a property

Jabarian and Imas make a point nobody has quoted in this whole affair. The threshold at which you declare a text artificial isn't a property of the software, it's a trade-off between the cost of letting a generated text slip through and the "risk of unjustified accusations." When the second cost looms larger, you raise the threshold.

A jury or a publisher ends up choosing that threshold, whether they realize it or not. Grasset, the author's French publisher, draws its own conclusion: "These tools that claim to detect AI are themselves AIs prone to hallucinating that the Bible was written artificially." The house also raises a question the public record barely touches: "Is it because Thélyson is Black and Haitian?"

The objection fails on the author, and holds on the instrument

The author's hypothesis has to be cut in two, because it doesn't hold up equally on both sides.

On him, it fails. The right control for his three-beat rhythm is his own pre-AI prose, and that got submitted twice, by the accuser and then by franceinfo. It came back human both times. His style doesn't trip Pangram.

A hostile reader lands in the same place without a detector. Manuel Cervera-Marzal, writing in Hors-Série on September 23, didn't like the novel and had the same suspicion a third of the way in: "The three-beat rhythms. The impeccably balanced antitheses." He then went and read the author's blog, where he found the same anaphora and the same ternary structures in columns from 2008 to 2014. He notes himself that he can no longer lay hands on those texts, "because his blog is now unreachable." His reading corroborates; no third party can redo it.

On the instrument, though, the objection holds — and in better form than the author gave it. A Carnegie Mellon team published a paper in May 2026 whose title is the finding: Base Models Look Human To AI Detectors. Evaluated by GPTZero and Pangram, text from base models — the ones that never went through instruction tuning — reads as overwhelmingly human, while text from their instruction-tuned counterparts gets flagged. We read the abstract, not the full paper. Its conclusion: "current detectors are tracking artifacts of instruction tuning and local context more than any invariant notion of machine-generated text."

So a detector isn't measuring "a machine wrote this." It's measuring "this sounds like the register of an instruction-tuned assistant." A human who writes in balanced antitheses sits in the neighborhood of that register, through no fault of their own.

Finding out more would take running the same tool on pages by Jacques Roumain, Jacques Stephen Alexis, or Aimé Césaire, the writers the author claims as his lineage, all of them predating generative AI. Nobody has done it. We haven't either, and we won't pretend to know what it would show.

Sébastien Gavois, the Next journalist who just ran the 460-article test, was asked in his comments section exactly that question: does a good detector return "100% human" on a literary style that's new or simply different? His answer: "I'd be careful about extrapolating… I don't know. That said, Pangram detects very well on Next articles, whatever the subject." He adds that he doesn't know how these tools would behave faced with a style or a vocabulary completely unlike their training data.

The person best placed to run that half-hour of checking declines to extrapolate. That's sturdier than "nobody has done it."