What does the evidence actually say?
Largest independent review of fourteen tools, Turnitin included: every one came in under 80% accuracy — "neither accurate nor reliable," easily evaded by paraphrasing.
Benchmark of 6M+ passages across 12 tools: very few work at a false-flag rate under 1% — directly contradicting "99% accurate" marketing.
91 human-written TOEFL essays run through 7 tools: misread as AI on average. Native-speaker U.S. eighth-grade essays: misread far less — on average.
One vocabulary-enrichment prompt — same human authors, same essays — moved the average false-flag rate down. Proof the tool measures word complexity, not authorship.
OpenAI's own classifier, before they shut it down in July 2023: caught a fraction of AI text, wrongly flagged human text.
One tool (ZeroGPT) read the text of the 1776 Declaration of Independence — written long before AI existed — as AI.
Turnitin's own documentation: below 1% false flags at the document level — a figure it states for documents marked 20%+ AI. At the sentence level, its chief product officer put the rate near 4%.
What is the number actually measuring?
These tools never read for meaning, and they have no record of who sat at the keyboard. They do one statistical thing. They estimate how predictable your wording is — then label the predictable end of the range "AI." Two terms do all the work:
- Perplexity — how "surprised" a language model is by your next word. In plain words: common, expected word choices produce low perplexity; unusual, unexpected ones produce high. The tools read low perplexity as machine-like, because models are built to choose the likely word.
- Burstiness — how much your sentence length and rhythm vary. Human writing lurches between long and short. Very even, uniform sentences read as machine-like.
Put together: the instrument rewards uneven, unpredictable writing as "human," and flags clean, even, predictable writing as "AI." That is a statement about your word choices — not about whether a person produced them. It's why the studies above catch a careful TOEFL essay and the Declaration of Independence in the same net. Both are clean and predictable. Neither was written by a machine.
"this text was written by AI"
- reads as a statement about WHO wrote it
how PREDICTABLE the word choices are
- low perplexity = common, expected words
- low burstiness = uniform sentence rhythm
predictable writing ≠ machine writing
- plain, formal, or careful prose looks predictable
- a second-language writer's measured vocabulary looks predictable
- so does the 1776 Declaration of Independence
Why does clean, formal, or second-language writing get flagged?
Because predictable vocabulary is exactly what the tool penalizes. Write plainly, write formally, or write in a second language — reaching for the safe, correct word rather than the surprising one — and your text sits at the low-perplexity end. That's the end the tool calls "AI." This is not a glitch. It's the mechanism working as designed, on writing it was never calibrated for.
The Liang study is where you can watch the mechanism move. The same human essays were misread 61.22% of the time, on average. One vocabulary-enrichment prompt later, the average false-flag rate fell to 11.77%. Same authors. More "surprising" words. Fewer flags. Nothing about the authorship changed — only the predictability of the wording did. That swing is the cleanest demonstration that the number reacts to word complexity, not to who wrote it. The fuller mechanism for second-language writers — and whether a number this shaky meets the standard a court uses for expert evidence — is its own page → Accused while writing in a second language →.
How do you use these facts in your own case?
These numbers don't argue your paper for you. They establish one narrower thing: a number from an AI checker can't carry an accusation by itself. Put that on the table. Here is what each fact is good for.
- Against "the tool is 99% accurate." The two largest independent tests disagree — and neither belongs to a vendor. The fourteen-tool review put every tool under 80% and called them "neither accurate nor reliable." The RAID benchmark, six-million-plus passages, found very few work at a false-flag rate under 1%. The marketing claim and the peer-reviewed record point in opposite directions. Use it to answer the accuracy claim — not to prove your authorship.
- If you write in a second language, or simply write formally. Liang names your exact situation. 91 human TOEFL essays, misread 61.22% of the time on average. Native eighth-grade essays, 5.19% on average. And the same essays dropped from 61.22% to 11.77% after one vocabulary prompt. That swing, same authors, is direct evidence the tool reacts to vocabulary — not authorship. Don't re-argue the mechanism here → Accused while writing in a second language →.
- The maker couldn't make a reliable one. OpenAI's own classifier caught 26% of AI text and wrongly flagged 9% of human text before they withdrew it in July 2023. This isn't a competitor's critique. It's the company that builds the most-used model conceding the problem is unsolved — and pulling its tool.
- The instrument flags writing that pre-dates AI. One tool (ZeroGPT) read the text of the 1776 Declaration of Independence as 97.93% AI. A single, memorable demonstration that a high reading can attach to text no machine could possibly have produced — useful precisely because it needs no expertise to understand.
- Even the vendor caps its own claim. Turnitin's documentation puts its false-flag rate below 1% at the document level — a figure it states for documents marked 20%+ AI. At the sentence level, its chief product officer put the rate near 4%. And Turnitin's guidance says its output should not be the sole basis for an accusation. The strongest statement that the number is a signal, not a finding, comes from the company selling it.
What do these facts not do?
None of them, by itself, proves that your paper was written by you. Be clear-eyed about what you're holding. Every fact above is about the instrument: AI checkers, as a class, misread human writing too often for one number to stand alone. That distinction is the whole game, and it works in your favor. It moves the ground from "your paper is AI" to "this number can't carry that claim." That's exactly where you want to be standing. A hearing decides on a low bar — at most schools, just past 50% (some set a higher "clear and convincing" bar) — and that bar is far easier to pull back under when the only thing on the other side is a number the research has already discredited.
What it means in practice: pair these facts with the evidence that you wrote it — the independent record of your process — and let the statistics do the narrower job of disqualifying the number. What actually holds up, and how the real cases split on a flag-alone versus a flag-plus-something-independent → What evidence actually proves you wrote it → · The cases — coming soon.
The one line that matters
A tool's number is a signal, not a finding. Turnitin's own guidance says its output should not be the sole basis for an accusation. → The cases — coming soon
Where to next? → What evidence actually proves you wrote it · The cases: who won and why — coming soon · Accused while writing in a second language · What happens at the hearing
Information, not legal advice. Research and rules vary and are still evolving; a finding for one person does not predict yours. Read your school's handbook and talk to a qualified attorney or campus advisor. We don't judge — we help you track the records.