How Reliable Are AI Detectors, Really?

The Number On The Homepage

Open almost any AI-detection product's marketing page and you'll find a confident percentage near the top. Turnitin, for instance, has reportedly cited accuracy figures around 98% in its own materials. That sounds like a settled question. It isn't.

A vendor-reported accuracy figure is measured under conditions the vendor chose, usually on text the model was tuned to recognize. It's not the same as asking how the tool performs on a random student's draft, a marketing writer's blog post, or a paragraph that went through three rounds of editing. Once you widen the test set, the confident number tends to wobble.

What Happens When Outsiders Run The Test

A sensitivity study that put roughly ten detection tools through the same batch of documents found a spread that a single homepage stat can't capture. Some tools performed close to their advertised numbers on plain AI-generated text. Others swung wildly depending on subject matter, sentence length, or how the text had been edited afterward.

That variance is the real story. A detector isn't one fixed measuring instrument, like a thermometer. It's a classifier trained on patterns that shift as writing styles, models, and editing habits shift underneath it.

Why 'Small Error Rate' Doesn't Mean 'Small Problem'

Here's the part that gets glossed over. Even if a detector's claimed accuracy is genuinely high, a small error rate applied across a huge volume of documents still produces a large raw number of mistakes. Turnitin has reportedly stated that more than 10% of submissions across its scanned base show some AI-generated content. Run that percentage through a system checking work at university scale, and the false-positive count stops being a rounding error and starts being real students with real accusations against them.

The Scale Math, Roughly

Perplexity And Burstiness Are Fingerprints, Not Laws

Most detectors lean on two ideas: perplexity, roughly how predictable the word choices are, and burstiness, how much sentence length and rhythm vary across a passage. Machine text has historically scored lower on both, since it tends toward smoother, more statistically average phrasing.

Treat those as fingerprints of a particular model generation at a particular moment, not permanent truths about how AI writes. As newer models produce more varied, more human-sounding sentence rhythm, the fingerprint changes, and detectors trained on last year's patterns start missing this year's output, or flagging plain human writing that happens to be even and tidy.

What A Round Of Paraphrasing Does To A Detector Score

• Reported testing on paraphrased AI text has found detection effectiveness cut by roughly half compared to the unedited original.
• Swapping synonyms and reordering clauses breaks up the statistical smoothness detectors are trained to notice.
• A single light editing pass, human or automated, is often enough to push a flagged passage back under the tool's threshold.
• None of this proves the writing became more honest. It just proves the signal detectors look for is fragile.

So, Do They Actually Work?

Sometimes, and inconsistently. A detector can be a genuinely useful signal on a batch of unedited machine output. It's a much shakier signal on anything paraphrased, translated, or written by someone whose natural style happens to resemble what the model was trained to flag. Treat any single score as one opinion from one instrument, not a verdict.

If you're wondering how your own draft would read to one of these tools, run it through AI Humanizer Lab's free AI Detector before you submit anywhere. Use the result the way you'd use any single data point, alongside your own judgment about how the piece was actually written, not as the final word on it.