What's Actually Happening Inside an AI Detector

The Mechanics Behind the Score

Somewhere on this blog is a post about the gut-feel signs that make a piece of writing feel machine-made — the missing personal story, the tone that's a little too smooth. This one is different. This is the version for people who want to know what an AI detector is actually calculating when it spits out a percentage.

Underneath the friendly interface, most detectors are running a handful of statistical checks against your text and comparing the results to patterns collected from millions of human and machine-written samples. None of these checks read for meaning. They're measuring shape, rhythm, and predictability. Two of the main ones are called perplexity and burstiness, and they're worth understanding on their own terms.

Perplexity: How Predictable Is Each Word Choice

Perplexity is a measure of how surprised a language model is by the words that come next in a sentence. Feed a model the phrase 'The sun rises in the' and it will guess 'east' with high confidence — that's a low-perplexity, highly predictable continuation. Language models are trained to be excellent at exactly this kind of prediction, so text they generate tends to stick to the word choices they'd have picked themselves. Low perplexity, in other words.

Human writing wanders more. A person might write 'the sun rises in the east, same as it did the morning my father left,' pulling in a detail no model would have predicted. That unpredictability shows up as higher perplexity. Detectors run your whole text through a model, tally up how surprised it is sentence by sentence, and use the average as one signal — low average surprise nudges the score toward 'machine,' higher average surprise nudges it toward 'human.'

Burstiness: The Up-and-Down of Sentence Rhythm

Burstiness looks at a different property: variation. Read a paragraph written by a person and you'll usually notice sentences of wildly different lengths sitting next to each other — a short one for punch, then a long one that circles through a few clauses before landing. That unevenness, that burstiness, is a natural side effect of how people think and revise as they write.

Generated text tends to smooth this out. Sentence after sentence lands in a similar length range with similar internal complexity, because the model is optimizing for consistency and fluency rather than for the kind of stylistic contrast a person reaches for on instinct. A detector measures the spread of sentence lengths and structures across your document — a flat, low-variance line reads as more machine-like; a jagged, uneven one reads as more human.

Four Detection Methods Side by Side

Pattern Analysis: The Habits Models Can't Quite Shake

Beyond the statistical scoring, many detectors also scan for structural habits that generative models fall into over and over. That includes a fondness for certain transition words, a tendency to open paragraphs the same way, a habit of organizing ideas into tidy three-part lists, and a general evenness in how ideas are structured from one paragraph to the next. Any one of these habits shows up in human writing too — the difference detectors are hunting for is how consistently they repeat across an entire piece.

Watermarking: A Signature Planted at Generation Time

Watermarking works differently from the other three methods, because it doesn't analyze the finished text from scratch — it relies on a signal planted by the model itself while the text was being generated. Some AI providers have experimented with subtly biasing which words their models pick, in a pattern invisible to a reader but detectable with the right key. The catch is that this only works if the text came from a model that actually applies a watermark, and if that text hasn't been edited enough to break the pattern. A lot of AI-generated text out there was never watermarked to begin with, which is why detectors mostly still lean on perplexity, burstiness, and pattern analysis.

A Percentage Is a Probability, Not a Ruling

Every detector score is an estimate of likelihood, not a determination of fact. A '92% AI' label doesn't mean the tool checked and confirmed anything — it means the text scored more like the machine-written samples in its comparison set than the human-written ones did, on the specific measurements described above. That estimate can be wrong in both directions. Genuinely human writing that happens to be plain, consistent, and low on surprising word choices can score as machine-made. Edited or paraphrased AI text can score as human.

That's also why serious instructors and platforms are increasingly cautious about treating a detector score as proof of anything on its own, and why it's worth reading any single number with some distance rather than as a verdict.

If a Score Doesn't Sit Right

• Ask what specific tool produced it, and whether the platform allows for a second opinion
• Keep your drafts, outlines, and revision history so you can show your actual process
• Remember that a score reflects statistical similarity, not certainty
• Read the detector's own documentation on accuracy — most publish caveats about their limits somewhere

Writing That Reads Like a Person Wrote It

None of this is a reason to write worse on purpose. If anything, understanding perplexity and burstiness explains why writing that naturally varies in rhythm, takes the occasional unexpected turn, and doesn't lean on the same three sentence structures reads as more human, whether or not a detector ever looks at it. That's the same instinct behind our Humanizer tool at AI Humanizer Lab — it's built to loosen up flat, uniform phrasing into text with the kind of natural variation people actually write with, rather than trying to trick any particular scoring method.