How Do AI Detectors Work? Perplexity, Burstiness and Classifiers

AI detectors work by measuring how predictable a piece of text is. Most of them run the text through a language model and check how likely each word was, given the words before it. Text that is consistently easy to predict looks machine-made. Text with more surprising, uneven word choices looks human. Many commercial tools add a trained classifier on top that has learned the difference from large sets of labelled examples.

That is the core idea. The details explain why detectors are useful, and also why they get things wrong.

The basic question a detector asks

A detector does not understand your argument, check your facts or know who you are. It looks at the surface of the text, the words and how they are arranged, and asks one statistical question: does this look more like what a language model would produce, or more like what people produce?

To answer that, detectors use a few kinds of signals. Most modern tools combine several.

Signal 1: Perplexity

Perplexity is a measure of surprise. Take a language model, feed it your text one word at a time, and at each step ask how likely the actual next word was.

Consider this sentence: "The cat sat on the ___." A model will predict "mat" or "floor" or "sofa" with high confidence. If the next word is "mat", perplexity is low. If it is "manuscript", perplexity is high.

Language models generate text by picking likely words. So their output, measured by a similar model, tends to have low perplexity all the way through. People pick words for many reasons: habit, humour, a specific memory, a slightly odd phrasing they like. That pushes perplexity up.

Some early detectors leaned heavily on this one number. The problem is that plenty of human writing is also predictable. A lab report, a legal summary or a five-paragraph essay written to a rubric can all score low on perplexity simply because the genre is formulaic.

Signal 2: Burstiness

Burstiness looks at how perplexity changes across the text. A person might write two plain sentences, then one with an unusual image, then a fragment. The predictability jumps around. AI output tends to hold a steady level from start to finish.

The same idea shows up in simpler measures, like how much sentence length varies. Human paragraphs often mix a 30-word sentence with a 4-word one. Model paragraphs tend to keep sentences in a narrow band.

GPTZero popularised the pairing of perplexity and burstiness when it launched in early 2023, and the terms have stuck. Its current models use more than those two measures, but they remain a useful way to understand what detectors are looking for.

Signal 3: Trained classifiers

The most common approach in commercial detectors today is a classifier: a machine learning model trained on a large set of texts labelled "human" and "AI". It learns whatever features separate the two groups, including ones no person would think to name.

Classifiers are often built on the same transformer technology as the chatbots themselves. They can be quite good on text that resembles their training data. They struggle with text that does not, for example:

• Writing from a newer model the classifier has not seen
• Languages or dialects that were rare in the training set
• Genres that were under-represented, like poetry or technical documentation
• Mixed texts, where a person wrote some parts and AI wrote others

This is why vendors retrain their detectors regularly, and why the same text can get a different score from the same tool a few months apart.

Signal 4: Watermarks

Watermarking is a different approach. Instead of guessing after the fact, the company that runs the model nudges its word choices in a hidden, statistical pattern while generating. A matching detector can then check for that pattern.

All three big chatbots now do this for at least some of their text. Google has marked Gemini app text with SynthID since 2024 (how SynthID works). Anthropic started watermarking Claude's text in August 2026 (what Anthropic has said), and OpenAI is adding its textGrain watermark to ChatGPT text in the EU from October 2026 (where it applies). Watermarks can be fairly reliable when they are present. The catch is that only the company that made the model holds the key to check for them, plenty of text in the wild carries no watermark at all, and heavy editing or translation weakens the signal.

So for now, almost every detector you will meet, including the ones used in schools, relies on the statistical methods above rather than watermarks.

How the pieces become a score

Different tools report results differently:

• A single percentage, often described as the share of the text likely to be AI-generated.
• Sentence or paragraph highlighting, showing which parts drove the score.
• Categories, such as fully AI, AI-refined or human-written. The RealWrite AI Detector uses four categories so you can see mixed authorship rather than a single verdict.

Whatever the format, the number is a probability estimate, not a finding of fact. A score of 80% does not mean 80% of the words were typed by a machine. It means the model is fairly confident the text resembles its AI examples.

Why AI detectors get it wrong

Once you know how detectors work, their failure modes are easy to predict.

Predictable genres look like AI

Formal, structured writing has low perplexity by design. Abstracts, cover letters, product descriptions and standardised essays all follow conventions that make the next word easy to guess. Human writers in these genres get flagged more often.

Non-native English writers get flagged more

People writing in a second language often use a smaller, safer vocabulary and more regular sentence patterns. Those are exactly the features detectors associate with AI. A 2023 Stanford study led by Weixin Liang, published in the journal Patterns, found that several popular detectors misclassified a large share of TOEFL essays by non-native speakers as AI-generated, while scoring essays by US students far more accurately. We cover this in depth in AI detector false positives.

Short texts do not give enough signal

Statistical measures need data. A 100-word paragraph simply does not contain enough words for a reliable estimate. Many tools set a minimum length for this reason, and results near that minimum are the least stable.

Editing blurs the line

Real writing is increasingly mixed. Someone drafts with AI and rewrites half of it. Someone writes by hand and uses a grammar tool to polish it. Detectors were trained mostly on clean examples of each, so mixed texts land in the uncertain middle.

Translation muddies the signal

Text written in one language and machine-translated into another picks up some of the regularity of the translation model. Text written by AI and then translated loses some of its original patterns. Either way, the detector is no longer looking at the writing it was trained to judge, and results become harder to trust.

Grammar tools can push text toward "AI"

Aggressive grammar and style suggestions tend to make prose smoother and more uniform. Accept enough of them and your original writing can start to look statistically similar to model output.

What detectors are good for

None of this means detectors are useless. Used well, they are a quick second opinion. A high score on a draft is a good prompt to reread it: are the sentences all the same length? Are there stock phrases in every paragraph? Is there anything specific in it?

That is how we suggest using any detector, including ours. Check a draft before you submit it, look at which parts score high, and revise those parts so they sound more like you. The how it works page shows that loop in RealWrite.

What detectors should not be is the only evidence in a decision about someone's work. Even the companies that build them say so. Turnitin, for example, tells instructors that its AI score should not be used as the sole basis for action against a student. Our Turnitin explainer goes into what its report shows.

The short version

• Detectors measure predictability, not meaning.
• Perplexity is word-level surprise. Burstiness is how much that surprise varies.
• Most commercial tools use trained classifiers on top, which need constant retraining.
• Watermarks are promising but rare in practice.
• Formal genres, non-native writers, short texts and edited texts are where detectors misfire most.

If you want to see how your own writing scores, you can run a free check with RealWrite. And if you are weighing up tools, our comparison of the best AI detectors looks at the main options side by side.