AI Detectors Can Score 97% and Still Be Wrong

A 97% contest score can mean “aced one practice test” — not proof of authorship. False positives, new models, and humanizers punch holes in it.

In a 2025 research contest, a detector built from simple writing stats scored about 97%. Point that same model at a different writing mix, and the score fell to about 67%.

Same tool. Same family of tricks. The writing changed — length, topics, how the AI samples were made — and the practice-test grade stopped traveling.

That drop is the whole story in miniature. The popular version of AI detection is paste the essay, read the percent, trust the red flag. I’d argue that version is wrong. An AI detector is closer to a smoke alarm than a courtroom: useful, and also capable of screaming when nothing is burning — or staying quiet when there is smoke.

A red flag means look closer. It does not mean proven guilty. Hold that line. Here’s why.

A few terms worth locking down

Wrong accusation (false positive) — the detector says AI wrote this, but a person did. In school, this is the scary mistake.

Missed AI (false negative) — the detector says a person wrote this, but AI did. Cheating slips through.

Practice-test score (in-domain) — tested on writing that looks like what the tool already saw while learning. New-world score (out-of-domain) is the opposite: different genres, lengths, topics.

One big accuracy number (F1) mixes “when it says AI, how often is it right?” with “of all the AI texts, how many did it catch?” Useful for ranking tools on one test. Bad as proof for your essay.

A humanizer rewrites AI text so a detector — or a reader — is more likely to say “human.” Ads often say “undetectable.” Strong detectors often still notice.

The core problem: a great score on the wrong test

In 2026, researchers (Pudasaini and coauthors, arXiv 2603.23146) built detectors from writing stats anyone can explain: paragraph count, word variety, how much a file zipper can shrink the text. Repetitive, predictable writing often compresses smaller — AI drafts can look “zip-friendly” that way. They plugged those stats into ordinary machine-learning models — the same broad family that powers many apps, not magic.

On one public contest (PAN-CLEF 2025), their best setup scored about 97% (F1 0.9734). On another (COLING 2025), about 80% (F1 0.8025). For this style of tool, those are near the top of the public boards.

Then they switched the tests on purpose. Train on Contest A, retest on Contest A: about 95–97. Train on A, test on Contest B: about 67–77. Train on B, retest on B: about 76–80. Train on B, test on A: about 78–83.

One clear drop: a common off-the-shelf model (XGBoost) went from about 97 on Contest A to about 67 on Contest B. It did not suddenly get stupid. The cheap tricks it learned stopped working.

They also used SHAP — a method that ranks which writing stats mattered most. On Contest A, it cared a lot about how mixed the grammar patterns were. On Contest B, it cared about paragraph count and how zip-able the text was. If the tool had learned a real “AI voice,” it should care about the same clues everywhere. It did not. It often learned quirks of that contest’s data — not a universal “AI wrote this” signal.

The 97% was real. It was also a practice-test grade, not a courtroom stamp.

Two kinds of mistakes

A high overall score can hide which mistake the tool makes. For schools, that difference is the whole story.

Even on Contest A — the easy matching test — one of their models still wrongly labeled about 6% of human text as AI (and missed about 2% of AI text).

Turnitin has said its document-level wrong-accusation rate is under 1% (for documents where it marks about 20% or more as AI-written). One percent of millions of student papers is still a large pile of people flagged for nothing.

Short text is worse. In one deep look at errors, many mistakes piled up on short passages. Texts with basically one paragraph often got pushed toward “AI.” AI text with many paragraphs looked more “human.” Layout and length pretended to be proof of authorship. They are not.

Smoke alarms that hate short rooms are still smoke alarms. They are not proof of arson.

New AI tools break old detectors

Contest A and Contest B were frozen in time. Chat tools were not.

The same researchers tested text from AI models released after the training data was locked: DeepSeek V3.2, GPT-5.2, and Gemini 3 Pro. Every sample was AI, so the only question was how often the detector missed it.

• GPT-5.2 — miss rate about 23.25% when the model was trained on Contest A; about 3.42% when trained on Contest B
• DeepSeek V3.2 — about 0.76% (trained on A) vs about 3.51% (trained on B)
• Gemini 3 Pro — about 0.43% (trained on A) vs about 18.40% (trained on B)

Same detector family. Three different AI writers. Totally different blind spots. Yesterday’s 97% does not travel free to tomorrow’s chat model.

Paid tools on one shared test

Pangram Labs sells a detector and publishes a technical report (arXiv 2402.14873). On their own test of 1,976 documents across 10 writing types and 8 AI models, they report about 99% accuracy and claim far fewer errors than DetectGPT and big commercial tools on that set. Treat the headline as the vendor’s own report until someone else repeats the test.

More useful: how rivals fail on the same papers. On that shared test, GPTZero tends to prefer “human” — it misses about 10% of AI text. Originality tends to prefer “AI” — it wrongly flags about 9% of human text. Pangram’s own report claims both error types stay low, with about 99% accuracy.

Teachers usually care more about not accusing the wrong student than about catching every AI paragraph. GPTZero leans that way — and still leaves a lot of AI unmarked. Originality leans the other way — so more real human work needs a person to clear it. Neither pattern is “broken” in the abstract. Both are choices about which mistake hurts less.

People who write English as a second language get hit hard by some tools. One famous study (Liang and coauthors, arXiv 2304.02819) showed commercial detectors flagging TOEFL practice essays as AI — average wrong-accusation rates above 60% on that 91-essay set. Pangram says it got 0% wrong accusations on the same set. That is the company’s own claim on a known hard set — check it, do not treat it as settled law.

Three ways detectors work (and why “make it sound human” often fails)

Many people think detectors only hunt for AI clichés. That is only part of it — and it is the part marketers sell hardest.

Simple “how surprising is this word?” tools watch for predictable, flat writing (low perplexity, low burstiness). A rewrite that makes the text bumpier can fool these weak tools.

Style-stat tools count things like paragraph breaks and how zip-able the text is. Breaking one long block into more paragraphs can fool those cues on one contest. The research above shows those cues break when the writing world changes.

Heavy trained tools (like Pangram) learn from huge piles of real human writing vs AI writing. The question is less “does this feel human?” and more “was this produced the way chat AIs produce text?” You can change the voice and still leave that fingerprint. A surface rewrite usually barely moves the score.

How a strong tool like Pangram learns

Why dig into training here? Because it explains why “make it sound human” often fails against this class of tool.

1. Collect lots of writing that is known to be human (mostly from before 2021, so AI did not sneak in).
2. For each human piece, ask an AI to write a twin — same topic, similar length, sometimes the same first line. Call that a mirror. Now the tool cannot cheat by learning “this topic means AI.”
3. After a while, most twins are too easy. So they hunt for hard cases: human texts the current tool wrongly calls AI. They make new AI twins of those, and train again. Repeat.

On a huge fresh test set the model had never seen during training, they say wrong accusations fell from about 2.29% to about 0.02% after that loop. The lesson they train for is: same story, different author process (person vs AI). A rewrite that is still written by an AI is still the AI process.

What happens when you try to “humanize” against a strong detector

A live test against Pangram’s API on 2026–07–23 checked that claim. Score near 1.0 means “almost certainly AI.” Score near 0 means “almost certainly human.” Real human sample text scored about 0.003. AI sample text sat near 0.994.

• Original AI text — 0.994
• One Claude rewrite — 0.95 (partly kept meaning; ~68% by automated check)
• Best of six style rewrites — 0.992 (~72% meaning kept)
• Translate to French, then back to English — 0.994 (~99% meaning kept — and still fully flagged)
• Four rounds of guided rewriting — 0.993 (meaning collapsed; ~43% kept)
• Replace about 70% of sentences with old public-domain human sentences — essay scored 0.035, news sample 0.101 (meaning diluted in both)

Read that list slowly. The rewrite that kept meaning best — translate and translate back — did not fool Pangram at all. The loop that made the writing feel most “human” wrecked the meaning and still scored 0.993. The only approach that got under 0.5 (called human) swapped in large amounts of unrelated human sentences. That changes the content. It is not a clean “same essay, now undetectable.”

Researchers who study these attacks talk about a three-way trap: hide from the detector, keep the meaning, and keep the cost low. You almost never win all three. One paper that frames it this way is TH-Bench (arXiv 2503.08708). The live numbers above are one corner of that trap.

Limits of that live test: samples were short; some sample texts labeled “human” still scored like AI, so those samples were poor baselines; nobody tried attacks that look inside Pangram’s private model. Treat this as one careful black-box story, not a law of nature.

Before you trust a detector

Treat it like any automated grading aid — same spirit as Five metrics for an agent eval pipeline:

1. Ask what the score was measured on. Writing like yours? Language like yours? AI tools your people actually use?
2. Ask for both mistake rates. “99% accurate” hides whether the tool wrongly accuses humans or quietly misses AI.
3. Try it on your own sample. Real student essays, support emails, blog posts — skip relying on Wikipedia-shaped text alone.
4. Re-check when AI models change. Yesterday’s detector can go blind on tomorrow’s chat tool.
5. “I ran a humanizer” is not a clean pass. Against a strong tool, a rewrite that keeps the meaning often still looks like AI. A sudden drop in AI score may mean the meaning was watered down.
6. For high stakes, keep a person in the loop. A flag is a reason to ask for drafts, notes, or a short talk — not an automatic verdict.

If you also use AI to score other AI writing, the same caution applies to LLM-as-a-judge: a number without a clear rubric and a look at failures will mislead you politely.

What to do with a red flag

Back to the opening: smoke alarm, not court.

A 97% contest score can mean “did well on this pile of test papers,” not “proved AI authorship.” A high AI score after a rewrite can still mean “an AI still wrote this,” even when the voice feels personal.

Before you punish anyone for a flag, run a small batch of known human writing from your world through the tool and count wrong accusations. For a wider view of scoring without worshipping one number, read Five metrics for an agent eval pipeline.

Sources

1. In-domain F1 ~97 / cross-domain drop / SHAP / cross-generator miss rates — Pudasaini et al. — Why AI-Generated Text Detection Fails (arXiv 2603.23146)
2. Pangram bench (1,976 docs); GPTZero / Originality error bias; TOEFL 0% claim; hard-negative training — Pangram Labs technical report (arXiv 2402.14873)
3. Commercial detectors biased against non-native English / TOEFL essays — Liang et al. (arXiv 2304.02819)
4. Turnitin document-level false-positive rate under 1% (with 20%+ AI threshold) — Turnitin on false positive rates
5. Humanizer trade-off: evade vs meaning vs cost — TH-Bench (arXiv 2503.08708)
6. Live Pangram API humanizer checks (2026–07–23) — author-run black-box harness against Pangram; not an independent third-party study

Originally published at bartlomiejkrupa.dev, where I write about Agentic Engineering · LLM Optimization · AI Cost Reduction.