Artificial Intelligence News

I Ran Shakespeare Through an AI Checker

The results were a comedy of errors

The Comedy of Errors after being put through an AI checker (Wiki Comms)

A few weeks ago, I wrote a piece explaining why I run my articles through AI checkers before publishing them. Partly out of curiosity, partly out of fun, to see if my writing has started resembling something a machine might write.

On average, my articles come back with an AI score somewhere between 10% and 20%. It’s not because I use AI, but because parts of my writing are statistically similar to the sort of text language models often produce.

That doesn’t necessarily mean my writing is bad. It just means it’s smooth, grammatically consistent, and, in places, a little predictable. Exactly the sort of writing I was taught at school: clear, plain English, without too many surprises.

So what about Shakespeare? A writer famous for inventing words, bending the English language to his will, and producing some of the most original writing ever published. Surely even the harshest AI detector would return 0%.

You would think so, wouldn’t you?

So I checked.

I fed Hamlet, The Comedy of Errors and A Midsummer Night’s Dream through three different AI detectors: Turnitin, ZeroGPT and Originality.ai.

These were the results.

Hamlet: 12%, 23%, 15% (average: 17%)

The Comedy of Errors: 13%, 12%, 18% (average: 14%)

A Midsummer Night’s Dream: 11%, 23%, 19% (average: 18%)

Surprising?

Not really. The thing we have to remember is that AI detectors don’t actually detect AI. They don’t work like a metal detector finding a coin buried in the sand.

Instead, they estimate the probability that a piece of writing resembles text generated by a language model. They look for statistical patterns such as consistency of style, predictable word choices, repeated sentence structures, and grammatical regularity.

Shakespeare contains plenty of these.

That doesn’t make Shakespeare artificial — of course not! It simply means some of his writing shares statistical characteristics with text that modern language models can also produce.

Imagine two cooks, one human, one robot, making exactly the same meal. A taste test might tell you the dishes are very similar, but it couldn’t tell you who cooked which one. AI detectors have the same problem. They only see the finished product, not how it was created.

Let’s do another thought experiment.

Suppose I asked an AI program to write a play about a Danish king in Elizabethan English. Many AI detectors would probably give that text a very high AI score. Not because it resembles Shakespeare, but because it resembles the sort of output a language model might produce.

Real Shakespeare plays receive AI scores around 20%. Not because the detector believes he used ChatGPT four centuries ago, but because the statistical patterns in his writing overlap with patterns language models have learned from enormous collections of text, including Shakespeare himself.

This is an important distinction.

The model doesn’t know Hamlet was written around 1600 or who William Shakespeare was in the historical sense. It simply analyses the words on the page. Then compares their statistical characteristics with those found in AI-generated writing.

This is why AI detectors should be treated with caution, particularly by editors.

If I see a score above 60%, I’m certainly more suspicious than if I see 15%. But even that isn’t proof. Professional journalists, technical writers, and experienced authors often produce polished, consistent prose because they’re good writers.

So what is the best detector?

I edit *Pitfall*, and I reject plenty of AI-generated submissions. Most of the time, it isn’t a detector that alerts me. It’s instinct. Remember that?

After reading thousands of submissions, you begin to recognise the patterns. Titles that sound as though you’ve read them before. Endless explanatory sentences separated by colons and em-dashes. Perfect prose that says nothing. Articles that read less like someone sharing an idea and more like the instruction leaflet that comes with a blood pressure monitor.

Of course, none of those things proves a piece was written by AI. Humans use colons and em dashes, too. And as I said above, writing sometimes has to be bland to get to the point.

Which is why, useful though they are, AI and AI checkers remind us that not every problem can be solved by technology. Especially when the subject is the written word.

[embed]Fool AI-Checkers. Make Your Writing Completely Incomprehensible Why do AI checkers flag non-AI writing and how can we stop itmedium.com

Thanks to Shamim Rajani for the idea.