No, thank you, Pangram/Substack.

AI detectors don’t work.

There are two emerging approaches to fighting AI slop. One is with AI detectors and the other is with human firewalls.

I saw that Substack chose the AI detector approach. There is an obvious problem with their choice and then a bigger, corruptive problem.

The obvious problem is that AI detectors don’t work. There are already tons of examples demonstrating false positives and false negatives.

These failures in their AI detector affect only the good actors. The bad actors have already run their AI-generated text through “humanizers” and will always be able to defeat AI detectors. So the obvious problem is that the AI detector is an unnecessary annoyance that will smear real writers while doing nothing to slow down slop lords.

It gets WAY WORSE. The corruptive problem is that Substack is publicly asking writers to change their writing to please their AI detector:

Run Pangram on your drafts prior to publication

I like Jon’s response to that:

Why would I ever need to run Pangram on something I wrote when I know I wrote it?

If you give in to this Substack system, you’re changing your own writing to make it less human just because the AI detector doesn’t actually know any better.

There is no court of appeals. So if Substack’s AI says your writing isn’t human, then you are stuck between either keeping your authentic voice while Substack smears your reputation or making the AI-suggested changes.

I keep saying Substack, but they are relying on a vendor, Pangram. We know Pangram and have beef with them. I’ve seen the founder publicly call out a Medium writer for being 100% AI generated, that I know for a fact wasn’t. That means that not only is the Pangram tool broken, but even their founder can’t tell the difference between human and AI.

Mostly, though, Pangram’s business model has a built-in conflict of interest. They run unsolicited analyses on prospective customers and then feed those analyses to unsuspecting press (sorry, Wired magazine, but you got played). Then Pangram simultaneously pitches its services to its target. The conflict is clear: they have every reason to hype up their results.

Was I annoyed when this happened to us? Yes, of course. We already knew we had slop. But Pangram stole the press framing and made it about volume rather than about recommendations. That serves their business, but it’s not actually all that important. Slop is a “if a tree falls in the forest, does anyone hear it?” situation. What matters is how often we recommend slop, and they don’t have any valid solution to that.

There is a better way: use human judgment. Honestly, the dumbest part of the AI era is a rush from technologists to put the human brain out of business. AI guys: do some user research! A lot of us like using our brains.

Someone in the Medium community coined the phrase “human firewall” and I think it’s the right framing.

Medium has been building a human firewall out of human curation systems that use human judgment because we see the whole AI slop problem differently than everyone else.

The core issue is who sees the slop, not whether it exists at all.

More importantly, the issue isn’t specifically about AI, it’s about truth.

An AI detector judges word choice, which turns out to be worthless, error-prone, and trivial to game.

A human firewall judges wisdom, logic, and truthfulness. Humans also aren’t perfect, but at least they are focused on something that matters.

So when a human judges AI slop, they are able to tell the difference between something wise that was AI-assisted (possibly because the writer is English as a second language) versus something that sounds human but makes no sense.

At the end of the day, readers want to read things that are good.

There are a million ways to build human curation systems. Ours is based on layers of human curators vouching for what to read.

• Direct follow. Look, if you follow someone, then you are the curator of your own experience, and we’re just going to trust that you know what you are doing.
• Reposting. This is a variation on direct follow. If you follow someone who reposts a story, then we are going to trust that you trust that person to vouch for stories.
• Publications. We now PAY editors in the community to serve as judges. Want to get recognized as a human? Go submit to a publication, plus get access to that audience.
• Boost. The most popular stories on Medium get a second look from our internal human curators.

Together, that makes up a human firewall. Every smart platform is going to end up here and we’re proud to have kicked off a curation era by being the first platform to actually pay curators. (Reddit, are you next?)

I’m not going to pretend this is perfect. I’m just saying it’s a good approach, and the other approach is bad.

Our approach falls down in a couple of places:

• Slop still exists on Medium, and you can find it if you look.
• Slop shows up in the Google index even when we tell Google not to include it. C’mon Google.
• There’s a pretty important edge case around following publications, which is that some publications are not very discerning. I don’t follow those publications and see no slop in my feed. But when I spot check readers who are seeing slop, it’s almost always that they are following a publication that is not keeping high standards. We should probably fix this, and it’s not itself a flaw in the human firewall approach.

Imperfections aside, this protects real writers and gives readers the thing they want: real writing.

There’s another emerging approach: provenance.

The fundamental problem with AI-generated text is not the AI part; it’s the truthfulness part. AI generated text masquerades as a human author but isn’t. But we are increasingly in a world where people do generate things with AI that they do want to share. This is another example of how the problem with AI generation isn’t strictly the AI, it’s the inability to discern who the identity of the speaker.

I’m nearly certain that Medium’s algorithms will start tracking provenance in terms of revision history. If you typed it yourself, that should get a little boost.

But our TK app is about to go even further. I just saw this show up in a staff variant and you’ll probably see it yourself in a few days. We are adding a subtle indicator to what was AI-generated within your post. It might look something like this:

You could go even further, letting readers examine what AI prompt led to that generation. But for now, UI affordances like this will help the good actors be clear communicators even if they are using AI occasionally.

Onward.