The Difference Between Evidence and the Appearance of Evidence

The Difference Between Evidence and the Appearance of Evidence

I’ve lived with severe allergies for as long as I can remember. Growing up, hospitals, specialists, blood tests, and prescription medications became a normal part of life. Ironically, that experience didn’t make me trust modern medicine blindly. It taught me that good science is often comfortable saying, “We don’t know yet.”

That’s probably why I’ve always been skeptical of many alternative medicine claims. Not because they’re alternative, but because they often seem capable of explaining everything from head to toe. Allergies, gut health, skin problems, fatigue, immunity, sleep, digestion – and sometimes, even cancer. We’re often reminded that correlation doesn’t always imply causation. Fair enough. But some claims seem to not even be correlated to begin with.. Before asking whether one thing causes another, shouldn’t we first ask whether the two are meaningfully connected in the first place?

Still, skepticism shouldn’t become dogma. So when my family encouraged me to visit a herbal practitioner they trusted, I decided to keep an open mind.

The consultation itself was inexpensive. A small blood sample was taken and examined under a microscope while the practitioner explained what he believed he was seeing. Then came the recommendations: several herbal products costing millions of rupiah.

Walking out, I couldn’t stop thinking about Gillette.

Gillette is famous for a simple business model often called the razor-and-blades model. You sell the razor relatively cheaply because the real opportunity comes afterward: the customer has to keep coming back for replacement blades. The first purchase gets you into the system. The recurring purchases are where the business becomes interesting.

And suddenly, the consultation started to look a little different.

The consultation was the entry point. The products were the destination.

Whatever problem you came in with, the road seemed to lead to the same place: one of the products he sold. That doesn’t prove that the diagnosis was invented or that the treatment couldn’t work. But it does create an obvious incentive. If the business depends on selling the solution, then the most valuable outcome of the consultation isn’t necessarily figuring out what is wrong. It is getting you to the product.

You just have to make the road there sound convincing.

That became particularly interesting when I asked about one of the recommendations: a probiotic spray for my skin.

I asked a very ordinary question: why probiotics?

I wasn’t trying to challenge him. I genuinely wanted to understand the mechanism. Maybe there is good evidence behind topical probiotics. Maybe there are studies I haven’t read. Maybe years of experience had convinced him that they worked. I was simply asking him to connect my condition to the treatment he was recommending.

Instead, he became defensive. He questioned whether I even understood what probiotics were, then argued that powdered probiotics couldn’t possibly work because probiotics are “alive.” The explanation sounded more like the kind of simplified idea people absorb from television advertising – think of the familiar messaging around probiotic drinks such as Yakult – rather than an explanation of the medical mechanism.

But the problem wasn’t whether his statement about probiotics was right or wrong.

He simply didn’t answer the question.

I wasn’t asking whether probiotics were alive. I was asking why a probiotic spray was relevant to my condition.

And that was a remarkably simple question to answer if there was a clear mechanism behind the recommendation.

Instead, the question itself seemed to become the problem.

That made me wonder whether the explanation was being avoided because answering it honestly would complicate the path toward the product. Maybe there was a legitimate explanation he couldn’t communicate well. Maybe he simply didn’t know. Or maybe the product came first and the explanation had to be built around it.

I can’t know which one it was.

But the incentive structure made the distinction worth noticing.

Later that evening, I found myself thinking about The Prestige.

A film about two rival magicians obsessed with creating the perfect illusion. One of the things the film illustrates beautifully is how a magician can control an audience’s attention. Sometimes the distraction is elaborate. Sometimes it’s as simple as a beautiful assistant walking across the stage.

The assistant isn’t the trick.

She’s there to make you look somewhere else while the trick happens.

That’s what makes the idea so powerful. The magician doesn’t necessarily have to show you something false. He only has to make something else more interesting than the thing you should actually be watching.

The microscope suddenly made a lot more sense to me.

Not because microscopes are extraordinary. Most of us first used one in a high school biology class. But put one on a consultation desk, beside someone speaking confidently about your blood, and it immediately gives the room an aura of science. You see the microscope. You see the image. You hear explanations about what’s happening inside your body.

And while you’re looking at all of that, you may forget to ask the more basic question:

Do the conclusions actually follow from what I’m being shown?

The microscope wasn’t fake. I’m not suggesting the images were fabricated. That’s not the point.

The point is that something can look scientific without necessarily providing scientific evidence for the conclusion being presented.

Add a few broadly applicable observations – you’re stressed, you worry too much, your body is inflamed – and it’s difficult not to think about the Barnum Effect, our tendency to interpret vague statements as uniquely personal. None of these things individually prove deception. Together, however, they can create something remarkably persuasive.

The appearance of evidence.

Then there is another question: who actually goes there?

If a practitioner is located somewhere remote, perhaps the distance itself becomes part of the credibility. If someone is willing to travel for hours to see you, it is easy to think that you must be worth the journey. The effort creates commitment. The distance creates exclusivity.

And the people making that journey probably aren’t a random sample.

People don’t usually travel that far when their symptoms are mild. They go when they’ve tried other things, when they’re frustrated, when they’re desperate, when they feel they’ve run out of options. They arrive at precisely the moment when they’re most eager for something to work.

Then imagine that person getting better.

Maybe the treatment worked. But symptoms also fluctuate. Someone who arrives at their worst may naturally move closer to their average afterward. That’s regression to the mean.

And what about everyone who didn’t get better?

They don’t usually become the story.

The person who improved tells everyone, “I went there and it worked.”

The person who didn’t improve simply goes somewhere else.

Bill Gates and Mark Zuckerberg both dropped out of college and became wildly successful. But does that mean dropping out was the reason they succeeded? I’m convinced the statistics would tell us otherwise.

We see the successes. We rarely see the denominator.

The survivorship bias.

None of this means the herbs don’t work. Maybe they do. I honestly don’t know. That’s a question for properly designed clinical research, not a single consultation or a collection of testimonials.

And there is nothing inherently wrong with trying – with making an effort to get better. But effort and direction aren’t the same thing. If you’re trying to reach the top of a mountain, swimming into the ocean is certainly an effort. It just isn’t necessarily an effective one.

What this experience changed for me wasn’t my opinion on herbal medicine. It was my awareness of how easily evidence can be assembled out of things that aren’t evidence themselves: a microscope, confident language, a remote location, an expensive product, a testimonial, or simply the fact that someone was willing to travel a long way to get there.

None of these things necessarily mean anyone is lying.

But none of them, by themselves, tell us whether the underlying claim is true either.

And perhaps that’s the distinction worth remembering.

Evidence tells us something about the claim. The appearance of evidence tells us something about how the claim is being presented.

The two can look remarkably similar.

So the next time something feels convincing — whether it’s in politics, the news, scientific research, statistics, economics, medicine, nutrition, investing, or AI — it may be worth asking what is actually making it feel convincing. Is it the evidence itself, or the numbers, authority, expertise, and narrative surrounding it?

Maybe the question isn’t simply whether something looks scientific.

Maybe it’s worth asking:

What, exactly, is the evidence – and what is merely the appearance of it?