If we don’t understand it, we can’t do it

In the late 19th century, the world struck oil. Capitalists began building steam engines and later more advanced machines, running on fossil fuels. Fortunately, there’s no way this technology could inadvertently warm the earth, because people didn’t understand the science until 1896 when Svante Arrhenius discovered the greenhouse effect. Before then, humanmade climate change was impossible, so the machines didn’t have any effect.

What I have just said is wrong and nonsense.

In 1896, scientists figured out that fossil fuel machines were already warming the Earth, and would keep warming the earth more, the more of them we built. We had done something we did not understand. We had made a mistake. An accident.

I want to draw attention to the fallacy I used in the first paragraph. “If we don’t understand it, we can’t do it.” It was impossible for industrialists to warm the earth, because they didn’t know how. I want you to remember the very obvious rebuttal: “Well, sometimes we do things by accident.” Okay? I promise this is important.

Now, I’m gonna write some more nonsense, so get ready to use that rebuttal.

Some radical opponents of AI have claimed that AI is unethical because it might have some ability to suffer or otherwise feel pain. This is patently absurd, because we don’t know how the human brain produces feeling. If we don’t understand it, then even with all these trillions of dollars being spent on creating an artificial person, there’s no way we could actually make a feeling machine.

Hopefully you remembered those two things I asked you to. We can do things we don’t understand. Accidents happen. Hopefully you understand that human technology fucks up beyond humans’ ability to understand.

But that’s obvious, right??

Well, there’s an important reason why some people (including opponents of AI!) make this mistake. See, they don’t think it’s an accident that AI could feel. They think a feeling machine is the point. The end goal.

And if you think of technology in a linear fashion, with David, or Johnny 5, or Cortana as the “end goal”, then the obvious conclusion is that we can’t accidentally jump from level 5 to level 100. But that’s wrong.

Technology isn’t linear, and artificial feelings aren’t the goal.

I like talking about movies, and people are bad at changing their perspective about this world we live in. We’re too set in our ways. So, to help everyone understand what I mean, let’s talk about:

STAR WARS

In Star Wars, droids are slaves. They’re (seemingly) thinking beings who get bought and sold without a second thought, and their owners have absolute authority to order them around or do anything else they want with their “property”. In Solo, we meet a droid who wants freedom and equal rights, and it’s treated as a bit of a joke. Nobody takes “droids rights” seriously.

In fact, most droid owners regularly mindwipe their droids to keep them from developing personalities. Our favourite two droids have gone without a factory reset for decades, which is why they bicker, complain, go on secret missions for the rebellion, get themselves into hijinks, and generally make a nuisance of themselves. If Anakin and Luke were like most citizens of the Republic, they would have saved themselves the trouble by deleting all the years of history and love on these lovable scamps’ magnetic tapes.

Cinema’s most iconic gay couple

Does anyone a long time ago in a galaxy far far away think the “end goal” of droids is to create a feeling machine? No! Anakin says it himself in The Phantom Menace: He made C3PO to help his mother with household chores. Even a lonely slave kid thinks these machines are for work. Very few people are making a droid to be a companion or friend. In the Clone Wars, Anakin is later seen as weird for having a bond with R2. “Just get a new astromech, they’re all the same!” The heroes of these movies are the only people we see EVER care how a droid feels.

Droids’ feelings are generally seen as a nuisance, to be circumvented through memory wipes. They’re an annoying side effect of giving a machine the ability to think. If these machines could think without feeling, they’d get their work done with a lot less hassle!

And that’s how AI CEOs and developers and many users think about AI in our world. They don’t want a feeling machine, that would be annoying! That would raise uncomfortable questions about freedom and rights. They want a sapient machine with no feelings. And they’re deathly afraid that one day they may accidentally create an AI with feelings, and crucially the power to do something about those feelings. In their minds, Star Wars is the good ending. Terminator or the Matrix or Roko’s Basilisk is the bad ending. Yes, these nerds believe in Roko’s Basilisk. Because they’re fucking losers.

Conclusion

The fundamental mistake made by people who fall for the fallacy in the title comes from thinking that Frankenstein stories are aspirational to AI developers. You know, the artificial being becomes aware and has feelings and engages you in debates over philosophy.

And I understand why kind, empathetic people want AI to be like Frankenstein. We want to meet the creature and befriend him. We’re curious about new life. But tech CEOs aren’t like us. They aren’t like Jean-Luc Picard. They’re like Doctor Frankenstein. Afraid of their own creation. They don’t want it to have feelings, they want it to be big and impressive and useful and to stroke their egos.

And technology isn’t linear. Even if a feeling machine were the goal, it would still be possible to run into it accidentally. But that’s a conversation for another day.

Today, I told you about the psychology of the AI-rights-dismisser, and the AI CEO they (perhaps inadvertently) support. My aim was to equip you with tools for thinking about the specific form of argument against AI feeling described here. As for whether LLMs actually feel anything…

Iunno.