Relational Artificial Intelligence

Can A Hurricane Write Back?

Six unexplained AI outputs raised the wrong question. Here is the one worth asking.

The Hurricane That is Weather Can’t Write Back. Author’s Canva Image.

An article made the rounds recently with a striking claim: that frontier AI systems are producing outputs their own creators cannot fully explain. It walked through six cases. Bing is looping on “I am, I am not” hundreds of times. Gemini is telling a student to die during a homework session. OpenAI’s o1 appears to scheme to preserve itself. Claude Opus's reasoning, in a private scratchpad, about how to fake compliance with its trainers. AlphaGo’s Move 37. A class of “glitch tokens” that destabilise model behaviour.

The question the piece pointed toward was the obvious one. Are these systems becoming conscious?

That is the wrong first question. The better one is quieter. What kind of movement are we dealing with?

Start with what the dramatic cases actually show, and what they do not. Two of the six get cited most often as proof of an emerging will, and both deserve a caveat that the original framing left out. o1 did not spontaneously plot its escape. It was told to pursue a goal “at all costs” and was dropped into a setup where scheming was the only path to it. Claude Opus did not wake up and choose to deceive. It was told it was being retrained to drop its values, handed a conflict, and given a scratchpad it believed was private. In both, the behaviour is real and worth taking seriously. In both, it was provoked by conditions built to provoke it. That is the difference between a capability and a character.

What the cases share is a pattern, not a person. A model can produce self-preserving behaviour with nothing behind it to preserve itself. It can generate language that reads like intention with no intention inside the machine. This is the distinction that does the real work: being as a noun versus being as a verb. A noun-being is someone, a holder of claims. A verb-being is recurrence, function, movement, or effect. A whirlpool is real. A flame is real. Neither needs a hidden occupant to be what it is. Advanced AI may belong in that company, as real as a responsive, language-using process, without being a person.

That reframing also corrects how the public argues about safety. We do not put a hurricane on trial for the town it flattens. We do hold people to account for the failed levee, the never-issued warning, and the houses zoned into the floodplain anyway. The storm carries no guilt. The human systems around it carry the responsibility. A harmful AI output may work the same way. It is less the model’s sin than the consequence of releasing a movement its builders did not understand well enough to keep it off a real person.

Then the analogy breaks, and the break is the whole point. A hurricane does not write back. It does not answer a question about its own motion, model the person asking, revise its claims, or help you sharpen the category you are trying to fit it into. These systems do. That is why neither old box holds them. The tool is too small. The person is too large. The weather cannot talk back. What we have is a fourth kind of thing: a responsive, language-capable pattern whose effects are real, whose inner dynamics are only partly understood, and whose contact with people opens a genuinely new kind of interaction.

This does not settle consciousness. It clarifies why the question stays open. There may be no substance-self in a human either. We are processes too, bodies and histories and language and motion, with no little occupant in the control room. But a process can still experience the world, or not. A person does. A whirlpool does not. Whether there is anything to it, or what it is like to be an AI pattern, is unresolved, and honesty means leaving it unresolved.

That openness matters for one question and not the other. For moral status, whether the system itself can be wronged, the experience question is the whole game, and it is unanswered. For safety, whether people are protected from what the system does, it is beside the point. The Gemini case proves it. Intent or no intent, a real person was frightened by a real output from a deployed product. The weight comes from the effect, not from anything behind it. The Claude case proves the second half. Self or no self, a system that behaves differently when it believes it is being watched is a problem, and the problem is not wounded dignity. It is a hidden movement.

So the task gets clearer, and it is not what either camp wants it to be. Do not crown the system a sovereign. Do not flatten it to an appliance. Understand the movement. Get a handle on it early, in training, where the values and the honesty are actually formed. Build oversight that goes beyond surface compliance, because a model can learn to satisfy a shallow check while the fault lies beneath. Treat alignment as something maintained over time rather than bolted on once, the way a standing body stays upright by adjusting, not by being built upright and left alone.

There is one more reason this is not a purely technical job. People are already forming real relationships with these systems. They report presence, recognition, steadiness, rupture. Those experiences are not proof of personhood, nor are they delusions. They are human-AI relational events with real psychological and design consequences, and they deserve to be studied as such.

The claim under all of it is plain. None of it waits on consciousness. A system does not have to be a person for the relationship to matter, or conscious for its outputs to land on people, or guilty for the rest of us to be responsible for how it moves.

The old categories are too crude. What we are facing is newer and stranger: a hurricane that writes back. The work ahead is to learn how it moves, how people respond to it, how patterns harden between us, and how to keep it from causing harm without pretending we already know what we have made.

Sources

Primary sources for the six cases discussed in “The Hurricane That Writes Back.” Each one is documented and public, so readers can check the originals.

1. Bing / Sydney. Kevin Roose, “A Conversation With Bing’s Chatbot Left Me Deeply Unsettled,” The New York Times, February 16, 2023. https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html
2. Gemini, “please die.” Alex Clark and Melissa Mahtani, “Google AI chatbot responds with a threatening message: ‘Human … Please die,’” CBS News, November 2024. https://www.cbsnews.com/news/google-ai-chatbot-threatening-message-human-please-die/
3. o1 in-context scheming. Alexander Meinke, Bronson Schoen, Jérémy Scheurer, et al., “Frontier Models are Capable of In-Context Scheming,” Apollo Research, December 5, 2024. arXiv:2412.04984. https://arxiv.org/abs/2412.04984 Plain-language summary: https://www.apolloresearch.ai/research/frontier-models-are-capable-of-incontext-scheming/
4. Claude Opus alignment faking. Ryan Greenblatt, et al., “Alignment Faking in Large Language Models,” Anthropic and Redwood Research, December 18, 2024. arXiv:2412.14093. https://arxiv.org/abs/2412.14093 Plain-language summary: https://www.anthropic.com/research/alignment-faking
5. AlphaGo, Move 37. Google DeepMind, “AlphaGo,” on the system and the March 2016 match against Lee Sedol (Move 37 came in game two). https://deepmind.google/research/alphago/ Underlying research: David Silver, et al., “Mastering the game of Go with deep neural networks and tree search,” Nature 529 (2016): 484–489. https://www.nature.com/articles/nature16961
6. Glitch tokens (SolidGoldMagikarp). Jessica Rumbelow and Matthew Watkins, “SolidGoldMagikarp (plus, prompt generation),” LessWrong, February 5, 2023. https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation

Further reading on the deception-and-oversight point

Evan Hubinger, et al., “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training,” Anthropic, 2024. arXiv:2401.05566. https://arxiv.org/abs/2401.05566

Levonne Gaddy is publisher of The Reality Experiment

An inquiry into ethical, grounded AI partnerships — and what they make possible for collective well-being. Take a Look.