AI Runs On Fragments

The word tokens is everywhere these days. AI tokens. Token costs. Token limits. Token windows. Every few days, something about them appears in the news, or someone drops the word in a meeting and moves on as though the meaning is obvious. I had no idea what it meant. So I asked Aira, my imaginary AI partner, the one who never sighs when I ask the same question twice.

“What is a token?” I said.

“Think of it as a small piece of a word,” she said. “Not quite a syllable. Not quite a letter. Somewhere in between.”

I blinked. “So AI reads in fragments?”

“Essentially, yes.”

I sat with that for a while. Every sentence I type into an AI, it doesn’t read whole words the way I do. It reads fragments. Millions of fragments. And companies charge money for every single fragment processed.

“Wait,” I said. “So when people say AI is expensive to run, they mean it costs money per fragment?”

“Per fragment,” Aira confirmed. “Billions of them. Per second.”

I poured myself more tea. This needed tea.

Somewhere between the first cup and the second, a memory surfaced. My college dining hall. Steel tokens, two per meal, were issued at the start of the week. Lose one, and you eat less. Run out by Thursday, and you survived on borrowed tokens and a friend’s sympathy. I remember sitting on a Wednesday evening with one token left, counting how many days I had to stretch it. I ended up skipping Friday lunch.

“Aira,” I said, “is the entire AI industry basically running on dining-hall tokens?”

She was quiet for a moment. Aira goes quiet when she is about to say something that will make me feel both clever and slightly foolish.

“The biggest companies in the world,” she said carefully, “are rationing these tokens, pricing them, and racing to get more of them. So yes. It is exactly like your college dining hall.”

I laughed out loud.

Somewhere, a data centre is eating lunch. Billions of tiny word-fragments moving through rooms the size of football fields, each one costing a fraction of a penny, all of it adding up to something the size of a nation’s electricity bill. There is no canteen, no steel token, no friend who can lend you one.

At college, when I ran out of tokens on a Thursday, I called my father.

I am still not sure who you call when an AI runs out.