NEUROSYMBOLIC AI | SERIES

Why Your Calculator Doesn’t Hallucinate

There are more things in heaven and earth than are dreamt of in your correlation model

Humans spent thousands of years carving reality at its joints to make sense of our world and control it.

From prehistoric roots, where storytelling and correlation were our primary tools for understanding the world, we spilled ink and blood to develop robust theories of causation, logic, mathematics, physics, machining, engineering, and computing — knowledge of the world.

And damn, we succeeded.

Look at this list of over 1,000 academic disciplines.

• We developed coordinate systems, cartography, meteorology, and navigation technology that opened the globe to travel and trade.
• We developed theories in mathematics and physics, methodologies for experimentation, and physical instrumentation that were crucial for building bridges, roads, skyscrapers, water distribution systems, and all the other structures we didn’t want to collapse.
• We wrote systems of law, political theories, and economic models that improved society.
• We built logistical supply chains across vast terrain — connecting billions of people while mitigating inevitable conflicts that come with complex regulations, laws, and scarcity.
• We discovered the neurological, biological, chemical, and physical causal mechanisms that undergird our ability to create medicine, perform life-saving surgeries, and enhance our lives.
• We invented computers, the internet protocols, web browsers, and applications that democratized information, connected billions of people, and made every other invention on this list more useful.

But then, with our newfound powers, we began dismantling the principles necessary to develop them — all in favor of our ancestors’ limited toolbox:

1. statistical correlation
2. reinforcement “learning”

Created by the author using imgflip.com.

Everything that AI is built on — the computer, the operating system, the network protocol, the programming language, the math libraries — exists because of formal specification, grounded and guaranteed. The current AI industry is choosing to ignore the principles that made it possible.

Look at a device built on formal systems you already trust: the calculator.

We shouldn’t need a Turing test for a calculator

Imagine two calculators side by side on your desk.

Calculator A works as you expect. The functioning of calculator B is a bit mysterious.

The first time you press 123, both calculators display 123.

After clearing the calculators, you repeat the process.

The hundredth time you press 123, you still see 123.

Now, you enter a different number — say, 124.

Calculator A displays 124. Calculator B displays 123.

What the heck?

Calculator B uses correlation and reinforcement to determine the output. User typed 12 followed by 3 a bunch of times. They must be doing it again. I’m going to ignore the 4 and replace it with a 3.

Obviously, a shitty calculator. But why?

1. The axioms and theorems of mathematics are not reducible to mere correlation relationships.
2. Calculator B’s output depends on the statistical pattern of what it has seen, not on the structure of the operation it’s supposedly performing. Calculator B doesn’t do arithmetic.

OK, point taken. Why is calculator A so much better?

Guarantees, specifications, idempotence, and grounding

1. Given the same inputs, calculator A will always generate the same output (the hundredth time or the billionth time).
2. The output is determined by the application of formal theory (math), hardware specifications, and physical theory.
3. The calculator is grounded in semantic and physical primitives: binary semantics map to circuitry state, and logic is performed via logic gates. The mathematical operations follow the rules of maths.
4. Each layer involved is standardized and formally specified, both physically and programmatically.
5. The interactions between layers are structurally and procedurally axiomatized into protocols.

What principle can we extract from this discussion?

If we want useful AI, or software of any ilk, we need to specify some symbolic structure and ground that structure in our knowledge of the world.

The same properties that make calculator A reliable are what make any AI system safe enough for mission-critical work.

Verifiable behavior, bounded outputs, auditable reasoning, formal guarantees about what the system will and won’t do. Pure correlation models can’t offer these. Hybrid architectures that combine neural pattern recognition with formal symbolic components can. Safety for powerful AI is formalization. The fourth piece in this series develops what those architectures actually look like.

Correlation models lack symbolic structure and grounding

Without it, correlation models struggle with “sample inefficiency, poor out-of-distribution generalization, lack of compositionality[,] and inability to reason logically” and “have no access to or awareness of the physical, perceptual, goal-oriented or social contexts in which [the correlations] occur” (Pavlick 2023¹).

The results have been put on blast across the web and media.

• “Models ace bar exams but invent case law, pass CPA tests but pick the wrong cells in a spreadsheet.”²
• All popular models have a greater than 15% false assertion rate (hallucination⁴) on an open-book test.⁵
• Barely 50% accuracy in financial reporting tasks.⁶
• Correlation models “can’t reliably stick to the rules of chess despite having them — and millions of games…in their database.”³

The correlation hammer works well to drive statistical nails, but the world contains multitudes. We are using a hammer to tie our shoes.

Building calculators out of nails. Photo by Anne Nygård on Unsplash.

We built the formal foundations. We have them in mathematics, in physics, in law, in medicine, in computing itself. We are still creating them. When we stop, the current generation of AI hits its ceiling.

The industry has chosen to forego those foundations. The next piece examines what it costs to reinvent knowledge that already exists — and to reinvent it badly.

Part 0 of a series on AI and formalization. The next parts cover what the major labs are doing instead — paying experts to write rules in natural language for neural models to learn back (Part 1), the costs of doing this at scale (Part 2), why the industry hasn’t pivoted (Part 3), what hybrid neurosymbolic architectures actually look like (Part 4), and the bounded domains where they should get built first (Part 5).

References

[1]: Pavlick E. 2023, Symbols and grounding in large language models. Phil. Trans. R. Soc. A 381: 20220041. https://doi.org/10.1098/rsta.2022.0041.

[3]: https://garymarcus.substack.com/p/why-do-large-language-models-hallucinate. Or, “…it is an absolute embarrassment that a technology that has collectively cost about half a trillion dollars can’t do something as basic as (reliably) check its output against Wikipedia or a CNN article that it’s handed on a silver platter” (ibid). [4]: I consider the AI term “hallucination” to be marketing propaganda; false assertions and guesses are not hallucinations.