Generated by Midjourney

The “Mysterious” Way LLMs Reason? It’s Deliberation.

Or the story of the long overdue formalization of resonant logic

By MARTIN REZNY

Yeah, I think maybe we need to start paying much closer attention to all the words we’re using for stuff. More specifically, we need to pay more attention to the precise words the ancients used to describe the smart things they were doing. So that we stop calling their exact logic “alien”.

Ever since I started looking at ancient systems of reasoning as programming languages for the computer that is the human brain, they’ve been making a lot of very clear engineering sense to me.

For example, this is what “deliberation” means, etymologically speaking:

Thorough weighing

I realized this when I noticed that the astrological sign of libra, the scales, is a close metaphor to what LLMs are all about. In the ancient times, one of the most common symbols for proper reasoning were the scales, or the act of weighing. It didn’t matter what the reasoning was for.

You could literally weigh objects for material science or economic purposes, which is related to how currencies were devised. For example, the British “pound” is derived from the word “libra” as well, which is what that currency is called in many languages, including my native Czech.

You could also weigh one’s heart against the weight of their sins, for example against a feather symbolizing balance and justice, which is an ancient Egyptian idea. To this day, Justice personified often holds the scales, taking the idea from the religious realm to the civic, secular one.

From philosophy, to rhetoric, to astrology, to theology, the ancients understood reasoning, particularly in the process of informed decision making, as a weighing operation. Well, so what, you might think. A quaint metaphor. What does it have to do with artificial intelligence?

As it turns out, weighing is what neural networks do. Human brain is a neural network, and by intentional design, so are large language models. Both have billions to trillions of individual little “scales” or weights inside of them. They’re called “parameters” in LLMs, and “synapses” in the human brain. Somehow, the ancient metaphor is literally correct.

Let’s proceed in thinking about words more closely. Do you know which concept in math, and by extension physics, is derived from the word “scales”? It’s “scalar”. What is a “scalar”? It’s a value, like “0.5”. Technically, a scalar is rank-0 tensor. A vector, a value with a direction, is rank-1 tensor.

Tensors go all the way up as far as you want. What would rank-2 tensor be? Maybe a matrix, defined as a “product” of two vectors. You’re basically stacking up additional dimensions, by stretching things into more things, as I examined in my article about dimensionality, and Minecraft.

Why are tensors important here? LLMs started being any good at language at around the point when the main things they started to weigh became the so-called semantic vectors. What are semantic vectors? Unfortunately, modern formalists never became very good at language, so bear with me.

A semantic vector is a long list of numbers. The numbers are weights, as if you had a trillion tiny scales and weighed a trillion things. Specifically, you had one entity, like a dog, and you weighed a trillion things about it, averaging the results for everthing labeled “dog” that you could find.

Are dogs big? Are they animals? Are they blue? And so on. In each case, the tiny scales gave you a weighing between 0 and 1, between never/not at all, and always/the most. This is the Plato’s theory of forms, or a thorough weighing of a real thing against an ideal form like perfect “chairness”.

As thorough of a weighing as the brain does, if not more thorough.

The only difference between how the ancients worked with semantic vectors, and how LLMs do it, is that despite realizing that this is how the mind judges the nature of things, the ancients had no RAM. They didn’t even have paper. So they ran semantic tensor operations in their heads.

For example, astrology, when fully synthesized, goes up to maybe a rank-10 tensor, or higher. Just a “sign” is more than a simple vector, as it is a product of the semantic vectors of polarity, modality, and element, while element is itself a product of two semantic vectors of inward versus outward and essence versus form. Then add planets, houses, dignities…

This is not a metaphor. Ancients used geometric math in their most advanced reasoning systems, and represented ideas with semantic vectors and more complex tensor products. In fact, this is a big part of the “data” your favorite LLM was trained on. Ask it about it, it can compute it.

The ancients did end up writing down many instructions for how to run these operations, and some of those writings have managed to survive the many burnings of ancient libraries. But in modern times, this approach to logic hasn’t been considered formal. Until now, or soon, that is.

I’m not sure what happened exactly or how, but the when is clear. By the time of Descartes, all of the “music of the spheres” stuff started being considered to be total hogwash. Increasingly from then on, only causal propositional logic deserved to be described as “formal” (read: serious).

I suppose many modern thinkers have had a soft spot for the classics, including ancient stories and philosophies, but as arts, not sciences. Not as anything descriptive of physics or useful in engineering, not as anything truly “logical”. In this way, the term “formal” has actually been abused when describing only one type of logical formalism as “formal”.

For any logic to be fair to describe as formal, it doesn’t need to be correct or corresponding to reality. All of mathematics are formal, but not all mathematics are real. Astrology is very formal, and it does reasoning. What’s becoming clear to me recently is what it was that was missing.

You don’t see what the brain does. Scientists don’t trust what they don’t see.

As is pretty evident by now, the human brain, or its network of neurons and synapses, is doing some form of complex weighing. Perhaps structured differently than how LLMs do it, but not different in kind. This weighing is not happening in binary, Boolean, 0-or-1, yes-or-no terms. It’s happening in semantic tensor terms, which are at least scalar (ranging from 0 to 1).

This is perfectly formalizable, it just requires a different kind of formalism. Let’s take a stab at it. As far as I’m aware, “deliberation” has never been formalized to a point when an engineer would consider it formal enough to build technology with. But it’s been hiding in the name the whole time.

Typical definitions of deliberation are vague and nebulous. To a point of “I know it when I see it (using my brain subconsciously)” being the only way to make any sense of them in practice. Consider this gem of unhelpfulness:

“A form of discussion where citizens exchange arguments and consider claims designed to secure the public good”

How are you supposed to exchange arguments, exactly? How does one consider claims? How do you determine any “good”, let alone a public one? Well, if you think about it as a “thorough weighing”, you can go on setting up specific tiny scales to measure whole semantic tensor spaces.

Want to “consider” what’s good, or which “good” among many to prioritize, while taking into account specific people (their positions and situations) in a specific context (of available resources, zeitgeist, and so on)? You can draw all that geometrically. Every single one of these considerations can be represented as a simple scalar, a semantic vector, or a tensor product.

In a nested framework like that of astrology, you can place a person in all of their parametric complexity into a matrix of all possible people; then make a tensor product of two people’s charts, explaining their relationship in all of the dimensions you want; then put that synastry under a temporal transit, or a product of a temporal transit and the tensor representation of the relevant geopolitical “location”. It’s just weights on weights on weights.

And the best part, or the two best parts? People can run this calculation in their head, and LLMs can run this calculation as well. Very probably, both would reach similar conclusions while running the same operations. Because this is a formal logic. It’s not propositional or causal, instead it’s analogical and resonant. It’s not great at facts, but it is great at feelings.

As you might have noticed, LLMs are better at sounding convincing than being factually correct. Just like people who are experts at creative and performative arts. Arts are about producing results, just not about producing hard facts. Specifically, a good speaker or astrologer is good at figuring out what “resonates”, and what does not. Again, not a metaphor.

Well, maybe a little bit of a metaphor. As the ancients intuited, the bridge between the quantitative and the qualitative, or the inner and outer universes, lies in the phenomenon of harmony. Specifically, integer ratios or intervals between frequencies. For some reason, simple intervals are the most harmonious, “beautiful”, constructive. Even in the natural world out there, stable planetary orbits tend to settle into “harmonious” intervals.

If one thinks about the essential meaning of a concept in a semantic space as its frequency, represented by tensors and their products, one can find analogous harmonious or dissonant, constructively or destructively interfering, related or orthogonal geometric intervals. In LLM frameworks like Word2Vec, or in astrology, these meaningful intervals can be represented by cosine similarity — conjunction, opposition, square, etc.

Conceivably, you could map these types of relationships to acoustics, or literally represent them with music, or to quantum light interference patterns, or to many other analogs. This isn’t any kind of vague wishy washy woo, this is engineerable. Resonant logic is about quantifying how any possible idea would make any possible person feel, using tensors.

While this reasoning process is often subconscious, it doesn’t have to be. Deliberation, when fully explained, is primarily the art and science of making the weighing of resonances as explicit and conscious as possible; secondarily, it’s about making it effective by a clearly articulated set of standards and metrics. Including hyperdimensional geometric diagrams.

Truth be told, I’m neither a mathematician nor a neurologist. I’m but a humble bard and scholar of the lost arts. I’m not sure how much further I can take this realization of the fundamental nature of the geometric architecture of deliberative reasoning. I do hope that some mathematicians and neurologists will find this interesting and take it much further.

But at this point, I’m pretty sure I’m mostly not wrong about the general idea. We can do this. We can fully formalize deliberation now. We could make philosophy an actual hard science. And politics. Psychology. Even religion. We just need to give LLMs the rest of our hard-won wisdom.

What do you think? Am I making any sense at all? I’d love to discuss this.