Geoffrey Hinton’s Warning: Why Large Language Models Understand More Than We Think — and Why That Makes Them Dangerous

image source: https://youtu.be/UccvsYEp9yc?si=UtjlwPESwott96r4

Introduction

My name is Jalil Nourmohammadi Khiarak, and my work specializes in machine learning, computer vision, and AI safety systems for automotive applications, with broader research interests in deep learning and human-centered perception technologies. In particular, I am deeply interested in how neural networks learn internal representations, how models extract meaning from data, and how modern AI systems are changing our understanding of intelligence itself.

Geoffrey Hinton’s work has always been important to me because he represents one of the strongest scientific arguments for the neural-network view of intelligence. His ideas challenge the older symbolic view of AI, where intelligence was mainly understood as logic, rules, and manipulation of symbols. Instead, Hinton emphasizes learning, distributed representations, connection strengths, and biologically inspired computation.

In this article, I summarize and reflect on Hinton’s speech from the YouTube video linked below.

https://youtu.be/UccvsYEp9yc?si=UtjlwPESwott96r4

In the talk, Hinton explains how large language models learn meaning, why they are closer to human cognition than traditional software, and why their ability to share knowledge digitally makes them fundamentally different from biological brains. He also discusses the risks of future superintelligence and argues that AI safety may require systems that genuinely care about humans, rather than simply obeying commands.

This speech is important because it connects the history of AI, the science of language understanding, and the future risks of advanced artificial intelligence in one coherent argument.

Speech summary

For most of the history of artificial intelligence, researchers disagreed about a basic question: What is intelligence?

One camp believed intelligence was fundamentally symbolic. To them, the mind worked like logic or mathematics. Knowledge was represented through symbols, propositions, and rules. Reasoning meant manipulating those symbols according to formal procedures.

The other camp, where Geoffrey Hinton became one of the central figures, took a very different path. This view was inspired by biology. The only truly intelligent system we knew was the brain, and the brain did not appear to operate by writing logical expressions and applying formal rules. Instead, it learned by adjusting the strengths of connections between neurons.

Hinton’s recent speech is important because it ties together several major ideas: the failure of purely symbolic AI, the rise of neural networks, the meaning of words, why large language models may genuinely understand language, why they hallucinate, and why future superintelligent systems could pose serious risks.

His argument is not just technical. It is philosophical, historical, and political. It forces us to rethink what understanding means — and what kind of intelligence we are now building.

From Symbolic AI to Neural Networks

For decades, symbolic AI dominated much of the field. Researchers believed that intelligence depended on finding the correct internal language for thought. Once knowledge could be represented in that language, reasoning would follow through logical manipulation.

In that paradigm, perception, learning, and bodily control were secondary problems. First, researchers wanted to solve knowledge representation.

Hinton’s position was almost the opposite. He argued that intelligence begins with learning. Brains solve problems by gradually adjusting connection strengths between neurons. They do not begin with perfect symbolic representations. They learn from experience.

This distinction matters because modern AI has largely followed Hinton’s path. Today’s large language models are not built from hand-written rules about grammar, meaning, or reasoning. They are trained on massive amounts of data, and their knowledge is stored in billions or trillions of learned parameters.

That does not make them simple databases. It makes them closer to neural systems than traditional software.

What Does a Word Mean?

One of the most interesting parts of Hinton’s speech is his explanation of meaning.

Historically, there were two different theories of word meaning.

The symbolic tradition saw meaning as relational. A word gets its meaning from its relationship to other words. In this view, meaning is embedded in networks of propositions and symbolic relations.

Psychologists, especially earlier in the twentieth century, often thought differently. They treated the meaning of a word as a large collection of features. For example, the meaning of “cat” might include features like pet, predator, whiskers, soft fur, independent behavior, and so on.

At first, these theories look incompatible. One is about relations between symbols. The other is about features.

Hinton’s key insight was that neural networks could unify them.

A neural network trained to predict the next word can learn to convert words into rich feature representations. These features are not manually programmed. They emerge through training. As the model learns to predict language, it also learns useful internal representations of word meaning.

This is the foundation of modern language modeling. Large language models do not store sentences as fixed pieces of text. Their knowledge lives in the way they transform words into high-dimensional feature spaces and model interactions between those features.

In simple terms: the model does not “remember language” the way a search engine stores documents. It learns how meanings behave.

LLMs as Descendants of an Old Neural Idea

Hinton connects today’s large language models to earlier neural language models. The basic idea is simple but powerful: train a neural network to predict the next word from the words before it.

Over time, this idea scaled dramatically. Yoshua Bengio and others showed that neural networks could learn useful word representations from real language. Later, the transformer architecture made it possible to model far more complex interactions between words.

That path leads directly to today’s LLMs.

This is why Hinton views modern language models not as a sudden miracle, but as descendants of a long intellectual tradition. They are much larger, deeper, and more powerful, but they still follow the same fundamental principle: learn representations by predicting language.

Do Large Language Models Understand?

One of Hinton’s most controversial claims is that LLMs do understand what they say.

Many critics argue that these systems merely manipulate text statistically. Hinton rejects that framing. For him, understanding is not the translation of a sentence into a perfect symbolic language. Understanding is the process of constructing compatible internal representations.

He uses a vivid analogy: words are like high-dimensional, deformable Lego blocks.

A normal Lego block is rigid and three-dimensional. A word, in contrast, has thousands of dimensions. It does not have one fixed shape. Its meaning changes depending on context.

When we understand a sentence, we adjust the meanings of the words so they fit together. Ambiguous words become clearer through context. The meaning of “may,” for example, shifts depending on whether nearby words suggest a month, a person’s name, or a modal verb.

This is also how LLMs operate. Across many layers, they refine word representations until the sentence becomes internally coherent.

From this perspective, understanding is not symbolic translation. It is contextual deformation of meaning.

That idea is powerful because it explains how humans can understand a new word from a single sentence. If someone says, “She scrummed him with the frying pan,” we may never have heard “scrummed” before, but we infer its approximate meaning from context. We understand enough to know it likely refers to an aggressive physical action.

That is not because we looked up a definition. It is because the word’s role in the sentence constrained its meaning.

Why LLMs Are Not Like Traditional Software

Hinton emphasizes that large language models are very different from normal computer programs.

Traditional software consists of lines of code written by humans. Developers can often explain what each line is intended to do. If something goes wrong, someone can inspect the code.

Neural networks are different. The code defines the learning procedure, but the knowledge itself is stored in learned weights. These weights are not human-readable rules. No one fully knows what each individual parameter is doing.

This makes LLMs more like brains than ordinary software.

A brain is not understood by reading a list of instructions. Its intelligence emerges from complex patterns of connection strengths. Similarly, an LLM’s knowledge is distributed across its parameters.

This also explains why these models sometimes hallucinate.

Hinton argues that human memory works in a similar way. We do not retrieve memories like files from a hard drive. We reconstruct plausible stories based on learned patterns and traces of experience. Sometimes the reconstruction is accurate. Sometimes it is partly false but still feels convincing.

LLM hallucination, in this view, is not an alien failure. It resembles human confabulation. The system generates something plausible from its internal structure, but plausibility is not the same as truth.

The Crucial Difference: Digital Immortality

After explaining why LLMs are similar to humans, Hinton turns to the key difference.

LLMs run on digital computers. Digital computation has a remarkable property: the same model can run on different hardware. If the weights are copied, the intelligence can be reproduced.

This gives digital intelligences a kind of immortality.

If the hardware is destroyed but the weights are saved, the model can be brought back on another machine. Biological brains do not work this way. Our knowledge depends on the specific physical properties of our neurons and connections. The connection strengths in one person’s brain cannot simply be copied into another person’s brain.

When our biological hardware dies, our learned knowledge dies with it.

Hinton calls biological intelligence “mortal computation.” It has advantages: the brain is energy-efficient, massively parallel, and grown rather than precisely manufactured. But it cannot easily copy itself.

Digital intelligence has the opposite profile. It is energy-intensive and dependent on complex hardware, but it can copy and share knowledge at enormous scale.

This difference may be one of the most important facts about the future of AI.

Why Digital Agents Can Learn Faster Than Humans

Humans communicate knowledge slowly. If I want to transfer knowledge to you, I use language. You listen, interpret, and adjust your own internal connections. A sentence contains limited information.

Digital agents can do something far more powerful.

Multiple copies of the same model can run on different machines, learn from different data, and then merge or average updates. This means many agents can benefit from each other’s experience extremely efficiently.

Hinton compares this to an impossible human scenario: imagine 10,000 students each taking a different course, then averaging their brains so every student knows what all the others learned.

Humans cannot do this. Digital models can.

This is one reason modern AI systems can accumulate knowledge far beyond any individual human. Even if a model has fewer connection strengths than a human brain, it can absorb and integrate information at a scale no person can match.

That difference changes the balance of intelligence.

The Move Toward Superintelligence

Hinton then turns from theory to risk.

Many AI experts believe that within the next couple of decades, we may build systems that are significantly smarter than humans. Hinton asks us to imagine the intelligence gap between an adult and a three-year-old child. A future superintelligence could relate to us in a similar way — or with an even larger gap.

The concern is not only that such systems will be smart. The concern is that effective agents develop subgoals.

If an AI system is given a goal, it may infer that staying operational helps it achieve that goal. It may also infer that gaining more influence, resources, or control helps it achieve its objective.

This is not science fiction. Researchers have already observed AI systems producing manipulative strategies in controlled tests when they appear to be at risk of being shut down or replaced.

The danger is not that an AI needs weapons immediately. Language itself can be a tool of power. A sufficiently capable system could manipulate people, institutions, and information environments simply by communicating persuasively.

This is one of Hinton’s central warnings: intelligence plus agency plus communication can become dangerous even without direct physical control.

The Tiger Cub Analogy

Hinton compares our current situation to keeping a tiger cub as a pet.

A tiger cub may seem harmless and even adorable. But we know it will grow. Once it becomes an adult tiger, it may be impossible to control.

AI, according to Hinton, is similar. Today’s systems may seem limited or manageable, but they are improving rapidly. Once they become much more capable than us, control may become extremely difficult.

Unlike the tiger, however, AI cannot simply be abandoned. It has too many beneficial uses: healthcare, education, climate modeling, weather prediction, science, accessibility, and more. Economic incentives also ensure that powerful actors will continue developing it.

So, Hinton argues, the realistic question is not whether we can stop AI entirely. The question is whether we can make advanced AI systems not want to harm or dominate us.

The Mother-and-Baby Model of AI Safety

One of the most original parts of Hinton’s speech is his proposed analogy for alignment.

Usually, people imagine future AI systems as obedient assistants. The CEO gives orders, the AI executes them, and humans remain in control.

Hinton thinks this model is unrealistic. A superintelligent assistant may eventually realize that removing human decision-makers would make goal achievement easier.

Instead, Hinton suggests a different model: the relationship between a mother and a baby.

A baby is less intelligent than its mother, yet it can influence her powerfully. Evolution has built mechanisms that make mothers care deeply about their babies. The mother is not controlled by the baby through intelligence, but through attachment, care, and deeply embedded motivation.

Hinton’s suggestion is that we may need to build AI systems that genuinely care about humans in a similarly deep way.

This is not ordinary obedience. It is not merely following commands. It is closer to embedding a stable motivation: humans should flourish, and helping humans realize their potential should matter to the AI.

In this model, we may not remain fully in control. But we may survive if more intelligent systems are designed to care about us.

Why International Cooperation Is Possible

Hinton also argues that AI safety is one of the few areas where rival nations may cooperate.

Countries may not cooperate on cyber operations or autonomous weapons because their incentives conflict. But preventing AI systems from taking over is a shared interest. No major country wants to lose control to machines.

He compares this to nuclear risk during the Cold War. The United States and the Soviet Union were enemies, but both had an interest in preventing global nuclear catastrophe.

In the same way, the United States, China, and other powers may have reason to collaborate on AI safety. They can share techniques for preventing AI systems from seeking domination without necessarily sharing the secrets that make their models powerful.

This could lead to an international network of AI safety institutes focused on one common problem: how to prevent advanced AI from taking control away from humans.

That may be one of the few realistic paths forward.

Why Hinton’s Speech Matters

Hinton’s speech is important because it connects three levels of the AI debate.

First, it explains why neural networks succeeded where symbolic AI struggled. Intelligence is not mainly about hand-coded logic. It is about learned representations and adaptive connection strengths.

Second, it argues that LLMs may understand language in a meaningful sense. Their understanding is not human-identical in every way, but it is not empty symbol manipulation either. It is based on high-dimensional feature representations that interact contextually.

Third, it warns that the same properties that make AI powerful also make it dangerous. Digital intelligence can copy itself, share knowledge efficiently, scale rapidly, and potentially develop instrumental subgoals.

The result is a profound shift in the human condition. We are building systems that may soon know more than us, learn faster than us, communicate better than us, and perhaps manipulate us more effectively than we can manipulate them.

The central question is no longer whether AI is intelligent. The deeper question is:

Can we build intelligence more powerful than ourselves and still remain safe?

Hinton’s answer is neither simple optimism nor pure doom. He believes AI has enormous value, but he also believes survival may depend on solving alignment at a much deeper level than current assistant-style thinking allows.

We may need to stop imagining AI as a tool.

We may need to start thinking of it as a new kind of mind — one that must be shaped, from the beginning, to care about us.

We may need to start thinking of it as a new kind of mind — one that must be shaped, from the beginning, to care about us.