Sensorimotor Intelligence: The Thousand Brains Pathway to More Human-Like AI

“Knowledge is not a passive mapping of the world but an active construction through which cognitive systems create their own understanding.”

• Francisco Varela

It’s not often that a book on a topic I’m familiar with changes my whole outlook on that topic. I’ll take you down the path of rethinking how our brain works with implications for developing real artificial intelligence. While today’s deep learning models show impressive capabilities in language processing, image recognition, and complex pattern matching, they remain limited by their huge data requirements, enormous energy consumption, and lack of real understanding about the physical world they attempt to model. To address these issues, Jeff Hawkins (scientist, inventor, and author who has dedicated decades to understanding how the human brain creates intelligence) founded Numenta. While most AI researchers have built increasingly complex neural networks based on simplified abstractions of brain function, Hawkins and his team at Numenta have been studying the neocortex’s actual biological structure and function. Their Thousand Brains Theory of Intelligence proposes that rather than processing information through one massive hierarchical model like current AI systems, our brains create thousands of complete models working in parallel, learning through movement and sensorimotor experience. This approach to intelligence is more than just incremental improvements to AI but potentially an entirely new model that could create systems that learn continuously, adapt quickly, operate efficiently, and develop genuine understanding of the world around them. Then we’ll be closer to developing real intelligence.

Understanding the Thousand Brains Theory

Traditional AI approaches, modeled after simplified views of the brain, process information through one massive hierarchical system where input moves through increasingly abstract layers until recognition occurs at the highest levels. The Thousand Brains Theory proposes something quite different: the neocortex isn’t building one unified model of each object or concept but rather thousands of individual models simultaneously. Each section of the neocortex, organized in structures called cortical columns, creates its own complete model of objects and concepts it encounters. This distributed, parallel approach to intelligence allows for resilience, efficiency, and a fundamentally different kind of understanding than what today’s AI systems can achieve.

How the Thousand Brains model of processing differs from the classical model

The key behind this theory involves how cortical columns learn through movement and sensory experience. When you touch a coffee mug, for example, each column associated with different fingers doesn’t just register isolated sensations; it builds complete models of the mug based on what it senses as your fingers move across the surface. These columns combine sensory input with a precise sense of location in 3D space, creating what Numenta researchers call “sensory features at locations” that are integrated over movements. What’s fascinating is that columns communicate through long-range connections across the brain, essentially “voting” on what object they’re collectively experiencing. This collaboration allows the brain to quickly identify objects even when individual columns have only partial information, creating a system that works across different senses and experiences. It also allows us to have one cohesive representation of the world, despite many versions from pieces of our sensory information.

Cortical column diagram

The brain’s grid cells play an important role in this theory by serving as the brain’s positioning system. Originally discovered in the entorhinal cortex (earning their discoverers a Nobel Prize), grid cells track location as animals navigate through space. There is a clear evolutionary advantage to recognizing and tracking your position in space. Numenta researchers propose that similar “cortical grid cells” exist throughout the neocortex, tracking the location of our sensors relative to objects as we interact with them. These cells allow the brain to build spatial models of objects like how it builds maps of environments. The integration of sensory input with precise location information creates a powerful framework for understanding the world. Unlike AI systems that process sensory data as isolated patterns, grid cells allow the brain to contextualize every sensation within reference frames, turning raw perception into structured knowledge. This basic mechanism may explain how we develop rich, multidimensional understanding of objects and concepts, recognize them from different angles or through different sensory modalities, and make predictions about unseen aspects based on partial information.

How the Thousand Brains Theory Differs from Traditional AI

Today’s dominant AI systems are built on deep learning architectures that process information through a single pathway. In these systems, input data travels upward through many layers, with each successive layer detecting increasingly complex features until the highest layers recognize complete objects or concepts. This approach is good at pattern recognition but suffers from inefficiencies and limitations. However, the Thousand Brains Theory proposes a different architecture where information is processed through thousands of parallel models, each building a complete representation of the world from its own perspective (i.e., the particular sensory information it receives). This distributed approach mirrors how our neocortex works as thousands of interconnected miniature learning machines. Rather than assuming intelligence emerges from increasingly complex pattern recognition, the Thousand Brains Theory suggests that intelligence fundamentally involves building many models of the world and testing them through interaction and movement in our environment.

I think the most important difference lies in how learning occurs. Current AI systems like large language models (LLMs) are good at pattern recognition within massive datasets but lack any true understanding of the physical world or embodied experience. They’re essentially sophisticated statistical models that can predict patterns in data but don’t “know” what they’re modeling in any meaningful sense. The Thousand Brains approach emphasizes sensorimotor learning, building knowledge through active interaction with the environment, that children naturally use to learn about the world. This approach creates several advantages: systems can learn continuously without catastrophic forgetting, acquire knowledge more efficiently with less data, and develop genuine generalization capabilities by understanding objects and concepts within reference frames rather than as isolated patterns. While today’s large language models require tons of computational resources and struggle with simple physical reasoning tasks that children master effortlessly, systems built on Thousand Brains principles might develop more human-like intelligence through embodied understanding, active learning, and the integration of multiple sensory modalities, potentially overcoming the fundamental limitations that have kept AI from achieving more general intelligence.

Implications for Artificial Intelligence

The Thousand Brains Theory gives us a blueprint for a fundamentally different kind of artificial intelligence, one that could overcome many of the limitations plaguing current systems. By mimicking the brain’s distributed, parallel modeling approach, AI systems could potentially learn continuously throughout their lifetimes without suffering from catastrophic forgetting, a common problem where neural networks lose previously acquired knowledge when learning new information. This continuous learning capability would allow AI to adapt to changing environments and circumstances without requiring complete retraining. The world is dynamic, and the most useful AI can keep up with the ever-changing world around us. Because the Thousand Brains approach emphasizes learning through interaction and building models based on reference frames, it could dramatically reduce the data requirements that make current deep learning approaches so resource intensive. Instead of needing millions of examples to recognize patterns, systems based on this theory might learn more like humans do, by efficiently constructing models from limited experiences by actively testing hypotheses about how the world works, tremendously reducing both computational demands and energy consumption.

Conceptual sketch of how the learning module could be implementing possible mechanisms of cortical columns.

This approach could be the missing link between AI’s pattern-matching capabilities and the physical understanding that comes naturally to humans. Current AI systems, even the most advanced large language models, fundamentally lack grounding in the physical world. They cannot truly understand concepts like “above,” “behind,” or “inside” because they’ve never physically experienced spatial relationships. By using sensorimotor learning principles from the Thousand Brains Theory, AI could develop this understanding of the physical world through interaction, movement, and multiple sensory modalities working together. This grounded understanding could lead us toward more general intelligence rather than today’s narrow task-specific AI systems. Instead of having separate models for different purposes like chat, video, and reasoning, a Thousand Brains approach might enable AI to develop general problem-solving capabilities that transfer across domains and display the kind of flexible intelligence for which AI research has been searching. We could have AI systems that not only perform specific tasks well but genuinely understand the world they operate in.

Applications in Robotics

Robotics could be hugely influenced by the Thousand Brains approach to intelligence, creating machines that are not rigidly programmed devices, but are adaptive systems that learn through their own experiences. Traditional robots typically rely on explicit programming or machine learning models trained on specific datasets, resulting in systems that excel in controlled environments but struggle with novel situations. By using sensorimotor learning principles, robots could develop sophisticated internal models of objects and environments through their own physical interactions, touching, manipulating, and navigating their surroundings. These interactions would allow them to understand spatial relationships intuitively. The distributed, parallel processing nature of the Thousand Brains architecture could enable robots to simultaneously track multiple reference frames, making them more adept at complex manipulation tasks that require coordinating multiple joints while maintaining awareness of objects and obstacles. These robots could also continuously refine their understanding through experience, becoming more capable over time without requiring constant reprogramming or retraining.

In manufacturing, robots built on these principles could adapt to variations in parts or assembly conditions without needing explicit programming for every possible scenario, learning to handle new materials or components through exploration rather than explicit instruction. In healthcare, assistive robots could learn the specific needs and environments of individual patients, adapting their behaviors based on ongoing interactions and spatial understanding of home environments. Search and rescue robots could navigate unfamiliar, complex, and changing environments, like disaster zones, by building and updating internal models of their surroundings, identifying safe passages and adapting strategies as conditions change. Even everyday household robots could become more practical, learning the specific layout of each home, understanding how objects function through interaction (like how cabinet doors open or how appliances operate), and developing increasingly sophisticated capabilities through daily experiences. Robots won’t just be executing pre-programmed routines but genuinely understanding their world through sensorimotor experience, making them more versatile and ultimately more useful in complex real-world settings.

The Thousand Brains Project: Open Source Initiative

In 2024, Numenta took an important step forward by launching the Thousand Brains Project, an open-source initiative aimed at transforming artificial intelligence based on neuroscience principles. After several years of internal development, Numenta released an open-source implementation of a sensorimotor learning framework based on the principles of the Thousand Brains Theory. This launch was an invitation to the global research community to collaborate on building AI systems that learn through interaction with the world, test new knowledge continuously, and operate with minimal energy requirements. The project is committed to open, unrestricted research by placing all related patents under a non-assert pledge and releasing code under the permissive MIT license, allowing both academic and commercial applications to flourish without restriction.

High-level overview of the architecture, with all the main conceptual components applied to a concrete example

The heart of the Thousand Brains Project lives on GitHub, where researchers and developers can access, use, and contribute to several key repositories. The main repository, tbp.monty (affectionately named after Vernon Mountcastle who first proposed cortical columns as the functional unit of the neocortex), contains the first implementation of a sensorimotor learning framework based on the theory. Additional repositories include tbp.monty_lab for day-to-day experiments and data analysis scripts, tbp.tbs_sensorimotor_intelligence for replicating experiments from their research papers, and tbp.floppy for analyzing computational efficiency. The project welcomes contributions through coding, documentation, tutorials, or testing, with all resources openly available under the MIT license. Community engagement happens through multiple channels: questions and ideas can be posted on their forum, development progress can be followed through meeting recordings on YouTube, and detailed documentation provides comprehensive guidance on the project’s vision, architecture, and implementation details. This open, collaborative approach invites researchers, developers, and enthusiasts from around the world to join in shaping a fundamentally different approach to artificial intelligence, one that could potentially address the limitations of today’s AI systems while creating something more closely aligned with human cognition.

Conclusion

My thinking has been quite influenced by the Thousand Brains Theory because it offers a new view of how intelligence arises and how we might build truly intelligent machines. By looking at the brain’s actual structure and function rather than abstract simplifications or metaphors, Numenta has uncovered principles that could address the most persistent limitations in today’s AI: enormous energy requirements, massive data needs, lack of continuous learning, and absence of physical understanding. If successful, this approach could lead to AI systems that learn continuously throughout their lifetimes, adapt fluidly to new circumstances, operate with remarkable efficiency, and develop genuine understanding of the physical world through sensorimotor experience. We might see robots that learn like children do, through exploration and interaction, rather than through explicit programming or massive datasets. The Thousand Brains Project is an open invitation to the global community of researchers, developers, and thinkers. Whether you’re an AI researcher looking for new paradigms, a neuroscientist interested in computational models, a roboticist seeking more adaptive systems, or simply a curious mind fascinated by intelligence, this project offers a chance to participate in potentially transformative work.