Hands-On Intelligence: Why the Future of AI Moves Like a Curious Toddler, Not a Supercomputer

“We are neural beings. Our brains take their input from the rest of our bodies. What our bodies are like and how they function in the world thus structures the very concepts we can use to think. We cannot think just anything — only what our embodied brains permit.”

- George Lakoff (a Berkeley buddy)

Children are the world’s best learners. Have you ever seen a toddler get a new set of colorful building blocks for the first time? After picking up each piece, she explores their shapes, colors, textures, and how they fit together. The child intuitively begins constructing towers and structures, quickly grasping the toy’s possibilities through exploration and play. Now compare this child’s abilities to current, state-of-the-art AI systems that require millions of labeled images and enormous computational resources to achieve even basic object recognition. These AI systems still have difficulty when encountering these same blocks in slightly different arrangements or under unfamiliar conditions, like under different lighting. We’ve been creating AI that is both data-dependent and fragile, performing well in narrow tasks, but struggling with the effortless learning activities children engage in daily. But what if our current approach to AI has been misguided? What if instead of throwing ever-larger datasets at huge, power-hungry data centers, we allow them to learn in the way we (and other mammals) naturally do through active exploration, physical interaction, and gradual understanding built through sensorimotor experiences? A new research paper by The Thousand Brains Project called “Thousand-Brains Systems: Sensorimotor Intelligence for Rapid, Robust Learning and Inference” suggests this shift in approach could change how we design truly intelligent systems.

How Your Brain Really Works (And Why It Matters for AI)

Neuroscientist Vernon Mountcastle observed something interesting about the mammalian brain: despite performing vastly different functions, from processing vision to controlling movement to understanding language, every region of the neocortex shares a similar structure. This insight led him to propose that intelligence emerges not from specialized brain regions, but from the replication of a single, powerful computational unit called the cortical column. Building on this insight, the Thousand Brains Theory suggests that each of these columns is a complete sensorimotor learning system in miniature. Rather than having one visual system, one motor system, and one memory system working in isolation, your brain contains thousands of semi-independent learning modules. Each processing unit builds its own models of the world through movement and sensation. Every time you reach for an object, multiple cortical columns throughout your brain are simultaneously learning about its shape, texture, and spatial relationships by actively predicting what they’ll encounter next as your hand moves.

Children intuitively learn about objects through interacting with them: by looking at them, touching them, and playing with them.

This biological architecture exposes why current AI approaches are missing the mark. Current deep learning systems gorge themselves on static datasets trying to memorize millions of examples without ever touching or moving through the world. However, your brain learns through active exploration and constant sensorimotor feedback. The child learning about the toy blocks doesn’t need to see thousands of photos labeled “block.” Instead, she picks it up, rotates it in her hands, traces edges with her finger, feels its weight, and gradually builds a rich, three-dimensional understanding of its structure and properties. Each movement generates predictions about what they’ll sense next — if I move my finger along this edge, will it curve or stay straight? This active prediction and correction creates robust, generalizable knowledge that works regardless of lighting conditions, viewing angles, or minor variations in the block’s appearance. It’s this fundamental difference between passive data consumption and active sensorimotor exploration that holds the key to building truly intelligent machines.

Meet Monty: The First “Thousand-Brains” AI System

Meet Monty, the first artificial intelligence system designed to learn like a living brain rather than a massive data-processing machine. Named after Vernon Mountcastle himself, Monty is a big departure from conventional AI architecture. Instead of being a singular neural network trained on millions of images, Monty is a modular system composed of interconnected components that mirror the brain’s structure: Learning Modules that build and use internal models of objects, Sensor Modules that process incoming sensory information, a Motor System that generates purposeful movements, and a Goal State generator in the Learning Module that drives its motivations and curiosity. These components communicate through something called the Cortical Messaging Protocol, which allows different parts of the system to share information about poses, features, and spatial relationships. Monty actively explores objects by moving around them, just like a curious child or a person examining something with their hands in the dark.

Monty is inspired by the brain’s modular structure, composed of specialized sensor, motor, and learning modules that communicate and collaborate using a unified messaging protocol, enabling Monty to recognize and interact with objects efficiently through active exploration.

The key breakthrough is how Monty builds “reference frames”, 3D mental maps that capture not just what an object looks like, but how its parts relate to each other in space. As Monty’s sensors move across an object’s surface, each Learning Module constructs its own internal coordinate system, binding sensory observations to specific locations within that 3D framework. This process mirrors how multiple cortical columns in your brain simultaneously build complementary models of the same object from different perspectives and sensory inputs. Just as your visual cortex, touch sensors, and motor systems all contribute to your understanding of a toy block’s shape and properties, Monty’s different modules can vote and share information to quickly converge on accurate object recognition and pose estimation. The result is an AI system that develops robust, structured representations through exploration and understands objects not as collections of pixels, but as coherent three-dimensional entities with meaningful spatial relationships, exactly as biological brains do.

Monty learns to recognize new objects by actively exploring their surfaces or visually scanning them, building structured internal representations that allow it to efficiently integrate and update knowledge from different viewpoints.

The Results Are Impressive!

The efficiency gains achieved by Monty are quite impressive. State-of-the-art Vision Transformers require millions of training images and massive computational resources; however, Monty achieves comparable performance with just 600 training samples. That’s nuts! It’s one hundred times smaller than the MNIST digit dataset that researchers use as a basic benchmark. Monty accomplishes this learning feat using 34,000 times fewer computational operations than Vision Transformers trained from scratch, and an almost incomprehensible 528 million times fewer operations than pre-trained transformers that have already digested 14 million images. Monty also solves one of deep learning’s most persistent problems: catastrophic forgetting. While traditional neural networks often overwrite previous knowledge when learning new tasks (like a student who suddenly forgets algebra the moment they start studying geometry), Monty can continuously learn new objects without losing its understanding of previous ones, just as we naturally do throughout our lives. No other AI systems today continuously learn at such a high rate.

Monty excels at both classification accuracy and pose prediction after limited training by selectively updating only relevant areas during learning and intelligently exploring objects during inference.

The performance results show a robustness that current AI systems can only dream of achieving. Monty is 98.6% accurate on object recognition even when researchers deliberately introduced noise, presented objects at completely novel orientations never seen during training, or subjected the system to other challenging conditions that typically break conventional AI. When researchers changed every object to the same uniform blue color (removing all texture and color cues that deep learning systems typically rely on) Monty still achieved 73.1% accuracy by focusing on the fundamental structural properties that actually define objects. Unlike deep learning systems that often latch onto superficial patterns like texture or background details, Monty emphasizes shape and spatial relationships, just like how we recognize a heart symbol whether it’s red, blue, or any other color. The system even demonstrates an emergent understanding of symmetry and automatically detects when objects look identical from different angles. It’s a sophisticated geometric insight that emerges naturally from its sensorimotor learning process rather than being explicitly programmed.

Monty recognizes objects very efficiently by selectively updating only relevant areas during learning and intelligently exploring objects during inference.

Monty’s Applications

Monty democratizes artificial intelligence development by reducing the resource barriers that have concentrated AI power in just a few tech giants. When AI systems can learn from hundreds rather than millions of examples, suddenly universities, startups, researchers in developing countries, and specialized industries can build powerful AI without needing massive datasets, enormous computing clusters, or teams of hundreds of engineers. AI development becomes about accessibility and innovation speed. Monty’s robustness means we can finally build AI systems that work reliably in the messy, unpredictable real world rather than just on carefully curated datasets. No more brittle systems that fail when lighting changes or objects appear at unfamiliar angles. Monty’s continual learning capability also solves one of AI’s most fundamental problems: the need to retrain entire systems from scratch every time new requirements emerge. Instead of replacing AI models, we can now develop systems that grow and adapt throughout their operational lifetime, accumulating knowledge and capabilities like living beings do. No more catastrophic forgetting!

Monty can reliably identify everyday objects (even under unfamiliar conditions) by actively sensing and moving around them.

The practical implications will be far reaching. In robotics, we’re moving from pre-programmed industrial arms to adaptive robots that learn to manipulate new objects through exploration. We could have robots in warehouses that can handle novel packages without months of retraining, or household robots that learn your kitchen layout and preferences through natural interaction. Medical applications could include diagnostic AI that adapts to individual patient variations, surgical robots that learn from each procedure, or rehabilitation devices that customize themselves to each patient’s unique needs and progress. The efficiency gains make sophisticated AI viable on edge devices for the first time, enabling smart cameras that understand scenes without cloud connectivity, autonomous vehicles that adapt to local driving conditions, or agricultural sensors that learn to identify crop diseases specific to individual farms. AI can become a distributed capability that can be embedded anywhere intelligence is needed, learning and adapting in real-time to local conditions and requirements.

The Bigger Picture: Rethinking Intelligence

What if our pursuit of bigger models, more data, and greater computational scale has been taking us down the wrong path entirely? For years, the dominant paradigm has been that intelligence emerges from scale so by building larger neural networks, feeding them more data, adding more compute, eventually human-level intelligence will emerge. I doubt this path will get us to AGI. Monty suggests that intelligence isn’t mainly about processing power or memorizing huge amounts of information; it’s about the interaction between perception, action, and learning that allows biological systems to understand the world through direct experience. Monty shifts the industry toward embodied cognition, which is the idea that true intelligence fundamentally depends on having a body that can move through and manipulate the physical world. When we strip away the sensorimotor foundation that shapes how we learn, we’re missing the very essence of intelligence. A child doesn’t learn what “heavy” means by reading definitions, they learn it by lifting objects, feeling resistance, and building an intuitive understanding through all of their physical interactions.

Monty groups objects into meaningful categories based on their shape and automatically identifies symmetrical orientations, like how we intuitively understand object structure.

Biological inspiration points to a different path forward for artificial intelligence; one that prioritizes understanding over memorization, exploration over data consumption, and adaptation over pre-training. Neuroscience still has many lessons to teach computer science, particularly about how brains achieve their efficiency, robustness, and flexibility through local learning rules, modular architecture, and continuous sensorimotor integration. Rather than building ever-larger monolithic systems that consume internet-scale data, we might instead focus on creating AI that learns like living beings do. We learn through exploration, active hypothesis testing, and the gradual construction of understanding through interaction with the world. Instead of trying to encode intelligence into static parameters, we would build systems capable of developing intelligence through experience, creating AI that truly understands the world in which it operates. New AI systems could learn language through conversation, develop motor skills through practice, and build scientific understanding through experimentation. Just like we do.

What’s Next?

While Monty’s initial results are most impressive, it’s important to recognize that this research represents just the opening chapter of a much larger story. Currently, Monty has only been tested on 3D object perception, which requires recognizing and understanding the pose of household items. This first test is admittedly a narrow slice of intelligence, but it is a convincing one that demonstrates the fundamental principles of sensorimotor learning. The immediate research horizons are already tantalizing, with hierarchical learning systems where multiple Monty-like modules work together to understand complex, multi-part objects. We will see true multi-modal integration that combines vision, touch, sound, and other senses and the development of complex behaviors that go beyond recognition to manipulation, planning, and goal-directed action. Future Monty systems won’t just recognize a screwdriver but will understand how to operate it. They can also learn to navigate complex environments by building spatial maps through exploration. The researchers acknowledge that many fundamental challenges remain unsolved, including modeling objects that move and change, representing compositional hierarchies, coordinating action policies that actually change the world, and scaling from supervised to fully unsupervised learning.

Monty can rapidly and accurately identify objects by allowing multiple sensory inputs to collaboratively share predictions and “vote,” speeding up object recognition as more sensors are used.

The Thousand Brains Project has the goal of achieving artificial general intelligence through biologically-inspired sensorimotor learning, creating AI that can learn and reason about abstract concepts, develop scientific understanding through experimentation, and adapt to entirely novel domains. Perhaps most importantly for the pace of progress, Monty is completely open source, available on GitHub under an MIT license along with all the code needed to replicate the experiments. This democratization of innovative AI research means that hundreds of researchers worldwide can begin building upon these foundations, exploring new applications, testing variations, and pushing the boundaries of what’s possible. The combination of a fundamentally new approach to AI, impressive initial results, and open accessibility could create a wave of innovation that accelerates the path toward truly intelligent machines that learn, adapt, and understand the world with the efficiency and robustness of biological intelligence.

Conclusion

Monty embodies a shift from the passive data consumption that has defined modern machine learning to the active, exploratory intelligence that characterizes living beings. As the AI field increasingly confronts the diminishing returns of simply scaling up existing approaches, where each new breakthrough requires exponentially more data, compute, and energy, biological insights like those underlying Monty offer a different path forward. AI systems will learn with child-like efficiency from minimal examples, remain robust in the face of real-world complexity, and grow and adapt continuously throughout their operational lifetime without forgetting what they’ve learned. Monty lays the groundwork for artificial intelligence that understands the world through experience rather than memorization, that develops intuition through interaction rather than pattern matching, and that exhibits the kind of flexible, generalizable intelligence we see in biological systems. If Monty’s approach proves as transformative as these early results suggest, we may be witnessing the beginning of a new era in AI that genuinely comprehends the world around them. The code is open, the biological principles are proven, and the potential is limitless. The question now becomes how quickly we can build upon this foundation to create truly intelligent machines that have remained just beyond our reach.