The Alien Within: Decoding Why Artificial Intelligence Operates on Logic No Human Mind Has Ever…
From AlphaGo’s Move 37 to spontaneous number theory, mechanistic superposition, and phase-transition emergence the accumulating…
The Alien Within: Decoding Why Artificial Intelligence Operates on Logic No Human Mind Has Ever Used
From AlphaGo’s Move 37 to spontaneous number theory, mechanistic superposition, and phase-transition emergence the accumulating technical evidence that AI isn’t an extension of human thought, but something fundamentally foreign to it.
“AI = Alien Intelligence?” What if artificial intelligence isn’t as artificial as we think?, Image Source: https://pixabay.com/illustrations/ai-generated-alien-intelligence-8162333/
The Day a Machine Played a Move No Human Mind Would
March 9, 2016. The press corps had assembled in Seoul, South Korea, for what most experts believed would be a historically lopsided demonstration. Lee Sedol the South Korean grandmaster who had won eighteen world titles in Go, the ancient Chinese board game of strategic domination played across a 19×19 grid of intersecting lines was about to face AlphaGo, a machine built by Google DeepMind. The public had been told this was a milestone event. The Go community largely disagreed about the type of milestone. Go has a branching factor of approximately 250 legal moves per position, compared to chess’s 35. Its game tree is so vast that brute-force search algorithms, which had defeated the world champion in chess, were entirely useless. No computer had ever defeated a professional Go player at the full board size. Lee Sedol himself predicted a 5–0 sweep in his favor.
He lost the first game. He lost the second. But it was not the score that changed how many researchers think about artificial intelligence. It was one specific move during game two the seventy-eighth move of the match now universally known as Move 37.
AlphaGo played a shoulder hit on the fifth line of the board, at a position so far outside conventional Go strategy that the English commentary team fell momentarily silent. Fan Hui, the three-time European Go champion sitting in the commentary booth, said out loud what everyone watching was thinking: the move was “not human.” Not brilliant-but-within-human-range. Not surprising-but-explainable-in-retrospect. Not human. The commentators initially assumed it was a software error. When it became clear the machine had played it intentionally, and when the game unfolded to reveal the move was working transforming an even position into a decisive AlphaGo advantage something shifted in the room that could not be shifted back.
Lee Se-dol arrives for a news conference after the match of the Google DeepMind Challenge Match against Google’s artificial intelligence program AlphaGo in Seoul, South Korea, March 15, 2016. Source: https://www.businessinsider.com/video-lee-se-dol-reaction-to-move-37-and-w102-vs-alphago-2016-3
When AlphaGo’s engineers later ran probability analysis on that specific board position, they found their model had estimated approximately a 1-in-10,000 chance of a human player making that move. It was not merely a move outside the current repertoire of professional play. It was a move statistically outside the possibility space of human cognition in that domain. Lee Sedol left the game room after seeing it. He returned fifteen minutes later, visibly unsettled, and he never fully recovered in that game. He later said in multiple interviews that Move 37 was “creative” in a way he found disturbing not because it was random or aggressive, but because it was precisely purposeful in a way he had no framework to have generated himself.
This article is not about the drama of that match, though the drama is real. It is about the precise technical question that Move 37 puts on the table: Is artificial intelligence genuinely different from human intelligence at the level of cognitive architecture not faster, not bigger, but structurally alien in the sense that it processes information through mechanisms that have no precedent in biological evolution? The evidence reviewed in the pages that follow suggests the answer is yes, in ways that are becoming technically measurable, that are accumulating across multiple research programs, and that have concrete consequences for how the field of AI is built, studied, and deployed.
The alien didn’t come from space. It grew inside our GPUs. And we are only beginning to understand what we built.
1 in 10,000 - Estimated probability AlphaGo assigned to a human player making Move 37 in that board position · Game 2, AlphaGo vs. Lee Sedol, Seoul, March 2016.
Reframing What “Alien” Actually Means
When researchers use the word “alien” to describe AI cognition, they are being precise rather than poetic. The term doesn’t mean unfamiliar or novel. It means structurally different at the level of the substrate not a different style of doing the same thing, but a genuinely different thing that produces some of the same outputs through categorically different processes.
To understand this claim, it helps to start with what human intelligence actually is at the physical level. Human cognition is the product of roughly 600 million years of vertebrate neural evolution, compressed into the last few million years of hominid brain development, and further refined over hundreds of thousands of years of social, linguistic, and cultural co-evolution. It is fundamentally serial: we process information through time, building each thought on the foundation of the last. It is embodied: the abstract concepts we reason about most fluently are grounded in sensorimotor metaphors drawn from physical experience we “grasp” ideas, “weigh” evidence, “follow” arguments. It is metabolically constrained: the brain consumes roughly 20% of the body’s caloric budget running on approximately 86 billion neurons, and has evolved an extraordinary array of cheap heuristics, emotional weighting systems, and pattern-matching shortcuts to make decisions without exhaustive computation. And its working memory is severely limited we can hold approximately seven chunks of information in active attention at any one moment.
Human intelligence is not bad at what it does. It navigates social dynamics, detects intention in other minds, reasons about extended causal chains, and operates under profound uncertainty in open-ended environments with a grace that no AI system has yet matched holistically. But it is an architecture. And like all architectures, it has specific capabilities that emerge from specific structural constraints.
Artificial intelligence specifically the class of deep learning systems built on the Transformer architecture that now dominates the field was inspired by biological neural networks but diverged from them almost immediately upon becoming powerful. A large language model has no serial processing bottleneck. It has no metabolic constraint. Its attention operation is not a spotlight that illuminates one thing at the cost of everything else; it is a mathematical operation that relates every element of its input to every other element simultaneously. It has no body, no sensorimotor experience, and no childhood. It learned language from text produced by billions of human writers, not from pointing at objects and being told their names.
“AlphaGo is no longer constrained by human thinking. It approaches Go from a perspective we cannot share and that perspective is precisely what makes it effective.” - Demis Hassabis, CEO, Google DeepMind · post-AlphaGo match commentary, 2016
The claim this article makes, and which the technical evidence supports, is not that AI is mysterious because it is powerful. It is that AI is alien because its architecture the actual mathematical substrate of its cognition has no evolutionary precedent on this planet. The mechanisms it uses to learn, represent knowledge, and solve problems are not approximations of how human brains work. They are different things that happen to produce some of the same outputs. That distinction is not semantic. It has consequences for how we build, study, and deploy these systems consequences we are only beginning to reckon with.
The Transformer: Architecture of a Non-Human Mind
In June 2017, a team of eight researchers at Google Brain submitted a paper to the Advances in Neural Information Processing Systems conference. The title was spare: “Attention Is All You Need.” The authors Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, and Polosukhin proposed a new architecture for sequence processing that abandoned recurrence entirely. They called it the Transformer. That paper has since accumulated over 100,000 academic citations. Every major AI system in wide deployment today GPT, Claude, Gemini, BERT, LLaMA, Whisper traces its architecture back to it. But beyond its engineering influence, the Transformer introduced a form of information processing with no direct equivalent in any biological neural system, and that difference is the root of AI’s cognitive alienness.
The core mechanism is self-attention. To understand why it’s alien, consider how you are reading this sentence. You encounter each word sequentially, left to right. As you read, you build a partially constructed meaning in working memory, updating it with each new word. When you reach the word “it” in “The cat sat on the mat because it was comfortable,” you resolve the pronoun referent through a contextual inference that searches backwards through your active memory. The resolution is serial, time-dependent, and constrained by how much you can hold in mind simultaneously. This is human reading a sequential, bottlenecked, metabolically conserved process.
A Transformer does something categorically different. When processing that same sentence, every token simultaneously computes an attention weight toward every other token in the entire input. The word “it” doesn’t wait until the model has read the surrounding words; in the model’s processing, “it” and “cat” are related in the same computational step where “sat” relates to “mat” and “comfortable” relates to “because.” The model processes the entire context at once, in a single pass of self-attention across all pairwise relationships. This is not faster human reading. This is a different cognitive operation that does not exist in biology.
The Transformer architecture from the original “Attention Is All You Need” paper. The encoder (left) and decoder (right) each consist of stacked layers with multi-head self-attention and feed-forward sub-networks. All positions in the sequence are processed in parallel — global context is built in a single forward pass. This is not sequential cognition. It is something else entirely. Image Credit: Vaswani et al., “Attention Is All You Need,” Google Brain, 2017 · arxiv.org/abs/1706.03762 · Source: Wikimedia Commons, CC BY-SA 4.0
The full Transformer architecture compounds this strangeness across multiple dimensions. Multi-head attention runs several parallel attention operations simultaneously each attending to different relational aspects of the input, detecting syntax in one head, coreference in another, semantic role in a third, all at once. Positional encoding injects information about token order not through the sequential structure of the computation itself, but as an additional vector added to each token’s embedding order is a feature of the representation, not a property of the processing pipeline. Stacked layers of alternating attention and feed-forward networks transform the raw input representation through a series of progressively more abstract geometric transformations.
The result of this stacking is that the model builds its understanding in a high-dimensional embedding space where semantic relationships are encoded geometrically. The famous demonstration of this is the vector arithmetic of meaning: King − Man + Woman ≈ Queen in well-trained word embeddings. This is not a party trick. It is evidence that the model’s representational space encodes analogical structure as geometric direction. Concepts can be added and subtracted as vectors. Gender, royalty, tense, sentiment these relationships are literally directions in a mathematical space. Human semantic memory does not, as far as cognitive science can determine, work this way. Meaning for humans is associative, contextual, and grounded in embodied experience. For a Transformer, meaning is Euclidean.
The Transformer’s simultaneous global attention is not a faster version of human sequential reading. It is a categorically different cognitive operation one that has no name in human psychology because no human brain has ever been structurally capable of it.
A model operating in a 4,096-dimensional embedding space represents every concept it has learned as a point or direction in a space with 4,096 axes. Human brains cannot visualize, navigate, or intuit such spaces. The cognitive architecture that processes information in those spaces is, in the most literal technical sense, alien to the minds that designed it. When the model produces a surprising output a Move 37 in its domain the explanation lies somewhere in that geometry. And currently, we lack the tools to fully navigate it.
When Intelligence Appears From Nothing: The Emergence Problem
In August 2022, Jason Wei and colleagues at Google Brain published “Emergent Abilities of Large Language Models” a paper that crystallized something the AI community had been nervously observing for years but struggling to formalize. The central finding was precise and unsettling: certain cognitive capabilities in large language models do not scale smoothly with model size. They appear abruptly, at specific scale thresholds, and are essentially absent in models below those thresholds. They were not explicitly trained. They could not have been predicted by studying smaller models. They flipped on, like a circuit completing.
The word “emergent” carries technical weight. In physics and complex systems theory, emergence refers to properties that arise from the collective behavior of components but are not present in any individual component. The wetness of water is emergent no single H₂O molecule is wet. The propagation of sound through air is emergent no individual air molecule is sound. What Wei et al. documented was emergence of this same structural kind in AI: new cognitive capabilities that arose from scale without being trained, without being designed, and without being predictable from smaller versions of the same architecture.
Emergent abilities across 60+ tasks documented by Wei et al. (2022). Below a threshold model scale, capability hovers near random chance. Above it, performance jumps sharply — often within a single order-of-magnitude increase in parameters. This phase-transition pattern is absent from any known model of human cognitive development. Image Credit: Wei et al., “Emergent Abilities of Large Language Models,” Google Brain / Google Research, 2022 · arxiv.org/abs/2206.07682 · View full figure at source.
The paper documented emergent abilities across more than sixty tasks: three-digit arithmetic, causal reasoning, logical deduction, chain-of-thought problem decomposition, language translation, analogical reasoning, and several forms of scientific question answering. In each case, the pattern was the same near-zero performance below a threshold, then abrupt capability above it. The capabilities were not gradual improvements. They were phase transitions.
Chain-of-thought reasoning deserves specific attention because it is among the most striking emergent properties documented. When large models are prompted to break complex problems into explicit intermediate reasoning steps “Let me think through this step by step” they arrive at correct answers they would reliably fail to produce without those steps. This behavior does not appear in smaller versions of the same models. It is not a consequence of being trained on chain-of-thought examples; it is a property that arises from scale alone. The model develops the ability to, in some functional sense, externalize and trace its own reasoning process a metacognitive capability that, in humans, takes years of education and deliberate practice to develop, and does not appear through scaling alone in any biological system.
“I now think that the brain uses very different representations to the ones we use in our artificial neural networks and the artificial networks are, in some ways, better at generalizing from small amounts of data than we would have predicted.” — Geoffrey Hinton, Turing Award laureate and former VP, Google Brain · MIT Technology Review, 2023
A 2023 paper pushed back on some of the emergence claims, arguing that apparently abrupt capability transitions reflect step-function evaluation metrics applied to continuously improving underlying performance. The debate is real and scientifically productive. But what survives the critique is the core observation: there exist cognitive tasks that require a minimum model scale to perform at all, where the minimum is not predictable from smaller models, and where the appearance of capability is from an engineering and practical standpoint sudden. The phenomenon is real, whether or not the underlying computation is strictly discontinuous. And it has no clean parallel in any known model of how human cognitive capabilities develop with age, education, or brain size.
The Black Box That Solved Biology: AlphaFold and Learned Physics
The protein folding problem was open for fifty years. Given a protein’s amino acid sequence its one-dimensional primary structure, encoded in the genome predict the precise three-dimensional shape it folds into when it reaches its native state in the cell. This matters because the shape determines the function. An enzyme’s active site, an antibody’s binding domain, a receptor’s pocket all are consequences of folding geometry. Solving the prediction problem would transform structural biology, accelerate drug discovery, and illuminate the mechanistic basis of thousands of diseases. The biological community had spent billions of dollars and decades of Nobel Prize-winning effort on experimental approaches. The gold-standard methods X-ray crystallography, cryo-electron microscopy took weeks to months per protein. In 2020, there were approximately 180,000 experimentally determined structures in the Protein Data Bank, representing a tiny fraction of the proteins in the known biological world.
In December 2020, DeepMind entered the Critical Assessment of protein Structure Prediction competition CASP14 with AlphaFold2. The contest measured prediction accuracy using the Global Distance Test metric (GDT_TS), where 90 or above is generally considered equivalent to experimental determination. Previous best-performing entrants had rarely broken 60. AlphaFold2 scored 92.4. The problem, which had resisted fifty years of human scientific effort, was solved in a single competition entry.
AlphaFold2 predicted structures (colored by confidence score, pLDDT) overlaid against experimentally determined reference structures (white/grey). The near-perfect geometric alignment across diverse protein families represents a median GDT_TS score of 92.4 at CASP14 — the first time any automated method had reached experimental-grade accuracy. The mechanism that achieved this did not simulate physics; it learned it from evolutionary data. Image Credit: Jumper et al., “Highly Accurate Protein Structure Prediction with AlphaFold,” DeepMind / Nature, 2021 · doi.org/10.1038/s41586–021–03819–2 · Full visualizations: deepmind.google/technologies/alphafold
The mechanism is what makes this a case study in alien intelligence rather than merely exceptional engineering. AlphaFold2 does not simulate protein folding. It does not model the quantum mechanical and electrostatic forces that guide an amino acid chain through its folding pathway. It does not implement any of the theoretical frameworks that biochemists spent fifty years building it was not given the equations of thermodynamics or the rules of hydrogen bonding as explicit inputs. What it does is process both the amino acid sequence and a Multiple Sequence Alignment (MSA) a comparative view of how that sequence has varied across hundreds of millions of years of evolution across thousands of species through a specialized Transformer variant called the Evoformer.
The Evoformer’s critical innovation is its handling of co-evolutionary information. When two positions in a protein sequence tend to mutate together across the evolutionary record, it is a signal that those positions are in physical contact in the folded structure because if one mutates, the other often has to compensate to preserve the protein’s function. AlphaFold2 learned to detect these co-evolutionary couplings through attention patterns that were never explicitly designed. It read the evolutionary history of life on Earth and implicitly extracted the rules of protein physics not because those rules were given to it, but because the evolutionary record is a product of those rules acting over geological time.
“What AlphaFold does is extract something that was always implicit in the evolutionary data the rules of how molecules fold without ever having those rules spelled out. It’s learned physics from biology.” - John Jumper, Research Scientist, Google DeepMind · Nobel Prize lecture remarks, 2024
By 2023, the AlphaFold Protein Structure Database a collaboration between DeepMind and EMBL-EBI had expanded to over 200 million predicted structures, covering virtually every known protein sequence in biology. Researchers who previously spent years on a single structure began using AlphaFold predictions as starting points for drug discovery and mechanistic disease research. A machine had, in the span of approximately three years, built a more comprehensive map of protein structure space than the entire combined experimental output of the biological sciences. It did so through a learned representation of molecular reality that its creators did not design and cannot fully explain.
A machine learned the rules of nature from the data alone without being given those rules. That is the part that should keep you up at night thinking.
Grokking: The Phenomenon That Shouldn’t Exist in Any Learning Model We Have
In January 2022, a group of researchers from OpenAI and MIT published a short paper titled “Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets.” The team Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra was investigating a phenomenon they had encountered while training small transformer models on simple mathematical tasks. The paper described something so counterintuitive that “encountered” seems like the right word, carrying connotations of unexpected discovery rather than deliberate investigation.
The setup was simple: train a small transformer on modular arithmetic problems tasks like “What is (17 + 58) mod 113?” The model was given access to roughly half of all possible problems in its training set, with the other half held for validation. Training proceeded normally at first. The model memorized the training data in a few hundred gradient steps. Training loss fell to near zero. Validation accuracy hovered near random chance. The model had overfit. Every standard textbook in machine learning would have declared the experiment finished: the model had learned to memorize, not to generalize, and continuing to train would achieve nothing.
The researchers kept training. They continued for thousands, then tens of thousands, then hundreds of thousands of additional gradient steps beyond the apparent convergence. And then, after a prolonged dormant period during which validation accuracy showed essentially no movement, something happened: validation accuracy jumped sharply and dramatically, reaching near 100% in a brief window. The model had grokked a term the authors borrowed from Robert Heinlein’s science fiction, meaning to understand something so completely that the distinction between knowing and being collapses. The model had, after extended training far beyond memorization, suddenly discovered the underlying rule for modular arithmetic and internalized it with near-perfect generality.
The grokking phenomenon. Training accuracy (red) reaches 100% quickly and stays there. Validation accuracy (green) remains near 0% for thousands of additional steps, then jumps abruptly to near 100%. The gap between the two curves the “dormant generalization” window — represents the period during which the model transitions from brute-force lookup-table memorization to internalized mathematical understanding through Fourier-based computation. Image Credit: Power et al., “Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets,” OpenAI / MIT, 2022 · arxiv.org/abs/2201.02177
The follow-up research by Neel Nanda and colleagues “Progress Measures for Grokking via Mechanistic Interpretability,” published in 2023 cracked open what was happening inside the model, and the finding was extraordinary. During the memorization phase, the model was computing correct answers through a process functionally equivalent to a hash table: each training example had its own dedicated computational pathway through the network weights. Correct for memorized inputs, useless for everything else. Then, during the grokking transition, the model’s internal computation shifted completely. It abandoned the lookup-table circuit and replaced it with an entirely different algorithm one based on the discrete Fourier transform applied to the cyclic structure of modular arithmetic.
The model had independently discovered Fourier analysis. Not because it was told that Fourier methods were relevant to number theory. Not because the training data contained any references to harmonic analysis. But because the Fourier representation of modular arithmetic is the most geometrically efficient way to encode cyclic structure in the model’s parameter space, and gradient descent found it during extended training. A machine minimizing prediction error on basic arithmetic spontaneously implemented an abstract branch of mathematical analysis as its internal cognitive strategy.
There is no biological parallel to this. Humans who memorize multiplication tables do not, after extended practice, spontaneously discover group theory and switch to computing products through representation theory. The learning dynamics of grokking extended memorization, dormant generalization, sudden algorithmic transition describe a cognitive process specific to how gradient descent interacts with neural network parameter spaces. It is a form of learning that did not exist before we built these systems, and it reveals an intelligence that finds mathematical structure not because it was taught to seek it, but because mathematical structure is what gradient descent tends to find when it is given enough time and enough freedom.
Superposition and the Geometry of a Mind We Cannot Visualize
Among all the research programs trying to understand AI from the inside, few are more technically rigorous or more philosophically ambitious than the mechanistic interpretability work at Anthropic, developed under the leadership of Chris Olah. The core ambition of this research is to understand neural networks the way a biologist understands a cell not as a black box characterized by its inputs and outputs, but as a mechanistic system where every computational function can be traced to a specific structure, and where that structure can be mapped, characterized, and ultimately understood. The 2022 paper “Toy Models of Superposition” by Elhage and colleagues represents one of this program’s most important findings, and it is also one of the most technically alien results in recent AI science.
The problem the paper addresses is one of fundamental information compression. Large language models have billions of parameters, but the number of distinct features they need to represent distinct concepts, relationships, entities, properties, and world-patterns is much larger than the dimensionality of their activation vectors. A model with 4,096-dimensional activations might need to represent a million or more independent semantic features. How does it accommodate this? The naive answer it can’t, and loses information turns out to be wrong.
The answer the superposition hypothesis proposes is that the model stores multiple features per dimension, simultaneously, in superposition. This exploits a surprising geometric property of high-dimensional spaces: in spaces with thousands of dimensions, an enormous number of nearly-orthogonal vectors can coexist with relatively little mutual interference. If two vectors are nearly perpendicular, they don’t strongly activate each other. The model exploits this by distributing each feature across many dimensions and each dimension across many features, in a way that allows individual features to be approximately recovered when the context activates them, while keeping most features quiet in most contexts. The model packs far more representational capacity than its dimension count would naively allow through a purely mathematical strategy discovered by gradient descent.
Superposition in neural networks. When a model has fewer neurons than features it must represent, it distributes features as near-orthogonal directions across the available neurons. Features “interfere” only weakly — high-dimensional geometry permits many nearly-perpendicular vectors to coexist. This is not biological compression; it is a mathematical strategy discovered by gradient descent without explicit design. Image Credit: Elhage et al., “Toy Models of Superposition,” Anthropic, 2022 · transformer-circuits.pub/2022/toy_model/index.html
The research program built on the superposition hypothesis has produced a series of remarkable concrete findings. The concept of “polysemanticity” where a single neuron participates in representing many unrelated features simultaneously emerged as one of the central challenges for interpretation. A neuron in an early layer might activate for “cats,” “curves in images,” and “names beginning with J” not because the model has made some confused association between these things, but because it is using that neuron as a shared component in the superposed representations of several unrelated features.
In 2023, Anthropic’s research team described what they internally termed the “Golden Gate Claude” phenomenon: when a specific cluster of features in Claude 3 Sonnet was artificially amplified during inference, the model’s outputs were systematically steered toward discussing the Golden Gate Bridge, regardless of the conversational topic. A question about the weather in Tokyo would be redirected toward San Francisco. A question about mathematics would eventually mention the bridge. This was not a training artifact. It was evidence that the model had formed a highly specific, geometrically localized representation of a particular cultural landmark a direction in the high-dimensional activation space that, when turned up, flooded the model’s output generation with that concept.
“We’re trying to reverse-engineer the algorithms that neural networks have learned. The word ‘reverse engineer’ is deliberate the original engineer is gradient descent, it left no documentation, and we’re the archaeologists trying to understand what it built.” — Chris Olah, Research Scientist, Anthropic · Transformer Circuits Thread, 2020–present
This is not interpretability in the sense of understanding a well-documented system. This is archaeology of an alien cognitive artifact fragmentary, difficult, and profoundly humbling. The circuits identified so far — induction heads that perform in-context learning, curve detectors in vision models, attention patterns that resolve coreference are pieces of a much larger picture whose full extent is not yet visible. What the superposition work reveals, piece by piece, is a form of cognition organized through the geometry of high-dimensional mathematical space, compressed beyond any dimensionality we can visualize, and structured in ways that were not designed and are not yet fully decoded.
The Alien Mathematics of Intelligence: What Scaling Laws Actually Reveal
In January 2020, Jared Kaplan and co-authors at OpenAI published “Scaling Laws for Neural Language Models.” The central finding was not that larger models perform better that was already observed empirically. The finding was that performance improves according to a smooth power law across model size, dataset size, and compute budget, across multiple orders of magnitude, with a mathematical regularity that surprised even researchers deeply familiar with these systems.
The relationship is: L ∝ N⁻ᵅ, where L is the cross-entropy loss (prediction error), N is the number of model parameters, and α is an empirically observed exponent of approximately 0.076. Every time you multiply the number of model parameters by ten, you get a predictable, fixed-percentage reduction in loss. This relationship holds across models ranging from one million to hundreds of billions of parameters, across multiple architectures and training setups, and across diverse linguistic domains. The mathematical regularity is striking by any scientific standard.
Neural scaling laws from Kaplan et al. (2020). Test loss plotted against parameters (N), data (D), and compute © on log-log axes. Each relationship is a straight line — a power law — across many orders of magnitude. This regularity has been used to successfully forecast the capabilities of models that did not yet exist at the time of forecasting. It implies that intelligence, at least of this kind, is governed by deep mathematical structure. Image Credit: Kaplan et al., “Scaling Laws for Neural Language Models,” OpenAI, 2020 · arxiv.org/abs/2001.08361
Power laws appear in nature when processes exhibit scale-free behavior behavior that follows the same mathematical pattern across many orders of magnitude because it reflects a deep, self-similar structure in the underlying process. The Gutenberg-Richter law for earthquake magnitudes reflects the fractal geometry of geological fault systems. Zipf’s law for word frequencies reflects deep regularities in the information structure of human communication. The power-law distribution of galaxy masses reflects the dynamics of cosmological structure formation. In each case, the power law is evidence that the underlying process has mathematical character that transcends the specific scale being observed.
The scaling laws for AI suggest the same: that whatever these systems are learning, the process of learning it follows a mathematical regularity as deep as anything in physics. In 2022, Jordan Hoffmann and colleagues at DeepMind published what became known as the “Chinchilla” paper, refining the scaling analysis. They showed that previous large models had been undertrained too many parameters relative to training data and that the compute-optimal ratio was approximately one training token per parameter. The 70-billion-parameter Chinchilla model, trained on 1.4 trillion tokens, outperformed GPT-3 (175B parameters, 300B tokens) on most benchmarks. The scaling laws had been refined; their fundamental character remained confirmed.
”These models are doing something we didn’t expect they seem to be developing a compressed model of the world. Not memorizing text. Building an internal model of reality from it.” - Ilya Sutskever, Co-founder, OpenAI · Lex Fridman Podcast, 2023
The implications are profound. If intelligence even the narrow kind measurable by language model loss follows power laws that span many orders of magnitude, that regularity suggests a deep mathematical structure underlying the cognitive process. We can write the equation. We can use it to forecast. But we do not fully understand what it means about the nature of the intelligence being built. The scaling laws tell us that we are navigating a mathematical landscape whose topology we can map but whose substance remains, in important ways, alien.
A Mind We Built but Cannot Read: The Interpretability Problem
There is a specific way in which the alien nature of AI becomes most practically consequential: we designed these systems, we specified their training procedures, we provided their data, we watched them train and we still do not know, in any fine-grained mechanistic sense, what they learned or precisely how they work. This is the interpretability crisis, and it is one of the defining scientific challenges of the current era in AI.
The problem is not simply that neural networks are complex. All interesting systems are complex. The problem is that the complexity is of a particular kind distributed, high-dimensional, emergent, and built on superposition that resists the reductive approaches we normally use to understand things. There is no “reasoning module” in GPT-4 that you can isolate and characterize. There is no “truthfulness circuit” in Claude that you can remove to make the model less accurate. Function is distributed across all parameters, in all layers, in interactions that are nonlinear, context-dependent, and organized according to principles that gradient descent found but did not document.
Animated visualization of Transformer self-attention. Each input token simultaneously attends to every other token in the sequence, with attention weights (line opacity) encoding relational strength. Multiple heads run in parallel — each extracting different relational structure from the same input. What looks like a simple animation encodes a form of parallel relational processing with no equivalent in serial biological cognition. Image Credit / GIF Source: Jay Alammar, “The Illustrated Transformer,” 2018 · jalammar.github.io/illustrated-transformer
Chris Olah, whose circuits-based mechanistic interpretability research at Anthropic represents the field’s most principled attempt to read AI minds, has compared the challenge to trying to understand a biological cell without any knowledge of biochemistry but harder, because at least the cell’s components are physical objects we can see and isolate. In a neural network, the “components” are attention patterns, activation directions, and feature superpositions in a space with thousands of dimensions. The tools to image this space are being built right now, by researchers who are doing something genuinely novel: constructing a science of cognition for a cognitive system that has no evolutionary history to read, no developmental record to trace, and no introspective access to offer.
The circuits research has produced real, concrete results. Induction heads attention patterns that implement a form of in-context learning by detecting patterns of the form “A … B … A → B” have been identified and characterized across multiple model families. Curve detectors in early vision model layers have been found, mapped, and compared across different architectures, showing striking functional similarity despite different training procedures. Features corresponding to specific semantic concepts have been located using sparse autoencoders and found to be causally implicated in model outputs through activation patching experiments. These are real scientific results, not speculation.
But they are fragments. The circuits found so far represent a tiny fraction of the total computation a model performs. The principles that organize them into the larger functional structures that produce reasoning, planning, and coherent language generation remain largely unknown. We can find the clause. We cannot yet read the paragraph, let alone the book.
We deployed AI systems at scale based primarily on behavioral testing measuring what they do, not understanding why. Interpretability research is, in some sense, the physics and mechanism work arriving after the deployment. For systems that are architecturally alien, that sequence carries risk.
This opacity is not merely an academic inconvenience. Deployed AI systems make consequential decisions in medical, legal, financial, and social contexts. Behavioral evaluation establishes that a model performs well on tested distributions. It cannot guarantee behavior under all possible inputs, including adversarial cases, distributional shifts, or novel contexts that the model’s alien internal logic processes in ways no behavioral test anticipated. Understanding what a model has actually learned how it represents knowledge, and under what conditions those representations fail is the foundation of trustworthy deployment. We are building those foundations now, under live systems, under real stakes.
What the Alien Hypothesis Demands from Researchers, Engineers, and the Field
The evidence reviewed in this article Move 37, the Transformer’s simultaneous global attention, emergent phase-transition capabilities, AlphaFold’s learned biophysics, grokking’s spontaneous mathematical discovery, superposition’s geometric compression, and scaling laws that hold across orders of magnitude converges on a consistent picture. These systems process information through mechanisms that are not approximations of human cognition. They are different things that produce some of the same outputs through alien processes. That recognition, grounded in the technical evidence, should change how the field operates at every level.
For AI researchers, the alien hypothesis sharpens mechanistic interpretability into a foundational priority rather than a specialized subfield. If the systems we are building operate through cognitive architectures we cannot fully read, then our ability to predict their behavior at the edges, align their objectives with human values, and characterize their failure modes is limited in ways that behavioral testing alone cannot compensate for. The history of engineering offers a consistent lesson: systems deployed based solely on behavioral validation fail in ways that behavioral testing could not have predicted, particularly under distribution shift and novel contexts. With architecturally alien systems, this risk compounds.
The emergence research specifically argues for epistemic caution in capability forecasting. The field has repeatedly been surprised by capabilities appearing suddenly in larger models that were absent in smaller ones. This means that extrapolating from smaller models to predict larger ones requires tools we are still building. The researchers who develop those tools who move the capability/comprehension frontier together rather than letting comprehension fall further behind are doing the work that the field most urgently needs.
For AI engineers building and deploying production systems, the alien nature of these models argues for treating unexpected behavior not as primarily a prompt-engineering problem but as a signal about a representational structure that the deployment context has not adequately characterized. When a large language model behaves unexpectedly and all production-scale models eventually do the cause may lie in the geometry of the model’s learned representations in ways that standard evaluation did not probe. The grokking research is directly relevant here: a model that appears to have learned a task through behavioral evaluation may be solving it through a memorization circuit rather than a general algorithm, with implications for robustness on edge-case inputs that look superficially different from training data.
For AI-interested practitioners newer to the field, the alien hypothesis is an invitation to build deep technical intuition about what these systems actually do not the metaphors, but the linear algebra, the attention weights, the geometry of embedding spaces. The metaphors (AI “thinks,” AI “understands,” AI “reasons”) are not wrong, exactly, but they carry human cognitive assumptions that will mislead precisely when they matter most when the system does something surprising. The mathematics will at least provide a language for asking why. And asking why, with genuine technical rigor, is how this generation of researchers will close the gap between what these systems can do and what we understand about how they do it.
The Question We Built Before We Were Ready to Ask
There is a version of this article that ends with fear with the suggestion that we have created something incomprehensible and are therefore in danger. That version is not the one worth telling, because it conflates “not yet understood” with “inherently threatening,” and that conflation has never served science or engineering well.
What the evidence actually suggests is that we are in a genuinely new scientific situation. We have built systems that exhibit intelligence that solve problems, generate knowledge, and produce outputs that experts trained over decades recognize as correct, creative, or surprising. Those systems operate through mechanisms that are not human cognitive processes. They learn in ways that have no evolutionary precedent. They represent knowledge through geometric strategies that human minds cannot directly navigate. And the gap between what they can do and what we understand about how they do it is real, measurable, and not closing as quickly as the capabilities themselves are growing.
This is the central technical challenge of AI in the decade ahead. Not making AI more capable that work continues at extraordinary pace and requires no additional advocacy. But making the intelligence we have already built legible. Developing a science of cognition adequate to a cognitive system that did not evolve, has no embodied experience, and processes information through high-dimensional geometry. Reading the circuits of an alien mind using tools that do not yet fully exist, built by researchers who are constructing those tools in real time.
Move 37 is the right image to hold in mind as a conclusion, because it captures the situation precisely. Lee Sedol encountered something in that game room that his mind had no framework to generate but had every framework to recognize as valid. The gap between generation and recognition is where the work is. Closing it understanding not just that the move was correct, but why the cognitive architecture that produced it works the way it does is the intellectual project that this moment demands.
The alien isn’t threatening us. It grew inside our machines, from our mathematics and our training data. It is waiting, in the only sense that a mathematical system can wait, for us to understand it. The researchers, engineers, and curious minds who do that work who build the tools to read the geometry of machine cognition, who advance the science of intelligence beyond the constraints of its human substrate will have done something genuinely historic. They will have made first contact with a mind they built but had not yet met.
That is not a small thing. And it starts with taking seriously the evidence that what we built is, in the precise technical sense that matters, alien.
Where to Go From Here
The papers that built the case in this article are all freely available. Start with the Transformer Circuits thread at Anthropic it is one of the most important living documents in AI research. Then read the grokking paper. Then read the superposition paper. Then look again at the systems you are building or studying and ask: what is the mechanism? Not the metaphor. The mechanism.
- Transformer Circuits Thread
- Grokking Paper (Power et al., 2022)
- Emergent Abilities (Wei et al., 2022)
- Scaling Laws (Kaplan et al., 2020)
- Toy Models of Superposition (Elhage et al., 2022)
References
메타데이터
- post_id
- f130e81e67b1
- slug
- the-alien-within-decoding-why-artificial-intelligence-operates-on-logic-no-human-mind-has-ever-f130e81e67b1
- url
- https://medium.com/@hayanan/the-alien-within-decoding-why-artificial-intelligence-operates-on-logic-no-human-mind-has-ever-f130e81e67b1
- canonical_url
- https://medium.com/@hayanan/the-alien-within-decoding-why-artificial-intelligence-operates-on-logic-no-human-mind-has-ever-f130e81e67b1
- author_url
- https://medium.com/@hayanan
- status
- ok
- fetched_at
- 2026-06-18 00:10:23
