Reflections on Trusting Trust: forty years late to the party

How Ken Thompson showed in 1984 that auditing source code is not enough. And why that still holds.

I was watching Samuel Aubertin’s talk at BSDCan. He applies the concept to Docker: poison a base image so that a backdoor silently propagates into every image built on top of it, without leaving any trace in the Dockerfiles.

Watching that, I realized it was referring to a paper I had heard about a few times without ever sitting down to actually read it: *Reflections on Trusting Trust*, by Ken Thompson, 1984. I had known the name for years. Never bothered to dig in.

I finally did, and I found it passionate, especially after reading this line from the conclusion:

You can’t trust code that you did not totally create yourself. No amount of source-level verification or scrutiny will protect you from using untrusted code.

Forty years later, it has not aged a day.

The man who shaped modern computing as we know it

Trying to summarize Ken Thompson’s impact without underselling it is harder than it sounds.

He co-created Unix with Dennis Ritchie at Bell Labs in the late 1960s. Unix became the foundation on which ARPANET, and then the Internet, were built. The first TCP/IP implementations ran on Unix, and the network culture and tooling that followed grew directly out of that environment. Along the way he also co-created the B language, the direct predecessor of C, designed the first Unix shell, the first ed(1) editor, and laid the architectural groundwork that most modern operating systems still rest on.

Less well known but just as consequential: Plan 9, the research operating system he co-developed at Bell Labs in the 1990s. Plan 9 pushed Unix ideas to their logical limits, treating everything as a file, generalizing namespaces, and rethinking how system resources are distributed. Concepts that seemed too radical at the time and that you now find at the core of containers, Linux namespaces, and the way modern distributed systems organize their resources.

He is also co-creator of Go with Rob Pike and Russ Cox, a language built for the cloud and concurrency era, now embedded in a large share of the world’s network infrastructure.

In 1983, the ACM awarded him the Turing Award jointly with Dennis Ritchie. And in the lecture that came with that award, he chose not to celebrate his own work. He chose to demonstrate an attack against the trust chain running through everything he had helped build.

The attack nobody sees coming because it leaves nothing to see

The idea is simple to explain and hard to fully absorb.

Say you want to plant a backdoor in login(8), the binary that handles password authentication on a Unix system. The obvious approach is to modify login.c, but that is also the obvious place to look: any source code audit would catch it immediately.

Thompson does not touch login.c. He modifies the C compiler, the tool that transforms source code into executable programs. He teaches it two behaviors:

1. Recognize when it is compiling login.c, and silently inject the backdoor into the resulting binary, without it appearing anywhere in the source.
2. Recognize when it is compiling itself, and reproduce both behaviors in the next version of the compiler.

He installs the modified compiler, restores the original source to its clean state, and recompiles. From that point on, the source is spotless. Any audit finds nothing. And yet every login(8) binary produced on that machine contains a working backdoor, indefinitely.

The compiler: top of the chain, blind spot of security

In a classic Unix ecosystem, the C compiler sits at the top of everything. It compiles the operating system. It compiles the build tools. It compiles other compilers. Virtually all the software running on your machine has passed through it at some point.

The implicit assumption behind the whole open source culture, that source code is the ground truth, collapses the moment that tool is compromised. You can audit every line of every project, do exhaustive code reviews, publish everything under the GPL: if the compiler you use to build it all is malicious, you are looking in the wrong place. The betrayal happens outside the perimeter you are watching.

That line from the conclusion, the one I quoted at the top, lands differently once you have worked through the mechanism. This is not a library vulnerability that a patch can fix. It is a structural problem with what it means to trust a system you did not build from end to end, and that problem does not have a neat answer.

How the compiler learns, and why it never forgets

The attack stays alive through self-replication. That is where a concept most people encounter as a programming curiosity becomes load-bearing: the quine.

A quine is a program that produces its own source code as output when it runs. Classic recreational exercise. But for Thompson it is the whole mechanism.

For the compiler to reproduce itself without its malicious code ever appearing in the sources, it has to carry its own description inside itself. When it recompiles its own clean source, it still needs to inject the malicious code into the resulting binary. It cannot point to external code because that code no longer exists. So it describes itself as embedded data and knows how to reconstruct itself from that description. The digital DNA of the poison.

In 2023, Russ Cox obtained the original nih.a archive directly from Thompson, the real code, preserved since 1975. His full analysis is available online. The complete attack fits in 99 lines of C and a 20-line shell script. Both reassuring, because it took real genius to think of it, and troubling, because now that it is explained, anyone can reproduce it.

Cox also mentions that Thompson told him he had been waiting for decades for someone to ask whether he still had the code. Nobody had asked. He still had it.

The Air Force, Multics, and an idea waiting to be demonstrated

Thompson does not claim to have invented the concept. In the paper, he points to a direct source: a security report commissioned by the US Air Force in 1974, covering an audit of Multics, the Bell Labs/MIT/GE operating system that preceded Unix.

That report had already raised the question: how do you trust a compiler delivered by a third party? The auditors noted that inspecting the delivered binary is not enough, and that the compilation chain itself is a potential attack vector. The concept was on paper. Nobody had put it into practice.

Thompson did. He deployed the attack in a real environment, inside Bell Labs. It worked. It was eventually detected not through a code audit, but because the compiler grew slightly in size each time it recompiled itself, a bug in his own implementation. The team that spotted the anomaly never understood what had actually happened.

Ten years between the Air Force report and the working demonstration. Nobody had connected the dots.

The concept is unchanged. The attack surface has exploded.

More than forty years have passed. Languages have changed. Build pipelines have grown into something of staggering complexity: cross-compilers, Docker containers, CI/CD chains, package registries, supply chains that span dozens of transitive dependencies and hundreds of anonymous contributors.

Thompson’s concept has not aged at all.

In 2023, Russ Cox rewrote the equivalent for the Go compiler in a few dozen lines. A backdoored Go compiler can reproduce itself across successive compilations from clean sources, leaving no trace. Nothing has fundamentally changed since 1975.

And that is exactly what Samuel Aubertin demonstrates at BSDCan. The vector is no longer a 1970s C compiler. In his talk, he applies the same reasoning to make(1), Docker, and other tools that the entire software construction process quietly depends on. A poisoned Docker base image propagating into all child images without any trace in the Dockerfiles, a compromised make(1) altering produced binaries without touching the Makefiles: same principle, new surfaces, tools that thousands of organizations run in production today without a second thought.

SolarWinds (2020) and XZ Utils (2024) show how far this carries in practice. For SolarWinds, attackers compromised the build pipeline of the Orion network monitoring software and slipped malicious code into a legitimate, signed update distributed to thousands of organizations, including several US government agencies. For XZ Utils, a contributor spent nearly two years earning the trust of the project maintainers before inserting a backdoor into the build scripts of a compression library found on virtually every Linux system. The source code was clean. The betrayal was in the build chain. It was caught by accident, by a Microsoft engineer who noticed that ssh(1) was slower than it should have been.

The question is no longer “did you audit the code?” It is “do you trust the tool that built the binary you are running, and the tool that built that tool?”

Some defenses exist. Samuel Aubertin proposes binary diffing with radiff2, the binary comparison tool from the radare2 framework. Beyond a simple byte comparison, radiff2 produces a disassembled view of both binaries and lets you compare them at the assembly level, making visible modifications that appear nowhere in the source, including injected strings. David Wheeler's Diverse Double-Compiling formalizes the same intuition: compile the same source with two independent compilers and compare the outputs. The Reproducible Builds project works toward making compilation chains deterministic and verifiable. Go published a bootstrap process tracing back to Go 1.4, written in plain C, specifically to close this window.

These practices remain rare. Most production software chains run on implicit, unverified trust in the tools that constitute them.

Thompson was right in 1984. The paper is three pages long. It takes twenty minutes to read. And it changes, in a fairly lasting way, how you think about trust in the systems you build.

Sources: Ken Thompson, “Reflections on Trusting Trust”, Communications of the ACM, 1984. Russ Cox, “Running the Reflections on Trusting Trust Compiler”, research.swtch.com, 2023. David A. Wheeler, “Fully Countering Trusting Trust through Diverse Double-Compiling”, PhD thesis, 2009. US Air Force / James P. Anderson, “Computer Security Technology Planning Study”, 1972–1974. Samuel Aubertin, BSDCan talk.