We Have 2 Years Left Before AI Becomes Supernatural

Every 7 months it becomes twice as better

Engineers at top AI labs are saying they don’t write code anymore.

The model does it.

And we might be 6 to 12 months away from AI doing most — maybe all — of what software engineers do end to end.

The doubling timeline

Task horizons — how long a task an AI can complete autonomously — are doubling every 4 to 7 months.

The compute going into models doubles every 4 months.

Follow that trend and by the end of 2026 AI systems should be able to complete what would take a human a full day.

Week-long tasks aren’t far behind. Month-long software engineering projects shortly after that.

Mythos Preview — the Anthropic model that got pulled by the US government — can already complete 17-hour tasks at 50% accuracy.

50% sounds bad. But 50% on a 17-hour task autonomously is extraordinary. A year ago these systems couldn’t do 17-minute tasks reliably.

Why this accelerates differently than people expect

Most people picture AI improvement as a linear substitution. This profession gets automated. Then that one. Gradual, predictable, manageable.

The actual trajectory looks different.

AI research is likely to be one of the first things automated. Once that happens — once AI systems are doing the research that makes AI systems better — the timeline compresses dramatically.

Instead of humans spending years improving each model generation, you have AI doing it faster, in parallel, at scale.

The economy looks mostly normal. Until someone has an army of superintelligences running in data centers. Then they can automate any sector of the economy they want, rapidly.

Scale-up is not linear. Right now labs deploy thousands of instances of a model.

Within 2 to 3 years clusters will be large enough to deploy millions simultaneously. This isn’t one AI improving slowly. It’s millions of instances working in parallel.

The two failure modes everyone serious is worried about

Domination.

A single AI system — or a small group of humans using AI — accumulates enough power to dominate everyone else.

The harm isn’t necessarily a rogue machine.

It could be perfectly aligned AI that’s perfectly aligned with the wrong person or group.

An AI-enabled coup. A single company or country capturing enough intelligence that the rest of the world becomes irrelevant.

The race.

No single actor dominates. Instead everyone races desperately to not fall behind. Every organization, every country, every company is locally incentivized to pour every available resource into capability rather than safety.

Nobody can coordinate. Nobody can afford to slow down. Nobody can spend resources on anything other than winning the race.

This is worse than an arms race because in an arms race at least the weapons sit in silos. In a capability race every organization is sprinting toward systems they don’t fully understand.

One participant called it a suicide race. Everybody loses if anybody’s AI goes out of control.

Why alignment isn’t keeping pace

Capabilities are improving exponentially.

Alignment — the work of ensuring AI systems actually do what humans want and remain safe as they become more capable — is moving like a snail by comparison.

There’s a specific problem that becomes harder as systems improve. You can’t rely on humans reviewing AI outputs at scale. AI will produce volumes of code, research, and decisions that no human team could meaningfully check.

The current approach — ask the AI to write securely, have humans review — breaks down when the volume exceeds human capacity to review. Which is coming fast.

The scalable oversight alternative

Instead of trying to review what AI produces, make AI prove what it produces.

Formally verified code. Mathematical proofs that a system cannot be hacked in specific ways. Specifications so precise that a proof checker — not a human — can verify them.

DARPA’s SeL4 operating system already demonstrates this is possible.

Red teams with full access to the development process couldn’t break out of its isolation. It was used in military helicopters. It works.

The problem — writing formally verified, provably secure code requires enormous human effort. That’s exactly the kind of effort AI becomes very good at reducing.

So the path is: use AI to help write the formal specifications and proofs that let us verify AI’s own outputs without needing to understand every line of code.

You can trust a system more capable than you are if you can verify its outputs through a proof checker rather than through comprehension.

The energy grid problem

This is where abstract AI risk becomes concrete and immediate.

In 2025 UK cyber attacks cost $14.7 billion. That was before AI dramatically lowered the cost and expertise required to execute sophisticated attacks.

What used to require a state actor — compromising critical infrastructure, disrupting power grids — could now be done by a small group empowered with capable AI tools.

AI adoption in high-stakes infrastructure has been slow precisely because the trust requirements are so high.

Why would a regulator trust an AI system managing grid frequency? You’d need formal safety cases. Proofs. Audit trails.

But the cost of not adopting is also high. Demand for electricity keeps growing. Without optimization, grid management becomes manual, inefficient, increasingly strained.

The optimistic version of AI in energy — optimization algorithms that can mathematically prove they maintain stability under specific conditions.

Formally verified models that give regulators something auditable. Systems that fail safely rather than catastrophically.

The dystopian version — rushed deployment of unverified AI in critical infrastructure, cyber attacks that exploit vulnerabilities the AI introduced, cascading failures in systems humans no longer fully understand.

The 2 to 4 year window

Multiple researchers speaking in this clip independently identify the same critical period.

The next 2 to 4 years are path-defining.

If that window is used to build scalable oversight infrastructure — tooling and workflows that let humans verify AI outputs without needing to blindly trust them — the trajectory is much more manageable.

If AI systems become capable of autonomous R&D before that infrastructure exists, the work of improving AI shifts to AI itself.

The speed and volume of what’s happening makes human intervention progressively harder.

The window is open. It’s closing.

The race between capability improvement and the tools to verify and steer that capability is the central question of the next several years.

Not abstractly. In terms of specific engineering decisions being made right now.

Whether the answer is domination by one actor, a chaotic race among many, or something more like flourishing coordination depends substantially on what gets built in that window.

What resilience actually means

The goal isn’t preventing anything bad from ever happening.

It’s preventing anything irreversibly bad from happening.

Maintain the core functions that allow society to continue operating. Keep options open.

Build systems that fail safely — backing up to conservative behavior when they encounter something outside their validated operating range.

Civilization has been resilient so far because we’re still here. That’s actually the definition.

The question AI introduces is whether the scale of potential failures has now exceeded anything civilization’s existing resilience mechanisms were designed to handle.

An atom bomb is one thing. No atom bomb makes another atom bomb.

An AI system capable of autonomous improvement is something genuinely different.

Which is why the emphasis on making these systems more steerable, more auditable, and more formally verifiable right now — before they become harder to steer — is not paranoia.

It’s the most important engineering problem of the next four years.

Want more stuff like this? Even more detailed? I go into deep dives every single day on my 📌Substack📌

At the price of a coffee I can give you enough knowledge to replace a bachelor’s degree in computer science or maybe even a masters, who knows.

💜 If you like my work and would like to support to me financially. Even a small donation would help a lot!

My PayPal — Please Support