Yes, We Taught AI to Hack — and It Mostly Works

Notes from Ariel Herbert-Voss’s talk at DEF CON Singapore, and what it means if you build or defend software for a living

I spend most of my working life worrying about two things: shipping software faster, and not getting owned while we do it. So when I sat down at DEF CON Singapore for Ariel Herbert-Voss’s talk “Yes We Taught AI to Hack (and It Mostly Works)”, I knew within about thirty seconds that I was going to be taking notes.

This is my attempt to write up what I heard, what I think it means, and why I walked out of that room equal parts excited and slightly uncomfortable. Full credit to Ariel — the talk was excellent, sharply argued, and refreshingly free of the AI hype-cycle nonsense that makes most of these conversations exhausting. If you ever get the chance to see her speak, go.

Ariel Herbert-Voss @ DEFCON Singapore 2026

A two-sentence primer, in case you’ve been hiding from LLMs

Ariel opened with the basics, so I will too. Large language models are, at heart, very fancy lookup tables. They predict the next token (a chunk of a word) given everything that came before. The “magic” is that with enough data, enough parameters, and enough compute, this next-token prediction starts looking suspiciously like reasoning.

The bit that matters for security people is this: as models get better at writing software, they get better at breaking it. The same skills — pulling context from many places, synthesising it into something new, holding a goal in mind — are exactly what vulnerability research requires. Code generation and code exploitation are two sides of the same coin.

What changed recently

For about seven years, the “scaling hypothesis” has held: more data plus more compute plus more parameters equals better performance. Boring, predictable, mostly linear.

What’s different now, Ariel argued, is that reasoning capabilities are scaling supralinearly. Train a model that’s twice as big for twice as long on twice as much data, and you don’t get something twice as good — you get something dramatically better. That’s the gap between Opus 4.6 and the latest generation, or between earlier GPT models and GPT-5.5.

For vulnerability research, the consequences are wild. She quoted some numbers from the 0-day clock project that genuinely made me re-read the slide:

Between 2023 and 2026, the average time-to-exploit has dropped from 5 months to 10 hours.

Palo Alto Networks reportedly compressed roughly a year of pentesting into under three weeks on their Glassmain project. And if you run a bug bounty programme, you’ve probably already noticed the change in the volume — and quality — of incoming reports as bounty hunters wire LLMs into their workflow.

The capability ceiling is rising

“The Capability Ceiling is Rising”

This was one of the slides that stuck with me. The point is that you used to need a lot of scaffolding — complex chains of prompts, tools, and validators wrapped around a model — to get useful security work out of it. That’s becoming less true. The model itself can now handle more of the process. Simpler agentic loops go further.

Concretely: in Anthropic’s experiments with the Mythos setup, the older Opus 4.6 had near-zero exploit success on the same tasks where the newer model produces results. That’s not a small step. That’s a phase change.

If you’re doing offensive work and you have access to frontier models, your tooling probably needs to get simpler, not more elaborate. Counter-intuitive, but consistent with what I’ve seen in my own DevOps practice — most of the clever scaffolding I built around early LLMs in 2023 is now just dead weight.

But the floor isn’t rising as fast

“The floor isn’t Rising as Quickly”

Here’s where Ariel got refreshingly honest. The improvements aren’t uniform. Some bug classes are getting absolutely steamrolled by current models. Others… really aren’t.

She cited the OSS-Fuzz numbers from the Mythos report. On low-severity tier 1 and tier 2 crashes, the newer model found nearly 600 — roughly double prior models. It also produced a handful of tier 3 and tier 4 bugs and around ten tier 5 bugs (think control-flow hijack territory). Older models managed only a couple of crashes at the equivalent tiers.

So: massive gains at low severity, modest gains in the middle, sparse gains at the deep end. Models still struggle with bug classes that require holding a lot of state, even though longer context windows have helped. Validation, exploitability assessment, and reliability all improve more slowly than raw discovery.

The practical consequence she summarised in a phrase I’m going to be quoting for months:

Defenders are unfortunately going to get hit by millions of monkeys with typewriters. Some of those monkeys will write a really good exploit. Defenders have to win every time. Attackers only have to get lucky once.

That’s not a new asymmetry — it’s the oldest one in our industry. But the cost of being one of those monkeys has collapsed.

What this means if you’re trying to defend something

A few things from the talk that I’m taking back to my own team:

Most damaging bugs are still simple. Default credentials, exposed admin endpoints, the boring stuff. You don’t need a frontier model to find those. The basics still matter, and the basics are exactly what an LLM-driven attacker is going to find first, fastest, and at scale.

Layering models beats picking “the best” one. Ariel made an interesting point about defence in depth: when you run multiple models against the same target, the overlap in findings is surprisingly low. There is no single best model for full coverage. Different models find different things. So if you’re building security tooling on top of LLMs, plan to layer them. Running multiple models on the same target leads to a surprisingly low overlap in findings; the best results come from layering models and approaches.

Open-source models are good enough for a lot. You don’t need access to Mythos-tier closed models to get value. If your token budget is tight or your org is way down the queue for frontier access, stringing open-source models together still finds plenty. This has been true since 2023 and it remains true.

Humans are still essential — but for different things. During Q&A, someone asked where humans still add value. Ariel’s answer: validation, prioritisation, and targeting. Models don’t really know what’s worth their time. They’ll happily burn $20,000 of compute exploring a code path that doesn’t matter. Putting a human in the loop on what to look at, and on whether a finding is actually exploitable in your real environment, is still where the leverage is.

Where this leaves us

The framing I liked best was Ariel’s analogy to fuzzing. When fuzzers first appeared, the early hot take was “well, we’ve solved bug finding.” That turned out to be wrong — the hard part wasn’t generating findings, it was figuring out which ones mattered. LLMs are following the same trajectory. Volume is no longer the bottleneck. Judgement is.

For builders and defenders, I think the takeaways are roughly:

• The pace of vulnerability discovery is accelerating, and the floor for “who can run a serious offensive operation” is dropping fast.
• This is genuinely the forcing function our industry has needed to do the unsexy fundamentals — patching, asset inventory, credential hygiene, defence in depth.
• The ceiling on what LLMs can find is now higher than most of us appreciate. Multi-step vulnerabilities that used to take months now appear overnight.
• The floor — meaning reliable, repeatable, end-to-end exploitation — is rising more slowly. Don’t panic.
• Build harnesses around bug classes you already understand deeply. You’ll know when the model is being useful and when it’s confidently wrong.

Ariel closed with a line I appreciated: don’t let the initial panic distract you, because we’re just getting started. That feels right. The capability curve is still steepening, the economics are still shifting, and most of us — myself very much included — are still figuring out which of our existing security practices are about to look quaint.

If there’s one thing I’m doing differently on Monday, it’s this: I’m raising the priority on the unsexy fundamentals. Credential hygiene, asset inventory, patch latency, defence in depth — the stuff every security person already knows matters. LLM-assisted attackers don’t change what we should be doing. They change how unforgiving the timeline is for actually doing it.

—

This article is based on Ariel Herbert-Voss’s talk “Yes We Taught AI to Hack (and It Mostly Works)” at DEF CON Singapore. Any inaccuracies in interpretation are mine, not hers — go watch the original if you can. Photos taken at the event.