I Ran Five Frontier AI Models Through a Government Test. Here’s Why I’ll Never Nap in a Self-Driving Car.

The slowest model took 10.5 seconds to read a bottle label. The fastest took 1.8. Neither number is the one that should scare you.

I recently built a tool for a federal exercise: point a vision model at an alcohol bottle label, read the mandatory Government Warning off the image, and decide whether it complies with 27 CFR Part 16. Then I benchmarked six configurations across five frontier models — Claude Sonnet 4.6, Claude Opus 4.8, Claude Fable 5, GPT-4o, and GPT-5.5 — against the same ten labeled images, recording accuracy and latency on every single read. The code and the full results are public.

Here is the headline. On a static, well-lit, perfectly framed photograph of a bottle that is not moving, not occluded, not in the rain, and not lit by oncoming headlights, the fastest current-generation configuration averaged about 2.5 seconds per image. The newest and most capable model in the set averaged 8.4 seconds and never once came in under 5.5. The slowest single read in the entire benchmark took 10.5 seconds.

Ten and a half seconds. A car at 70 mph covers a hundred feet every second.

Now, before anyone screenshots that and posts it as proof that self-driving cars are ten seconds behind reality, I have to be honest with you, because a lot of articles like this one are not. Those numbers do not measure what a car’s autopilot does. My benchmark measured general-purpose reasoning models reached over a network, doing document comprehension. A production driving stack runs purpose-built perception models on dedicated silicon bolted to the car, at tens of milliseconds per frame, with no internet in the loop. Nobody is round-tripping your windshield to a datacenter and waiting for a verdict. If someone tells you otherwise, close the tab.

So why am I still writing this?

Because the benchmark caught something else. Something that transfers perfectly. And almost nobody talks about it, because it is much less fun than a big scary number.

The failure wasn’t slowness. It was a confident yes.

GPT-4o scored 9 out of 10 on my test set. That sounds excellent. The one it missed was a file called non-bold-warning.png: a label where the mandatory warning text simply is not printed in bold, which is an actual statutory defect.

It did not flag the image as ambiguous. It did not return low confidence. It did not ask for a second look. It returned a clean, confident PASS on a defective label. Every run. Meanwhile every current-generation model in the set caught it, 10 out of 10. One model generation was the entire difference between catching the defect and waving it through with no signal that anything had gone wrong at all.

That single test case is the whole argument against unattended autonomy, and it has nothing to do with milliseconds.

A slow system tells you it is slow. Latency is an engineering problem, and engineering problems have engineering answers: brake earlier, widen the following distance, degrade gracefully, hand back control. What you cannot engineer around is a system that is wrong and certain at the same time. The failure mode that kills people is not the car that hesitates. It is the car that looks directly at the ladder that just bounced out of the truck bed ahead of you and reports, with total confidence, clear road.

Bold versus not-bold. In perfect lighting. On a flat, motionless image.

Sit with how easy my test was. Every one of those ten images belonged to a category these models have seen an enormous number of times: a bottle label, photographed straight on, in good light, filling the frame. The hardest discriminator in the whole set was a binary question about typography. Is this text bold, or is it not bold. That question split model generations cleanly in half.

Now change the question. A shredded semi tire on wet asphalt at dusk. A mattress at an angle no training set ever framed. A ladder mid-bounce, half in shadow, three feet from a pothole that casts a very similar shape. A deer that is not yet a deer, just a shape at the edge of the headlights.

The set of things that can fall off a truck is not a fixed list. It is open-ended, and it is adversarial in the way reality is adversarial: it does not care what your test set contained. Every benchmark you have ever read, including mine, measures performance on a closed set of things somebody thought of in advance. The road is not a closed set.

I built the escape hatches myself. That’s what actually convinced me.

Here is the part of my own tool that changed how I think about riding in a car that drives itself.

I never let the model decide anything. The model only reads the image into structured text. Every actual judgment is made by plain, deterministic, unit-tested code, so the verdict is explainable and reproducible and does not depend on a model’s mood. There is a third verdict alongside pass and fail, called NEEDS REVIEW, for everything readable but ambiguous, and it exists precisely because guessing is worse than admitting uncertainty. And for the single hardest check, the bold one, there is an option to run several independent reads in parallel and treat disagreement between them as uncertainty rather than picking a winner.

I designed all of that defensive machinery for bottles. Bottles sitting still on a table. Bottles that were not approaching me at 70 miles an hour. And I still would not let the model make the final call by itself.

Every one of those safeguards ends the same way: hand the hard case to a human being. That is the entire architecture. It works because there is always a human available to receive the hard case.

A car at highway speed has no one to hand it to. Unless someone is sitting right there, awake, eyes up, hands near the wheel.

The uncomfortable half: the human is a point of failure too

Here is the trap, and it is worse than anything in the benchmark.

Partial automation is good enough to make you stop watching, and not good enough that you can stop watching. Those two facts live in the same car at the same time. Every flawless mile is quietly training you to trust it more. Aviation has known this for decades and has a name for it, automation complacency, and airline pilots get recurrent training on it. You get a click of a stalk and a chime.

Then the moment arrives. The system needs you, and it gives you a second or two of notice, and you spend most of that reacquiring context you stopped holding twenty minutes ago. What is happening. Which lane. What is that shape. Is it braking. The handoff is the most dangerous instant in the whole system, and it is the one instant the system is worst at.

That is the thing I keep coming back to. Human beings are what turn a predictable situation unpredictable, instantly. A person builds an assist good enough to relax around, another person relaxes, and the exact same species is standing at both ends of the failure. The machine did not become unreliable. The arrangement of humans around it did.

Count the points of failure on one trip

The lens and whatever the weather has put on it. The exposure, when you come out of a tunnel into direct sun. The sensor. The model. The confidence estimate, which is its own model and can be wrong about being wrong. The planner deciding what to do about it. The actuator doing it. The software update that shipped on Tuesday and changed behavior you had already internalized. The driver who stopped paying attention back in March. And the driver in the other car, who has no system at all and is looking at their phone.

Each link can be excellent. Chain them and multiply the failure probabilities and you get a number nobody wants printed on the window sticker.

This is not an anti-AI essay

I build with these models. I benchmarked them, picked one, and shipped it, and it does the job well enough that a compliance agent gets a useful answer in a few seconds instead of squinting at a photo. The technology is genuinely good and it is getting better on a schedule that keeps surprising me.

The benchmark did not teach me that AI is bad. It taught me the shape of how it is wrong. Not often. Not loudly. Not on the easy cases. It is wrong rarely, confidently, and precisely on the one case you built the test to catch, and it does not tell you when it happens. That is a fine property for a bottle label, where the cost of being wrong is a human taking a second look. It is a different property entirely at 70 miles an hour.

So use the assist. Use the lane keeping, use the adaptive cruise, use the automatic emergency braking, all of it. It will almost certainly make you a safer driver on net.

And keep your eyes on the road anyway. Because for the one moment the system is confidently wrong, you are the NEEDS REVIEW.

The project, the code, and the full per-model benchmark table are public: github.com/thomasmeadows/treasury-test-label-scanner — the raw numbers are in test-labels/benchmark-results.md. Run it yourself and check my math. https://treasury-test-label-scanner.vercel.app/