Stop Renting Intelligence You Don’t Need

I’ve watched close to a dozen teams reach for the same default when a new AI feature comes up: pick the biggest, most capable frontier model available, because it’s the safest-sounding choice and it can technically do anything. (Illustrative pattern drawn from work across multiple teams, not one specific engagement.) What I’ve watched happen next is that most of what the feature actually needs turns out to be narrow: parse a structured log line, validate a field against a schema, sort a support ticket into one of six categories. None of that needed a model that can write poetry or pass the bar exam. It needed something fast, cheap, and predictable on one specific job.

A model that can do anything is optimized for the wrong thing

The frontier race is a real achievement, and it’s answering a question most production workloads aren’t asking. A huge share of what actually runs day to day is repetitive and well-defined, the kind of task a domain expert could write explicit rules for if you asked them to. Handing that task to a general-purpose model built to reason about anything means paying for a capability the task never uses, on every single call.

Latency is the first cost that shows up

I watched a fraud-detection check built on a hosted frontier model pass every test in staging, then start missing its latency target under real peak traffic. The model wasn’t reasoning worse. Every call was a network round trip into a shared queue, competing with every other customer hitting that provider at the same moment. A small, distilled model sitting on the same box as the service that calls it doesn’t have that queue to wait behind. Its latency stays boring and predictable, because nothing external sits in the path.

Your data boundary is only as strong as your last API call

A compliance review can stop a project with one question: where does this data go once it leaves our system, and who else can see it. When the honest answer involves a third-party endpoint, that’s a boundary owned by someone else, one you can only audit from the outside. A small model running inside your own infrastructure removes the question entirely. The data never leaves, so there’s no external retention policy to trust and no provider agreement to re-read every renewal cycle.

Someone else’s business decision becomes your outage

A workflow built on a hosted model API can run cleanly for a year, then break overnight because of a price increase, a rate limit change, or a model deprecation, none of which the team using it had any say in. That’s not a technical failure. It’s a dependency on another company’s roadmap, dressed up as a technical choice. A locally hosted, version-pinned model doesn’t carry that exposure, because nobody outside the team decides when support ends.

A model trained on your schema doesn’t have to guess at it

A general-purpose model asked to validate rows against a real database schema usually gets it right, and occasionally hallucinates a plausible field name that doesn’t exist, because it’s pattern-matching against schemas in general, not the specific one in front of it. A model fine-tuned on the company’s actual data isn’t estimating a resemblance. It has seen the real shape of the problem before.

The uncomfortable conclusion

None of this is a case against frontier models. It’s a case against reaching for one by default, out of habit, for tasks that were never going to need everything it can do. The real cost of using a model that can do anything, for a task that only needed one thing, isn’t obvious in a demo. It shows up later, as latency you can’t explain, a data boundary you can’t fully audit, and a dependency on someone else’s pricing decisions. The smartest engineering decision is often the smaller model, chosen on purpose.