The AI Engineer’s Salary Is About to Split in Two. On Which Side Do You Want to Be ?

I want to tell you about two engineers I know of. Same company. Same title. Same four years in the role.One just got promoted to Staff with a $340K total comp package. The other is on a performance improvement plan.

I have been thinking about what separated them for weeks now. Not because the outcome was surprising — in hindsight it was almost inevitable but because neither of them saw it coming. I suspect a lot of people reading this are closer to that uncertainty than they realise.

So let me say the thing that most career articles will not say directly.

The Split Is Already Happening. Most People Just Haven’t Felt It Yet.

The numbers are not subtle anymore.

Software engineering job postings are down roughly 70% from their 2022 peak. Entry-level positions have dropped 34% compared to 2020. At the same time, AI engineer roles have more than doubled since mid-2024. Median total compensation for AI engineers in the US now sits around $245,000 — with the top quartile clearing $350K.

But here is the number that stopped me cold when I first saw it. AI talent compensation has split into two completely separate markets. Enterprise ML engineers earn $170K–$245K total. A cohort at frontier labs commands $600K–$1M+ for the same job titles.

Same title. Four to six times the pay. That is not a salary range. That is two different careers wearing the same name badge.

And the split is not just happening at the frontier labs. AI engineer salaries grew 9–12% year-over-year in specialised roles in 2026. Generalist software salaries? Flat. The divergence is quiet and it is everywhere.

Here's what worries me about this: the engineers on the wrong side of it are not going to feel it in their day-to-day work. They will feel it at their next comp review. Or the one after that. Or the moment they try to move companies and discover that the roles they thought they were qualified for have quietly moved the goalposts.

I Want to Be Honest About What Side One Looks Like

Because I think most writing about this topic is not.

The engineer on the PIP is not lazy. He is not incompetent. He is genuinely good at his job — the job that existed eighteen months ago.

He knows how to call the OpenAI API. He has built LangChain pipelines. He can fine-tune a model with some guidance. He understands, at a conceptual level, how transformers work. All of that is real. None of it is nothing.

But here is the uncomfortable truth. Every single one of those skills has commoditised faster than anyone predicted. The things that made you stand out in 2023 are now listed under “basic requirements” in the job postings for the roles you want in 2026. The floor rose. The premium moved somewhere else. And nobody sent an announcement.

Over 75% of AI job listings now ask for domain experts, not generalists. Generalists face increasing competition from candidates who command salaries 30–50% higher for the same years of experience. That is not a small gap. That is a structural shift in what the market values.

What makes this hard to catch is that it does not feel like falling behind. It feels like keeping up. You are learning new tools. You are shipping features. Your manager likes you. Everything looks fine from the inside — right up until the moment it does not.

He felt fine. Until the promotion cycle. Then the gap that had been quietly widening for eighteen months became visible all at once.

That is the pattern I keep seeing. Not a sudden fall. A slow drift that compounds until someone puts a number on it.

So What Does the Other Side Look Like?

She wasn't building flashier things. She was not smarter. She was asking a different question.

Not: how do I use this model?

But: how do I know if this model is actually working?

That reorientation — from building to verifying, from shipping to understanding — turns out to be worth a lot of money right now. Not because it is rare in the abstract. But because very few engineers have sat with that question long enough to develop a real answer.

A Karat survey of 300+ CTOs found that 73% say strong engineers are worth at least 3x their total compensation. The engineers they are describing are not the ones who build impressive demos. They are the ones who can take a demo and turn it into a system that runs reliably when no one is watching — and who can tell you, from inside the system, when something starts going wrong.

That is the capability the market cannot find enough of. And it is the capability she spent the last year building while he was deepening skills the market already had plenty of. Twelve months later, she is fielding recruiter calls she is not bothering to return. He is writing a self-assessment for a PIP that he still does not fully understand.

I am not telling you this to be harsh. I am telling you this because I think it is important to say clearly: the divergence is not about talent or effort. It is about which problems you chose to care about, and when.

The Three Things That Are Actually Creating the Gap

I want to be specific here. Because “build systems thinking” is advice that sounds meaningful and tells you nothing.

The First One: Learning to Define What Good Looks Like

Picture this. A team ships an AI feature. A few weeks later users start complaining that the answers feel off. The engineers pull up the dashboard. Latency is normal. Error rate is zero. The system is flagged healthy. The answers are still wrong.

Three weeks pass before someone manually samples outputs and realises the model has been confidently hallucinating a specific category of fact since a quiet dependency update two sprints ago.

I have heard versions of this story more times than I can count. And every time, the post-mortem comes down to the same thing: nobody had defined, in a form the system could check automatically, what a correct answer looked like.

This is what I mean by evaluation architecture. Not running a model on a benchmark — that’s table stakes. I mean the ability to define what good looks like for a specific system in a specific context, precisely enough that a computer can verify it continuously, and then building the infrastructure that actually does that.

This skill is genuinely rare. And it is the one that companies struggling to move from demo to production are searching hardest for right now. If you can walk into a room and explain how you would measure whether an AI tool is working — not in theory, in practice, for this specific use case — you are already ahead of most candidates.

The Second One: Understanding How Information Actually Flows Through a System

There is a category of failure in agent systems that I find endlessly fascinating and deeply underappreciated.

A developer joins a new team. The agent system they have inherited has been running for eight months. It works reasonably well on simple queries. On anything complex or multi-step it produces output that is technically coherent and subtly wrong — the kind of wrong that takes a domain expert to catch.

They spend two weeks reading the codebase. The prompts look fine. The model is fine. The individual tool calls look fine.

What is not fine is invisible in any single component. It is the information architecture — what gets loaded into the agent’s attention at each step, what gets dropped between steps, what degrades across a long session, and what was never being preserved across handoffs in the first place. The system was losing context and nobody had ever built it with that problem in mind.

This is what I call context engineering, and I want to be clear that it is completely different from prompt engineering. Prompt engineering is about phrasing a single input well. Context engineering is about understanding how information flows — and degrades — through an entire system over time. It requires a different way of seeing the problem. And the engineers who have developed it tend to catch failure modes in architecture reviews that everyone else walks right past.

The Third One: Being Able to See Inside a Running System

An engineer gets a message at 11pm. A customer has flagged wrong pricing information from the AI assistant. The engineer pulls up the logs. Request completed successfully. Response was well-formatted. Latency 340 milliseconds.

They spend two hours trying to find what went wrong. Cannot reproduce it. Cannot trace it. Write it off as a one-off. It was not a one-off. It was the seventeenth time the same underlying failure had happened. They had no way to know because they had no way to look.

Production observability for AI systems is knowing when your system is wrong without waiting for someone to tell you. It’s not whether the request was completed but whether the agent did the right thing, in the right way, for the right reason.

The engineer who has built this capability is the one who, when something breaks, can find the exact moment things went wrong — not “the output was bad” but where in the chain, which step, what caused it. That specificity transforms a two-hour debugging session into twelve minutes. And it transforms how a team feels about deploying changes, because they know they will catch problems before users do.

The Honest Career Advice

Most of the content about AI careers right now tells you to learn the next framework, take the next course, get certified in whatever launched last month.

I think that advice is missing the point.

The engineers on the right side of this salary split are not the ones with the longest certification lists. They are the ones who have built something that failed in production, diagnosed why, fixed it, and — this part matters — written about what they learned.

That last piece is not just about visibility, though visibility matters. It is about the discipline of translating experience into understanding. The engineer who can write clearly about a real problem they worked through — what they tried, what broke, what they now see that they did not see before — is developing a habit of mind that shows up directly in how they approach the next problem.

She had been writing. Not daily, not for an audience of thousands. But consistently, about real things she was actually working through. By the time her promotion cycle came around, her future staff manager had already read two of her pieces. She did not have to explain what she knew. He already knew what she knew. The interview was a conversation, not an audition.

That is a different kind of career asset. And it compounds in a way that certifications do not.

One Question Worth Sitting With

I will leave you with the diagnostic I keep coming back to when I think about this split. When your AI system produces wrong output in production — not a crash, not an error, wrong output, how long does it take you to know?

If the answer is hours, you have partial visibility. If the answer is days, you have logs but not evaluation. If the answer is not until a user tells you, you are flying blind inside your own product.

That gap is not a reflection of how smart you are. It is a reflection of which skills you have been building. And it is also, right now, the clearest signal of which side of the salary split you are on — or drifting toward.

The two engineers I started with were not separated by talent or effort or even opportunity. They were separated by which problems they chose to pay attention to, twelve months before the outcome became visible.

The good news — and I mean this genuinely — is that none of this is fixed. The skills are learnable. The problems are solvable. Engineers are moving from one side to the other every month.

Twelve months from now, where you end up is still very much your choice to make.

Salary data drawn from: Levels.fyi May 2026 · Robert Half 2026 Salary Guide · PwC Global AI Jobs Barometer · KORE1 AI Engineer Salary Guide · Acceler8 Talent 2026 · Pin.com AI Compensation Benchmarks · Data Science Collective