Software No Longer Runs Only in Code: The New Abstraction Layer in the Age of LLMs

A while ago, I rewatched an old Yalın Kod video by Bilgem Çakır, published around nine years ago. On the surface, the video starts with a simple question: “Does a software developer really need to understand hardware?” But as it unfolds, it ends up touching something much more fundamental: abstraction.

The main idea of the video was roughly this: the history of software engineering is, in many ways, the history of building new layers of abstraction. From machine code to assembly, from assembly to C, from C to managed languages, and from there to frameworks, cloud platforms, and serverless systems, we have continuously learned to think at higher levels.

That is a huge achievement. It increased our ability to solve larger and more complex problems. But every abstraction also makes the underlying reality a little less visible.

I think this topic deserves to be revisited today, because a completely new layer has entered the picture: the LLM/AI layer.

In the past, developers were moving further away from hardware. Today, in some cases, developers are even moving further away from code itself.

We no longer just write functions. Sometimes we describe intent. We write prompts. We design agent workflows. We give tools to models. We expect them to read documents, call APIs, generate JSON, make decisions, and sometimes even write code on our behalf.

This is incredibly powerful. But it also makes the old discussion around abstraction much more important.

Abstraction Is Not Bad, But It Is Not Free Either

First, it is important to make one thing clear: abstraction is not a bad thing. In fact, it is one of the strongest ideas in software engineering.

Being able to break a problem into smaller parts, think about those parts independently, and then return to the bigger picture when necessary is an extremely valuable skill. A good engineer is often someone who can do exactly that. Sometimes they look at the whole system. Sometimes they go deep into a function. Sometimes they think about the data model. Sometimes, when a performance issue appears, they go all the way down to memory access patterns.

But the problem is this: while an abstraction layer gives us local simplicity, it also adds new complexity to the system as a whole.

For example, when we hide one function behind another function, we may make the code more readable. But at the same time, we create a new interface, a new contract, new tests, new edge cases, and sometimes new performance costs.

In other words, abstraction does not make complexity disappear. It simply moves it somewhere else.

I think one of the biggest misconceptions in the AI era starts exactly here. When we let a model handle a task, we tend to assume that the complexity has disappeared. But in most cases, that complexity has simply moved out of the code and into the prompt, the context, the tool calls, the retrieval system, or the behavior of the model.

The New Layer: Building Software Through Natural Language

With LLMs, a new abstraction layer has been added to software development. This layer is different from traditional framework layers, because we are not only abstracting code anymore. We are also abstracting intent.

A user asks for something:

“Read these documents, extract the people involved, separate the evidence, summarize the charges, and generate a case wiki.”

In the past, we would probably build a clear pipeline for this. There would be a parser, an entity extractor, a service that writes to a database, and a UI component that displays the result.

Today, we can say something like this to an LLM agent:

“Analyze this file and create a case wiki.”

At first glance, this looks much simpler. But behind the scenes, there are many engineering questions that still need to be answered:

• What text did the model actually see?
• Did it see the whole document or only part of it?
• Which information did it treat as certain?
• Which information did it infer?
• Can it cite its sources?
• If it assigns the wrong role to the wrong person, how will we detect that?
• If it needs to call a tool, will it actually call it?
• If it does not call the tool, what is the fallback?
• Does the JSON it produces match the schema?
• Even if it matches the schema, is it semantically correct?
• Can this system be tested?

None of these questions disappeared. They now exist in a more abstract and more slippery layer.

Model Intelligence Does Not Mean System Reliability

LLMs can produce impressive answers. Sometimes they answer so well that it becomes very easy to believe the system genuinely understands what it is doing.

But this is the point where we need to be careful: a model sounding intelligent does not mean the system we built around it is reliable.

This reminds me of the old Big-O discussion. In theory, an algorithm that runs in O(log n) should be faster than a linear search. But in the real world, because of cache misses and memory access patterns, a linear scan over a compact array can sometimes outperform a tree lookup. Big-O is a useful abstraction, but it does not describe the full behavior of real hardware.

Today, we have the AI version of that same issue:

Using a bigger model does not automatically mean you have built a more reliable system.

Sometimes, a smaller model with well-prepared context and a solid validation layer can be more reliable. And sometimes, a very powerful model can produce a confident but wrong answer simply because the retrieval layer gave it the wrong document.

So the new equation is this:

Model intelligence ≠ System reliability

The quality of an LLM system is not determined only by the model’s raw capability. Context quality, data sources, tool contracts, output schemas, evaluation, logging, user approval flows, and failure handling are at least as important as the model itself.

Not Just Prompt Engineering, But Context Engineering

For a while, everyone was talking about prompt engineering. There was a feeling that if you could just write the right prompt, the model would solve everything.

I think that view is now incomplete.

In real applications, the problem is not just about writing a nice instruction. The real problem is making sure the model has the right information, the right tools, and the right boundaries at the right time.

I think of this more as context engineering.

What you give to the model matters just as much as what you do not give to it. Which documents you place into context, how you filter outdated information, how you handle conflicting sources, and how you define when the model should say “I don’t know” directly affect the final result.

The context given to an LLM can almost be seen as a new kind of memory space. But this memory is not unlimited. There is a context window. There is token cost. There is the risk of distracting the model with unnecessary information.

So in the AI era, performance is not only about CPU cycles or database query time. Sometimes the performance problem is:

• An unnecessarily long prompt
• Bloated context
• Poor chunking
• Weak retrieval
• Too many tool calls
• Unnecessary agent steps
• Deterministic work being delegated to the model

That is why building a good AI system is not about saying, “Let’s give everything to the model and let it figure it out.” It is the opposite. It is about giving the model the right context within the right boundaries.

Tool Calling Is the New Interface Design

In LLM-based systems, tool calling reminds me of interface design in traditional software.

When we tell a model, “You can send emails,” “You can delete files,” “You can write to the database,” or “You can call this API,” we are opening up a permission boundary.

At that point, the question is not only whether the model can call the right tool. The tool’s contract also needs to be designed correctly.

For example:

• Should the tool only read data, or should it also write data?
• Does a write operation require user approval?
• Is there a rollback mechanism?
• Is the tool input validated with a schema?
• Does the data sent by the model pass business-rule validation?
• What happens if the model misinterprets the tool result?

In traditional software, a poorly designed interface spreads complexity throughout the system. In AI systems, poorly designed tools do the same thing. In fact, they can be even more dangerous, because the model may take a wrong action while explaining it in very reasonable natural language.

So giving tools to an LLM is not as simple as saying, “Here, use this.” It is a serious question of permission design, safety, and system boundaries.

An Agent Is Not Always the Answer

In the AI world, the word “agent” is very attractive. When we call something an agent, it immediately sounds smarter, more autonomous, and more modern.

But not every problem needs to be solved with an agent.

Sometimes, a simple deterministic pipeline is the better solution. In fact, for many production scenarios, I think the healthier approach is this:

Keep deterministic things deterministic. Use LLMs where ambiguity actually exists.

For example, if you can reliably extract legal article numbers from a legal document using regex, you do not need to ask an LLM to do that. Candidate person extraction, date detection, and document type classification can often be handled with classical methods. The LLM can then be used for ambiguity resolution, summarization, relationship extraction, or converting structured information into human-readable language.

This approach feels healthier to me, because it treats the LLM as one component inside the system rather than making it the entire system.

Otherwise, it is easy to end up with something like this:

User request arrives → agent thinks → agent searches → agent decides → agent writes → agent reviews → another agent validates

This architecture may look impressive in a demo. But in production, it can be difficult to debug, test, control, secure, and pay for.

The New Engineering Skill: Moving Between Layers

I think the most important idea from that nine-year-old video is still valid today: a good engineer is someone who can move between abstraction layers.

The difference is that the number of layers has increased.

In the past, we might have thought in terms of:

• Product level
• Architecture level
• Code level
• Data structure level
• Memory / CPU level

Today, new levels have been added:

• User intent
• Prompt level
• Context level
• Retrieval level
• Tool/API contract
• Agent workflow
• Evaluation and observability
• Cost and latency

So the mental map of the modern software engineer has become wider.

On one side, they need to understand the user’s language. On another side, they need to know how to express that intent to a model. They also need to control what information the model can access, design safe tools, and test whether the generated output is actually correct.

That is why the idea that “we no longer need to know how to code because AI exists” feels very shallow to me.

I think the opposite has happened.

Knowing how to code is still important. But it is no longer enough by itself. On top of code, we now need to understand model behavior, data context, product intent, safety, validation, and system design.

LLM Abstraction Should Not Make Us Lazy

The most dangerous thing about LLMs is not that they make mistakes. We are used to systems making mistakes. The real danger is that they can present their mistakes in a very convincing way.

When a compiler fails, it usually complains. When a test fails, we see red. When an API returns a 500, we know something is wrong somewhere.

But an LLM can give a wrong answer in clean English, with good headings, structured paragraphs, and a confident tone.

That is why a large part of engineering in the LLM era is not about teaching the model to produce answers. It is about placing the model’s answers inside a reliable system.

That requires certain habits:

• Do not trust critical information without sources
• Constrain model output with schemas
• Log tool calls
• Require user approval for critical actions
• Build evaluation sets
• Separate the places where simple code is enough from the places where an LLM is actually useful
• Measure retrieval quality separately
• Make “I don’t know” a valid system behavior

These things may sound boring. But the difference between a demo and a production system often lives exactly here.

Conclusion: AI Did Not End Software Engineering; It Expanded Its Abstraction Space

The conclusion I draw from all of this is simple:

LLMs did not eliminate software engineering. They expanded the abstraction space in which software engineering operates.

Now we do not only need to make sure the code works correctly. We also need to make sure the intent is understood correctly, the context is constructed correctly, the model uses the right tools, and the output is verifiable.

That is why the good engineer of this new era is not simply someone who knows a framework. And it is not simply someone who knows how to write prompts either.

To me, the good engineer of the new era is this:

Someone who can build a reliable bridge between user intent and deterministic systems.

On one side, there is natural language: ambiguous, incomplete, and open to interpretation.

On the other side, there are software systems: systems that need schemas, contracts, tests, permissions, and security.

LLMs can become a very powerful bridge between these two worlds. But whether that bridge is solid or not still depends on engineering.

So maybe the shortest definition of engineering in the AI era is this:

The art of connecting ambiguous natural-language intent to reliable deterministic systems.

I think that is the real issue. Not just using AI, but designing systems that understand the cost, boundaries, and responsibility of the AI layer.

Additional Work: Executable Wisdom

While preparing this article, I also turned the same idea into a small GitHub repository: Executable Wisdom

github.com/efekurucay/executable-wisdom

The purpose of the repository is to adapt old but timeless engineering content to today’s agentic AI world.

The core principles from a video, article, or talk are extracted, and then transformed into reusable outputs such as:

• skills,
• prompts,
• checklists,
• workflows,
• review rubrics.

As the first case, I took the abstraction/hardware narrative that inspired this article and converted it into several skill files that can be used in the LLM era.

https://www.efekurucay.com/