DSPy 1 — Beyond Prompt Engineering: A Practical Introduction to DSPy

From hand-written prompts to measurable AI workflows

Infographic generated by NotebookLM

Most people begin working with large language models through prompts. We write an instruction, inspect the answer, adjust the wording, and try again. At first, this feels natural, almost conversational. If the model misses an important point, we add more context. If the answer is too vague, we ask it to be more precise. If the format is wrong, we tell it to return JSON, Markdown, bullet points, or a shorter summary.

This way of working is useful, and it is not going away. Prompt engineering is still an effective way to explore what a model can do, especially when a human is reading the answer and can immediately decide whether it is acceptable. The problem appears when the prompt stops being an exploratory instruction and becomes part of a software system. At that point, the question is no longer simply whether the model can answer once, but whether it can behave consistently across many inputs, under changing conditions, and in a form that other code can use.

This article is the first in a short series on DSPy, a Python framework for building language-model applications in a more systematic way. The aim of the series is to explain how DSPy changes the workflow from hand-written prompt adjustment to something closer to software engineering: define the task, build a baseline, evaluate it, and then optimise it.

The prompt that worked yesterday

Suppose we want to build a small customer-support classifier. A user sends a message, and our system needs to classify it into one of several categories, such as the following:

Refund Request
Technical Issue
Billing Question
Cancel Subscription

In a simple prototype, we might write a prompt like this:

You are a customer-support intent classifier.

Classify the following message into one of these categories:
Cancel Subscription, Refund Request, Technical Issue,
Account Update, Billing Question, Complaint, Unknown.

Return only the category name.

Message:

"Why is my bill higher than usual this month?"

A model might reasonably return:

Billing Question

That is not a bad answer: the user is asking about a charge, and the wording does not explicitly say that they want their money back. If we try another example, such as the following, the same prompt might return Cancel Subscription.

"Please cancel my subscription before it renews next month."

At this point, the prototype looks encouraging. The prompt appears to understand the label set, follow the format instruction, and produce a usable category.

Now consider a more awkward message:

"This is unacceptable. I want my money back, and I want to cancel everything."

This is no longer a clean single-intent example. The message contains dissatisfaction, a refund request, and a cancellation request. If the model returns Refund Request, it has captured one part of the message. If it returns Cancel Subscription, it has captured another. If it returns Complaint, that is also understandable, but probably less operationally useful if the support system needs to route the user to the right team.

The problem here is not simply that the prompt is too short. We could add definitions, examples, and tie-breaking rules, and we probably should. The deeper issue is that the prompt has become the place where we are trying to store the task definition, the label taxonomy, the output format, the ambiguity policy, and the routing logic. As the task becomes more realistic, the prompt becomes less like a simple instruction and more like an unstructured software component.

This is where the limits of casual prompting become visible. Prompt engineering can help us improve individual outputs, but it does not automatically give us a disciplined way to measure whether the system is improving overall. It also does not automatically tell us whether a new prompt is better across a representative set of examples, or whether it merely performs better on the few cases we happened to inspect manually.

Prompt engineering is useful, but limited

It is tempting to frame prompt engineering as an outdated practice, but that would be too simple. Prompt engineering is often the right starting point. When we are exploring a task, learning how a model behaves, or drafting a one-off response, it is entirely reasonable to interact with the model directly and refine the instruction by hand.

The limitation is not that prompts are useless. The limitation is that prompts are unstructured artefacts being asked to carry too much responsibility. A production prompt may contain a role description, task instructions, examples, output format requirements, error-handling rules, style guidance, and business constraints. Over time, it can become difficult to know which part of the prompt is doing useful work and which part is merely historical residue from earlier experiments.

This creates a maintenance problem. If a prompt contains twenty carefully worded clauses, changing one sentence may improve one case and damage another. A developer reviewing the change may struggle to understand whether the new wording is a genuine improvement, a local patch, or simply a different form of ambiguity. In traditional software development, we would not normally accept such an opaque and weakly tested component at the centre of a system.

Prompt engineering also makes model comparison harder than it first appears. If we switch from one model to another, the same prompt may behave differently. If we then adjust the prompt for the new model, we are no longer comparing only the models; we are comparing a model-and-prompt combination. This may be acceptable in practice, but it reinforces the need for a more systematic way to evaluate behaviour.

From prompts to programs

DSPy approaches the problem from a different angle. Instead of asking us to focus first on the exact wording of the prompt, it asks us to describe what the language-model component should do. In DSPy, this is usually expressed through a small task interface called a signature.

For example, our customer-support classifier can begin with a compact description:

"message, labels -> intent_label"

This should be read almost like a function signature. Given a message and a set of possible labels, produce an intent_label. The line does not contain the full prompt that will be sent to the model. Instead, it describes the task boundary: what information goes in, and what kind of answer should come out.

This is the core shift from prompt engineering to prompt programming. We are no longer treating the prompt as the main object we manually sculpt. We are treating the language-model call as part of a program: a component with inputs, outputs, and behaviour that can be evaluated. DSPy then handles the process of turning this task definition into prompts and model calls.

The word “programming” is important here, but it should not be misunderstood. DSPy does not make language models deterministic in the same way as ordinary Python functions. The model is still a statistical system, and its behaviour still depends on the model, the data, the task, and the surrounding context. What DSPy gives us is a more structured way to build around that uncertainty.

The basic DSPy workflow

The practical DSPy workflow can be understood in three stages: build a baseline, evaluate it, and optimise it. This is one of the reasons DSPy feels closer to machine-learning practice than traditional prompt engineering. We do not merely write an instruction and hope it works; we create a first version, measure it, and then try to improve it against a defined criterion.

The first stage is the baseline. A baseline is a simple working version of the system, not necessarily a good one. For our classifier, the baseline may simply call a language model with the signature shown above and return one predicted label. The baseline matters because it gives us something concrete to inspect and something measurable to improve.

The second stage is evaluation. Instead of judging the prompt only by looking at one or two outputs, we prepare examples with expected answers. For a classifier, that might mean a small dataset of customer messages and the correct intent labels. We then define a metric, such as whether the predicted label matches the expected label. This makes the discussion more precise: we can ask not merely whether the output looks plausible, but how often the program gets the task right on examples it has not seen.

The third stage is optimisation. Once we have examples and a metric, DSPy can search for better ways to run the task. Depending on the optimiser, this may involve selecting useful examples to include as demonstrations, refining instructions, or testing alternative configurations. The important point is that optimisation is guided by evaluation rather than by intuition alone.

A useful way to remember the structure is this: the signature describes the task interface, the module describes the execution strategy, and the metric describes the quality target. In later articles, these three pieces will become concrete. We will start with a simple classifier, measure its behaviour, and then use DSPy to improve it in a controlled way.

A running example for this series

The customer-support classifier above is only a motivating example of the problem. For the rest of this series, we will use a different running example: an IT operations incident classifier.

The task is deliberately modest, because the purpose is to learn the DSPy workflow rather than to build a complete enterprise monitoring or incident-management platform. A classifier is also a good starting point because it has clear inputs, clear outputs, and a simple evaluation metric.

The classifier will use labels such as:

Database Issue
Scheduler Issue
Application Server Issue
Network Issue
Access Permission Issue
Data Quality Issue
Storage Capacity Issue
Unknown

These labels are intentionally close enough to create some ambiguity. An alert such as “The overnight batch failed after the database connection timed out” may sit between Scheduler Issue, Database Issue, and Network Issue, depending on whether the system should classify the failed operational process, the immediate technical error, or the suspected infrastructure cause. This is useful for learning, because real LLM applications rarely fail only on clean textbook examples. They fail at the boundaries between categories, where the task definition itself needs to be made more precise.

In the next article, we will build the first runnable DSPy version of this classifier. We will use a typed signature, where the supported incident categories are written directly into the task contract using Python’s Literal type. This gives us a clearer interface from the beginning, but it does not tell us whether the classifier is accurate — which is why the following article focuses on evaluation. This gradual progression is intentional: it mirrors how many LLM applications evolve from a simple prototype into a more structured system.

Where DSPy is useful

DSPy is most useful when an LLM task needs to run repeatedly. Classification, information extraction, structured summarisation, retrieval-augmented generation, and multi-step agentic workflows are all good candidates. In these cases, the problem is not just whether the model can produce a good answer once, but whether the application can produce acceptable outputs across many inputs and under changing conditions.

DSPy is also useful when different parts of an application depend on language-model outputs. In a multi-step workflow, one model call may classify a request, another may retrieve context, another may generate an answer, and another may check whether the answer is grounded. If one step produces unreliable output, later steps may amplify the error. A structured and measurable approach becomes increasingly valuable as the workflow grows.

However, DSPy is not necessary for every interaction with a language model. If we are asking for a one-off summary, drafting a short email, or brainstorming ideas, a normal prompt is often sufficient. In such cases, the overhead of writing code, preparing examples, and defining a metric may not be justified. DSPy becomes more attractive when the task is repeated, the quality matters, and improvement needs to be measured.

What comes next

This article has focused on the motivation for DSPy rather than its full API. The important idea is that prompt programming changes the unit of work. We are no longer only writing prompts; we are defining LLM components that can be run, evaluated, and improved.

In the next article, we will build the first small DSPy program using the IT operations incident classifier introduced here. We will define a simple signature, call a language model through dspy.Predict, and inspect the returned prediction. That first version will not yet be fully evaluated or optimised, but it will give us the baseline from which the rest of the series can develop.

References and further reading

Stanford DSPy project and documentation. https://dspy.ai/

Omar Khattab et al. (2023), DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines, arXiv:2310.03714. https://arxiv.org/abs/2310.03714

Serj Smorodinsky and William Brett Kennedy, Building LLM Applications with DSPy, Manning Early Access Program. https://www.manning.com/books/building-llm-applications-with-dspy