The Hardest Part of AI Is Defining What We Mean

A product-side reflection on DPO, visual language models, and why preference learning still depends on human clarity.

AI Generated Image

One of the most humbling lessons I have learned working near AI systems is that the hardest problems are rarely only technical.

They are often definitional.

I have been involved in work where visual language models are used to interpret real-world images. On the surface, the task sounds straightforward: look at an image, understand what is happening, and decide how the system should respond.

But that simple description hides a much harder problem.

Object detection can tell you what is present. It can identify vehicles, people, objects, markings, obstacles, and other visible elements in a scene. A visual language model, or VLM, is asked to do something more subtle: describe the situation, infer context, and make a judgment about what the image means.

The difference is the difference between recognizing objects and understanding a situation.

That distinction matters enormously in practice.

Take something that sounds simple: identifying congestion in a road scene. Most people, shown a photograph of cars packed closely together, would make an intuitive judgment almost instantly. But when you try to teach a machine to make that same determination, you are forced to answer questions humans rarely need to ask explicitly.

What counts as congestion?

Is a line of vehicles waiting at a light congestion, or just normal traffic behavior? What if the image does not show enough surrounding context to make that distinction? What if vehicles appear stopped, but are actually parked? What if the scene looks abnormal from one camera angle but perfectly normal from another?

Every definition creates consequences.

If the condition is defined too broadly, the system may flag ordinary situations as problems. If it is defined too narrowly, it may miss the cases that matter. If we add more rules, we gain control but also introduce more failure points. If we rely too heavily on model judgment, we gain flexibility but lose predictability.

The more precisely you try to define a condition, the more edge cases appear.

And every edge case is a place where the model may confidently give the wrong answer.

This is what I have come to think of as the definition problem. It is not simply a flaw in the technology. It is a gap in how humans communicate knowledge we have never needed to formalize before — because until recently, we were the ones doing the seeing.

It was in this context that I became interested in Direct Preference Optimization, or DPO.

I am not an AI researcher, and this is not meant to be a technical explanation of the algorithm. I am writing from the position of someone working close enough to applied AI to see how these methods enter real products and workflows — and how much depends on the quality of the human judgment surrounding them.

The appeal of DPO, as I understood it, is practical. Instead of only teaching a model from examples of correct outputs, DPO allows the model to learn from preference pairs: one answer is preferred, another is rejected. In theory, this is powerful because many real-world tasks are not only about factual correctness. They are about judgment.

A response can be technically plausible but still not useful.

A description can be detailed but miss the point.

An answer can be fluent but misaligned with what the situation actually requires.

Preference learning seems attractive because it gives humans a way to say: not just “this is right,” but “this is better than that.”

For applied AI work, that distinction feels important.

But in practice, I also saw how difficult preference design can become.

For narrow binary questions, preference pairs are relatively straightforward. If the task is to answer yes or no, the preferred and rejected responses can often be constructed clearly. The image either contains the condition being asked about, or it does not.

But open-ended prompts are different.

When the question becomes “describe what is happening in this image,” the space of possible answers expands dramatically. There may be many acceptable descriptions, many bad descriptions, and many descriptions that are not exactly wrong but are still less useful than others.

Designing meaningful rejected answers becomes much harder.

What should the model learn from the contrast? That one answer is too vague? Too specific? Too confident? Missing context? Overreacting? Underreacting? Using the wrong level of detail?

Preference learning depends on humans knowing what they prefer — and being able to express that preference clearly enough for a machine to learn from it.

That is a much harder requirement than it first appears.

What I took away from the experience is not that DPO is ineffective. Quite the opposite: I can see why it has become such an important method. But I also learned that no optimization method can rescue a fuzzy definition.

If the task is unclear, the preference signal will be unclear.

If the preference signal is weak, inconsistent, or too small to cover the range of real-world ambiguity, the model may not improve in the way we hope.

The experiment taught me something I now think about far beyond AI: tools are only as good as the clarity we bring to them.

A model can optimize for a preference.

But first, humans have to know what they mean.

When the definition is fuzzy, even the best optimization cannot sharpen it.

That is a problem no algorithm solves on its own.

This essay reflects my personal learning and does not describe any specific company system, product, dataset, or internal result.