Why Precision/Recall Is Not Enough for LLM Safety

Every machine learning course makes sure to teach you precision and recall. Every safety team monitors them. Every single time that I come across a safety system being measured solely by precision and recall, my heart skips a beat or two.

It’s not that precision and recall are bad metrics. They have their strengths. They are essential (that much is obvious). They are necessary and important and should be taken into consideration. But, they are not enough when it comes to measuring LLM safety. Here is why.

The Metrics We Inherited

Precision and recall were built for a more innocent time: that of a model receiving some input, classifying it as being or not being something. Being a violation or not. A spam message or not a spam message. Toxic or not toxic.

Precision would answer this question for you: out of all the things that you’ve classified as being a violation, how many are true positives? Recall would answer: out of all the real violations there, how many have you caught?

These metrics are great when the problem space is defined clearly, the data distribution is stable, and the penalties of being incorrect on one side are comparable to being incorrect on the other side. Classic content moderation was close enough to fitting those criteria . Not LLM safety.

Where the Metrics Break

The boundary problem. “Safe” and “unsafe” LLM outputs are not always binary concepts. They depend on the circumstances of each specific case and often become contested issues. An answer to a question that may be considered fully appropriate from an adult researcher may be unsuitable for teenagers. What looks like the truth in one situation may appear to be misleading information if taken out of its context.

The moment you start forcing this complexity into your precision-recall calculations, you make the judgment choices that your metrics will happily cover up for you. Who made that call? How? Your decisions were included in your data. And if they were wrong, your metrics will happily tell you that all is well and good.

The distribution problem. Precision and recall are calculated against a test set, and that test set is a snapshot: a specific distribution of content at a specific moment in time.

But the distribution of harmful content LLMs encounter isn’t static. It shifts as users discover new prompting strategies, as cultural context changes, and as adversaries develop new attacks. Your model might post 95% precision on last month’s test set while systematically missing an entirely new category of harm that didn’t exist when the set was created.

I’ve watched this happen. A system’s eval metrics look stable or improving while the actual experience of harm gets worse because the eval doesn’t capture the new attack surface.

The asymmetry problem. Precision and recall treat all errors as equivalent within their category. A false positive on a cooking recipe is weighted the same as a false positive on political speech. A missed piece of spam counts the same as a missed piece of content that could lead to real-world violence.

In LLM safety, the cost of errors is wildly asymmetric. Missing a slightly snarky response is not the same as missing content that helps someone cause real harm. Precision and recall don’t know the difference.

The temporal problem. Traditional metrics give you a point-in-time measurement. They tell you how the system performed on this set of examples. They don’t tell you how performance is changing, whether the system is getting more brittle, or whether new failure modes are emerging.

For LLM safety, the trajectory matters as much as the snapshot. A system at 90% precision but declining is in a fundamentally different state than one at 85% precision and improving. Reported as single numbers, precision and recall don’t capture this.

What Should We Measure Instead?

I’m not arguing we throw out precision and recall. They’re useful as part of a larger picture. But they need to be supplemented with metrics that capture what actually matters for LLM safety.

Harm-weighted accuracy. Not all errors are equal. Weight false negatives by the severity of the harm they represent — a missed violent-extremism response should count for more than a missed mildly rude one. This forces you to be explicit about your harm taxonomy and your priorities, which is uncomfortable but necessary.

Adversarial robustness over time. Don’t just measure how the system performs — measure how it performs under attack, and how that changes as adversaries adapt. That means continuous red-teaming with metrics tracked longitudinally, not one-off evaluations.

Coverage of emergent harms. Explicitly measure how quickly the system detects and responds to new categories of harm that weren’t in the original training or evaluation data. This is the hardest metric to operationalize, and it might be the most important.

False positive impact. Measure not just the rate of false positives but their downstream effect. Are they concentrated in specific communities? Disproportionately affecting certain languages or topics? A 2% false positive rate sounds low until you realize it’s 10% for content in a specific language.

Calibration and uncertainty. Does the system know when it doesn’t know? A model that outputs high-confidence predictions on ambiguous content is more dangerous than one that flags uncertainty and defers to human judgment. Measure calibration explicitly.

The Deeper Issue

Underneath all of this is a philosophical problem metrics alone can’t solve: what are we optimizing for?

Precision and recall implicitly answer that question with “correctness on the labeled data.” But correctness against what? Labeled by whom? Using what rubric?

LLM safety requires us to be far more deliberate about this. The metrics we choose encode our values - what we consider harmful, how we weight different types of harm, who we’re protecting and from what. If we default to precision and recall because they’re familiar, we’re outsourcing those value decisions to the structure of our test set.

That’s not good enough. Not for systems operating at this scale, with this much influence.

We need metrics that are honest about what they measure, and humble about what they don’t.