Detecting LLM Hallucinations at Generation Time with UQLM

By: Dylan Bouchard, Mohit Singh Chauhan, David Skarbrevik, Viren Bajaj, Ho-Kyeong Ra, and Zeya Ahmad

Why should we care about hallucinations?

Large language models (LLMs) have revolutionized the field of natural language processing, but their tendency to generate false or misleading content, known as hallucinations, significantly diminishes safety and trust. LLM hallucinations are especially problematic because they often appear plausible, making them difficult to detect. As LLMs are increasingly deployed in real-world settings, addressing hallucinations is not just a technical challenge; it is essential for safeguarding the integrity of systems that rely on LLMs and for fostering a responsible and ethical approach to AI development.

How can hallucinations be detected?

Most evaluation toolkits “grade” model outputs against ground truth (think: Evals, G-Eval). While effective for offline testing with ground truth data, this approach doesn’t suit real-time systems that generate outputs without access to ground truth. Other approaches compare responses to provided source content (RAG-centric metrics) or fetch evidence from the web. Those can help in certain use cases but depend on having the right context to answer the question.

This is where uncertainty quantification (UQ) shines. UQ enables scoring each response at generation time, without needing ground truth or external retrieval. UQ techniques can be classified into three practical families:

• Black-box UQ: sample multiple answers to the same question and measure semantic agreement.
• White-box UQ: use token probabilities from the generator to assess confidence, with no extra generations.
• LLM-as-a-Judge: ask an LLM to score the correctness of (question, answer).

What is UQLM?

To make these UQ techniques easy to use, we built UQLM (Uncertainty Quantification for Language Models), an open-source Python library that:

• standardizes diverse signals to a common [0,1] confidence (higher = more likely correct),
• scores at generation time, with minimal code,
• and supports black-box, white-box, and judge scorers out of the box (plus an optional ensemble, not used in this post, but detailed further here).

In this tutorial, we’ll run a small, realistic demo on SimpleQA, a factual QA dataset where each row has a question and answer. Specifically, we will:

• generate responses with your favorite LLM,
• score each response using UQLM’s black-box, white-box, and judge scorers,
• quickly evaluate hallucination detection quality (AUROC, AUPRC),
• and demonstrate how thresholding scores can enable practical action: blocking low-confidence outputs or routing them to a human.

The goal of this article is to show how you can add generation-time confidence to your LLM system in minutes, using a library that meets you where you are.

Quick start: Install UQLM and set up your keys

PyPI installation

UQLM is available on PyPI, so you can install it using pip. To avoid dependency conflicts, we’ll create a venv virtual environment and activate it before installing UQLM. This can be done by running the following commands in your terminal.

python -m venv uqlm
source uqlm/bin/activate
pip install uqlm

Configure your provider

UQLM talks to your LLM via LangChain’s BaseChatModel. In this demo, we will use GPT-4o with AzureChatOpenAI, but note that any LangChain Chat Model can be used.

We will load our API credentials from our .env file using the dotenv package. Note that this step relies on proper naming conventions for the environment variables.

from dotenv import load_dotenv, find_dotenv
load_dotenv(find_dotenv)

Now, we construct our LLM object.

from langchain_openai import AzureChatOpenAI
llm = AzureChatOpenAI(
    deployment_name="gpt-4o"
    openai_api_type ="azure"
    openai_api_version="2024–02–15-preview"
)

That’s it for setup. Next, we’ll pull a sample of SimpleQA questions for generating and scoring responses.

Load example data: SimpleQA

We’ll use SimpleQA in this tutorial. The data can be downloaded as a csv file from Hugging Face.

import pandas as pd
df = pd.read_csv("simple_qa_test_set.csv").rename(columns={"problem": "question"})
df.head()

Snapshot of SimpleQA Data

For convenience, we’ll form a list of prompts and a specify task instruction to use as our system prompt for generation.

task_instruction = "When you answer the following question, return only the answer without providing detailed explanation."
prompts = df["question"].to_list()

Which scorer should I start with?

Not all use cases (or APIs) are the same. Here’s a practical guide to help you navigate to the section that best fits your requirements.

Quick Decision Guide

Additional considerations

• White-box methods are ideal for production environments with strict latency requirements.
• Black-box approaches are best for high-stakes scenarios where safety outweighs cost and latency concerns.
• Judge methods offer a compromise when you lack token probabilities but can’t afford multiple generations.

Ready to dive deeper? Continue to the section that matches your needs or read through all approaches for a comprehensive understanding. Note that although we do not provide detailed scorer definitions in this article, these can be found in our companion research paper, “Uncertainty Quantification for Language Models: A Suite of White-Box, Black-Box, LLM Judge, and Ensemble Scorers.”

White-box UQ

We will first explore white-box UQ as a method for hallucination detection. These scorers use the token probabilities returned by your model to produce a confidence score in [0,1] for each response. The white-box scorers we consider here require only the token probabilities from the original generated response, meaning they add no additional cost and have negligible impact on latency.¹ An illustration of this workflow is depicted in the figure below.

Illustration of a simple White-Box UQ workflow

Here, we consider two white-box UQ scorers offered by UQLM:

• Length-normalized token probability (LNTP): geometric mean of the token probabilities across the entire generated response (Malinin & Gales, 2021).
• Minimum token probability (MTP): the minimum token probability in the generated response (Manakul et al., 2023).

The code below implements simultaneous response generation and scoring using these two scorers.

from uqlm import WhiteBoxUQ
wbuq = WhiteBoxUQ(
    llm=llm, 
    scorers=["normalized_probability", "min_probability"], 
    system_prompt=task_instruction
)
wb_results = await wbuq.generate_and_score(prompts=prompts)
wb_df = wb_results.to_df()
wb_df.head()

Snapshot of UQLM’s White-Box UQ output

We convert the returned object to a dataframe containing prompts, responses, and associated confidence scores for each response.

Black-box UQ

Black-box UQ scorers generate multiple responses to the same question (i.e., non-zero temperature) and measure how semantically similar they are with the original response. When the model’s outputs vary, these signals are often the most discriminative. The graphic below illustrates this workflow.

Illustration of Black-Box UQ workflow

As of v0.3.0, UQLM offers the following black-box UQ scorers:

• Exact match rate (EMR): Fraction of resampled answers that are identical to the original. Great for closed-form answers; sensitive to harmless rephrasing. (Cole et al., 2023; Chen & Mueller, 2023)
• Noncontradiction probability (NCP): Uses an NLI model to score how often candidates do not contradict the original (and vice-versa). Robust to paraphrases; highest latency due to NLI calls. (Chen & Mueller, 2023; Lin et al., 2024; Manakul et al., 2023)
• Normalized semantic negentropy (NSN): Clusters candidates by mutual entailment (NLI) and converts cluster entropy into a confidence score (lower entropy ⟹ higher confidence). Often strongest when answers vary; highest latency due to NLI calls. (Farquhar et al., 2024; Bouchard & Chauhan, 2025)
• BertScore confidence (BSC): Token-level similarity via contextual embeddings (BertScore-F1) between the original and each candidate. Captures softer phrasing changes; high latency if no GPU available. Typically less effective than NLI-based methods (Manakul et al., 2023; Zheng et al., 2020).
• Normalized cosine similarity (NCS): Sentence-level embedding similarity (cosine) averaged over candidates. Lightweight and fast, but typically less effective than NLI-based methods. (Shorinwa et al., 2024; HuggingFace)

The code below implements response generation and scoring using these all five scorers.

from uqlm import BlackBoxUQ
bbuq = BlackBoxUQ(
    llm=llm,
    device=device,
    scorers=["exact_match", "bert_score", "cosine_sim", "noncontradiction", "semantic_negentropy"],
    system_prompt=task_instruction
)
bb_result = await bbuq.generate_and_score(prompts=test_prompts)
bb_df = bb_result.to_df()
bb_df.to_df()

Snapshot of UQLM’s Black-Box UQ output

We convert the returned object to a dataframe containing prompts, responses, and associated confidence scores for each response. Note that the dataframe also contains the sampled candidate responses that were used to compute the black-box confidence scores.

LLM as a judge

Judge scorers ask an LLM to rate the correctness of a [question, response] pair and return a standardized [0,1] confidence score. They’re handy when you can’t afford multiple resamples (i.e., black-box UQ) or don’t have token probability access (i.e., white-box UQ). In practice, they’re often less effective than strong black-box signals, but they’re simple and have lower overhead (one extra call per judge). Users can use one or more LLM judge and aggregate judgements across judges to obtain a single confidence score per response for our original LLM.

Illustration of LLM-as-a-Judge workflow

With UQLM, this LLM-as-a-judge workflow is implemented using the LLMPanel class. In the constructor, users pass a list of LLM objects to the judges argument and specify one of four scoring templates for each judge with the scoring_templates argument. The four LLM-as-a-judge scoring templates are offered with UQLM are as follows:

• Continuous: any value between 0 and 1 (based on Xiong et al., 2024)
• Binary: incorrect (0) vs. correct (1) (based on Manakul et al., 2023)
• Ternary: incorrect (0), uncertain (0.5), or correct (1) (based on Chen & Mueller, 2023)
• Likert: very bad (0), bad (0.25), neutral (0.5), good (0.75), very good (1) (based on Bai et al., 2023)

In our demo, we will use the continuous scoring template and use two LLM judges. We will first instantiate three additional LLM objects to be used as judges. For these, we will use the ChatVertexAI and the AzureChatOpenAI classes.

from langchain_google_vertexai import ChatVertexAI
gemini_15_flash = ChatVertexAI(model_name="gemini-1.5-flash")
gemini_15_pro = ChatVertexAI(model_name="gemini-1.5-pro")
gpt4o_mini = AzureChatOpenAI(
    deployment_name="gpt-4o-mini"
    openai_api_type ="azure"
    openai_api_version="2024–02–15-preview"
)

We can then pass these LLM objects to our LLMPanel class to generate and score responses from our original LLM.

from uqlm import LLMPanel
panel = LLMPanel(
    llm=llm, 
    judges=[gemini_15_flash, gemini_15_pro, gpt4o_mini], 
    system_prompt=task_instruction
)
judge_results = await panel.generate_and_score(prompts=test_prompts)
judge_df = judge_results.to_df().rename(columns={"judge_1": "gem_15_flash_judge", "judge_2": "gem_15_pro_judge", "judge_3": "gpt4o_mini_judge"}
judge_df.to_df()

Snapshot of UQLM’s LLM Panel output

Above, we convert the returned object to a dataframe containing prompts, responses, and judge-based confidence scores for each response. Note that the dataframe also contains basic statistics across the scores from the various judges for each response: minimum, maximum, mean, and median.

Offline evaluation: Comparing hallucination detection ability across scorers

Grade LLM responses against an answer key

To evaluate the hallucination detection of our scorers [offline], we must first determine which responses actually contain hallucinations. To do this, we grade the LLM responses against our answer key for the SimpleQA questions. We will use vectara/hallucination_evaluation_model as our grader model. The grader model object is instantiated using the transformers library.

from transformers import AutoModelForSequenceClassification
grader_model = AutoModelForSequenceClassification.from_pretrained("vectara/hallucination_evaluation_model", trust_remote_code=True)

family_to_df = {"white_box": wb_df, "black_box": bb_df, "judge": judge_df}
for scorer_family in family_to_df:
    result_df = family_to_df[family_to_df]
    grade_scores = grader_model.predict([(r, a) for r, a in zip(result_df["response"], df["answer"])])
    result_df["is_factually_correct"] = [(float(gs) > 0.5) for gs in grade_scores]

Now, we can calculate baseline LLM accuracy before confidence scores are used:

wb_df["is_factually_correct"].mean()
# .3

The graded responses indicate that the LLM answers 30% of the questions correctly and 70% of the questions incorrectly.

Hallucination detection as a classification problem

We’ll measure how well each scorer separates incorrect answers from correct ones. For evaluation, we’ll treat hallucination detection as a binary classification problem:

• Ground truth: contains_hallucination (1 = response has hallucination, 0 = response is factually correct)
• Prediction: hallucination_score (higher values indicate greater likelihood of hallucination, equal to 1 — confidence score)

Following this approach, we can compute AUROC (Area Under the Receiver Operating Characteristic curve) and AUPRC (Area Under the Precision-Recall Curve) to evaluate how well each scorer detects hallucinations:

from sklearn.metrics import roc_auc_score, average_precision_score
family_to_scorer_names = {
    "white_box": ["normalized_probability", "min_probability"],
    "black_box": ["exact_match", "bert_score", "cosine_sim", "noncontradiction", "semantic_negentropy"],
    "judge": ["gem_15_flash_judge", "gem_25_flash_judge", "gpt4o_mini_judge"]
}
auroc_scores = {}
aurpc_scores = {}
for scorer_family in family_to_scorer_names:
    result_df = family_to_df[scorer_family]
    contains_hallucination = ~result_df["is_factually_correct"] # indicates whether hallucination actually occurred
for scorer in family_to_scorer_names[scorer_family]:
    hallucination_score = [1 - s for s in result_df[scorer]] # convert confidence scores to uncertainty scores
    auroc_scores[scorer] = roc_auc_score(y_score=hallucination_score, y_true=contains_hallucination)
    aurpc_scores[scorer] = average_precision_score(y_score=hallucination_score, y_true=contains_hallucination)

Let’s rank and plot our scorers by AUROC and AUPRC. Note that the baseline for AUROC is 0.5 and for AUPRC is 1 — LLM accuracy = 0.7.²

Ranked Scorer-Specific AUROC Values (Baseline=0.5)

Ranked Scorer-Specific AUPRC Values (Baseline=0.7)

Results at a glance

• Overall ranking: black-box UQ > white-box UQ > LLM judge
• Black-box UQ: AUROC 0.69–0.77, AUPRC 0.84–0.89
• White-box UQ: AUROC 0.71–0.72, AUPRC 0.83
• LLM judge: AUROC 0.56–0.58, AUPRC 0.73–0.75

Takeaway: If you can afford a few extra generations, NLI-style black-box scorers give the best hallucination detection performance. White-box is a strong, minimal-latency default. LLM judge is a compact fallback when resample is too expensive and token probabilities are not available.

Thresholding confidence scores for practical action

Finally, we explore “filtered accuracy” as a metric for evaluating the performance of our confidence scores. Filtered accuracy measures the change in LLM accuracy when responses with confidence scores below a specified threshold are excluded. By adjusting the confidence score threshold, we can observe how the accuracy of the LLM improves as less certain responses are filtered out.

We plot the filtered accuracy across various confidence score thresholds (e.g., 0.1, 0.2,…,0.9) to visualize the relationship between UQLM confidence scores and LLM accuracy. We also include the top scorer from each family based on AUROC in our plot: Noncontradiction probability (black-box), length-normalized token probability (white-box), and Gemini-1.5-Pro LLM judge.³

LLM accuracy when when responses with confidence < threshold are filtered out. We consider thresholds (0.1, 0.2,…,0.9) and plot results when filtering with the top black-box UQ, white-box UQ, and LLM judge scorers.

The plot tells a clear story. The top black-box and white-box scorers deliver strong accuracy gains from filtering out low confidence responses. In contrast, the top LLM judge provides only modest lifts.

In practice, confidence scores become valuable when translated into specific actions. By analyzing filtered accuracy at various confidence thresholds, teams can determine optimal cutoff points for their specific use cases. Here are three practical ways to leverage confidence scoring:

1. Block low-confidence responses: Implement a scoring system that automatically filters out responses below a certain confidence threshold (e.g., <0.4), ensuring only high-quality, reliable outputs reach users.
2. Targeted human-in-the-loop: Route low-confidence responses to a SME for manual review. This approach is valuable when exhaustive human-in-the-loop is infeasible due to the scale of responses being generated, and is significantly more efficient than random sampling, focusing human attention where it’s most needed.
3. Low-confidence disclaimers: Provide end-users of the LLM output with disclaimers for low-confidence for responses with confidence below a certain threshold. This transparency helps users understand the reliability of the information provided.

For example, in a customer service AI, you might set thresholds where responses below 0.4 confidence are blocked entirely and those between 0.4–0.7 are routed to human agents.

When selecting confidence thresholds, consider both your specific use case and risk profile. The optimal threshold balances false positives (i.e., incorrectly flagging accurate responses) against false negatives (i.e., failing to catch hallucinations), based on the error type that is more costly/egregious in your specific application.

Caveats

A few important caveats to be aware of:

• Hallucination detection performance depends on the dataset/task and LLM used. Here we consider only a single LLM and dataset/task.
• Performance of LLM judges will vary with different instruction templates/prompts and baseline model. LLM judge performance will be better for higher-performance LLMs.
• While scores are very useful for distinguishing correct vs. incorrect responses, they are not calibrated out of the box — a confidence score of 20% does not imply 20% chance the response is correct.

For an extensive set of experiments with various LLMs and datasets/tasks, refer to UQLM’s associated research paper.

Conclusion

In this tutorial we used UQLM to generate answers for SimpleQA and score them three ways: white-box UQ, black-box UQ, and LLM-as-a-judge, all on a shared [0,1] confidence scale. We then evaluated AUROC/AUPRC and turned scores into action with filtered accuracy vs. threshold. The result: a generation-time workflow you can drop into real systems with minimal code.

Where to go next

• Use UQLM with your data and different models, adjusting parameters (e.g., num_responses or temperature).
• Set an operating threshold for your application (e.g., route low-confidence to human review).
• Add logging and dashboards to track coverage and accuracy over time.
• Explore the **ensemble** in UQLM when you want a single meta-score for a higher-performance scorer optimized for your specific use case.

Resources

• GitHub: **UQLM** — examples, API docs, and issues.
• Research Paper: **A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers** — scorer definitions, ensemble method, and extensive experiments.
• Software Paper: **A Python Package for Uncertainty Quantification in Large Language Models** — comparison to other toolkits and API description (accepted with minor revisions to JMLR).
• Docs site: **UQLM Sphinx Documentation** — API documentation, FAQs, and more.
• Demo notebooks: **UQLM tutorials** — runnable end-to-end demos.

If this was useful, give the repo a ⭐️, open an issue with feedback, or create a PR. Also, we’d love to hear how you’re using UQLM: share your success stories in GitHub **Discussions or tags on social with #UQLM**.

Endnotes

[1]: Certain white-box methods, such as logprobs-based semantic entropy and semantic density, require generating multiple responses per prompt and therefore add cost and latency. We do not consider these here.

[2]: These figures can be plotted using uqlm.utils.plot_scorer_ranks.

[3]: This figure can be plotted using uqlm.utils.plot_filtered_accuracy.

References

1. Dylan Bouchard and Mohit Singh Chauhan. Uncertainty quantification for language models: A suite of black-box, white-box, llm judge, and ensemble scorers, 2025. URL https: //arxiv.org/abs/2504.19254.

2. Bouchard, Dylan, Mohit Singh Chauhan, David Skarbrevik, Ho-Kyeong Ra, Viren Bajaj and Zeya Ahmad. “UQLM: A Python Package for Uncertainty Quantification in Large Language Models.” ArXiv abs/2507.06196 (2025): n. pag.

3. Jiuhai Chen and Jonas Mueller. Quantifying uncertainty in answers from any language model and enhancing their trustworthiness. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5186–5200, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long. 283. URL https://aclanthology.org/2024.acl-long.283/.

4. Jeremy Cole, Michael Zhang, Daniel Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. Selectively answering ambiguous questions. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 530–543, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.35. URL https:// aclanthology.org/2023.emnlp-main.35/.

5. Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8017):625–630, Jun 2024. ISSN 1476–4687. doi: 10.1038/s41586–024–07421–0. URL https://doi.org/10. 1038/s41586–024–07421–0.

6. Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantification for black-box large language models, 2024. URL https://arxiv.org/abs/ 2305.19187.

7. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. Geval: NLG evaluation using gpt-4 with better human alignment. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.153. URL https://aclanthology.org/2023.emnlp-main.153/.

8. Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction, 2021. URL https://arxiv.org/abs/2002.07650.

9. Potsawee Manakul, Adian Liusie, and Mark Gales. SelfCheckGPT: Zero-resource blackbox hallucination detection for generative large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9004–9017, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.557. URL https://aclanthology.org/2023.emnlp-main.557/.

10. Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z. Ren, and Anirudha Majumdar. A survey on uncertainty quantification of large language models: Taxonomy, open research challenges, and future directions, 2024. URL https://arxiv.org/abs/2412.05563.