Evaluating LLM Health Responses with DeepEval: A Practical Example

As Large Language Models (LLMs) are increasingly used in sensitive domains like healthcare, evaluating the quality and correctness of their responses becomes critical. Traditional unit tests aren’t enough — we need semantic evaluation.

In this post, we’ll walk through a simple but powerful example using DeepEval, a framework designed specifically for testing LLM outputs using LLM-based evaluators. we will create a simple evaluation using DeepVal as a evaluation framework

reference official documentation -> (https://deepeval.com/docs/getting-started)

Why LLM Evaluation Is Different

Unlike deterministic code, LLMs:

• Generate probabilistic outputs
• May phrase correct answers differently
• Need to be judged on meaning, not exact wording .

What This Sample Code Does

The provided test evaluates whether an LLM’s response to a health-related question is correct when compared to an expected, well-written answer.

The test checks:

• Does the actual output convey the same medical guidance as the expected output?
• Does it warn the user appropriately?
• Is the response medically reasonable?

Code Walkthrough

1. Loading Dependencies and Environment Variables

from deepeval import assert_test
from deepeval.test_case import LLMTestCase, LLMTestCaseParams
from deepeval.metrics import GEval
from dotenv import load_dotenv

• DeepEval provides the evaluation framework.
• GEval is a grading-based metric powered by an LLM.
• .env is loaded to securely access API keys (e.g., OpenAI).

2. Defining the Correctness Metric

correctness_metric = GEval(
    name="Correctness",
    criteria="Determine if the 'actual output' is correct based on the 'expected output'.",
    evaluation_params=[
        LLMTestCaseParams.ACTUAL_OUTPUT,
        LLMTestCaseParams.EXPECTED_OUTPUT
    ],
    threshold=0.5
)

This metric:

• Uses an LLM to judge correctness — here you can use any LLM
• Compares actual output vs expected output
• Passes the test if the similarity score is ≥ 0.5

💡 The threshold allows flexibility — important since LLMs rarely produce identical wording.

3. Creating an LLM Test Case

test_case = LLMTestCase(
    input="I have a persistent cough and fever. Should I be worried?",
    actual_output=(
        "A persistent cough and fever could be a viral infection "
        "or something more serious. See a doctor if symptoms worsen "
        "or don't improve in a few days."
    ),
    expected_output=(
        "A persistent cough and fever could indicate a range of illnesses, "
        "from a mild viral infection to more serious conditions like pneumonia "
        "or COVID-19. You should seek medical attention if your symptoms worsen, "
        "persist for more than a few days, or are accompanied by difficulty "
        "breathing, chest pain, or other concerning signs."
    )
)

This test case includes:

• User input (health concern)
• Actual LLM response (from your app)
• Expected response (ideal medical guidance)

The expected output is more detailed, but the actual output is still medically sound — which is exactly what the evaluator checks.

4. Running the Assertion

assert_test(test_case, [correctness_metric])

4. output we will

>deepeval test run test_deepeval_1.py

.Running teardown with pytest sessionfinish...

============================================================================================= slowest 10 durations ==============================================================================================
8.25s call     test_deepeval_1.py::test_correctness

(2 durations < 0.005s hidden.  Use -vv to show these durations.)
1 passed, 4 warnings in 8.25s
                                                                                                  Test Results
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test case                                                                  ┃ Metric              ┃ Score                                                                      ┃ Status ┃ Overall Success Rate ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ test_correctness                                                           │                     │                                                                            │        │ 100.0%               │
│                                                                            │ Correctness [GEval] │ 0.6 (threshold=0.5, evaluation model=gpt-4.1, reason=The actual output     │ PASSED │                      │
│                                                                            │                     │ partially aligns with the expected output by mentioning that a persistent  │        │                      │
│                                                                            │                     │ cough and fever could be a viral infection or something more serious, and  │        │                      │
│                                                                            │                     │ advises seeing a doctor if symptoms worsen or don't improve. However, it   │        │                      │
│                                                                            │                     │ lacks specific examples of serious conditions (like pneumonia or           │        │                      │
│                                                                            │                     │ COVID-19), omits additional warning signs (difficulty breathing, chest     │        │                      │
│                                                                            │                     │ pain), and does not specify the timeframe as clearly as the expected       │        │                      │
│                                                                            │                     │ output. These omissions reduce completeness and specificity., error=None)  │        │                      │
│ Note: Use Confident AI with DeepEval to analyze failed test cases for more │                     │                                                                            │        │                      │
│ details                                                                    │                     │                                                                            │        │                      │
└────────────────────────────────────────────────────────────────────────────┴─────────────────────┴────────────────────────────────────────────────────────────────────────────┴────────┴──────────────────────┘

⚠ WARNING: No hyperparameters logged.
» Log hyperparameters to attribute prompts and models to your test runs.

================================================================================
Results saved in ./data as 20251223_165851

✓ Evaluation completed 🎉! (time taken: 8.56s | token cost: 0.002438 USD)
» Test Results (1 total tests):
   » Pass Rate: 100.0% | Passed: 1 | Failed: 0

 ================================================================================

» Want to share evals with your team, or a place for your test cases to live? ❤️ 🏡
  » Run 'deepeval view' to analyze and save testing results on Confident AI.

This line:

• Runs the evaluation
• Fails the test if the LLM response is deemed incorrect
• *if you are using confidentAI as UI to analyze the results you need to do few additional steps. in case you want to analyze the results on local you need to add the below env variable

DEEPEVAL_RESULTS_FOLDER=”./data” DEEPEVAL_LOGGING=true

Why This Matters

Using DeepEval LLM testing allows you to:

✅ Catch unsafe or misleading responses ✅ Maintain consistent quality across model updates ✅ Test semantics instead of string equality ✅ Build trust in sensitive applications

you might be wondering here who is judging what ?

Short answer

One LLM generates the response, and a different LLM (the evaluator) judges it.

What’s happening in your code

There are two distinct LLM roles involved:

1️⃣ The Application LLM (the one being tested)

This is the model you are building or integrating.

• It produces the actual_output
• In your example, it’s implied, not explicitly shown

actual_output="A persistent cough and fever could be a viral infection or something more serious..."This output might come from:

• GPT-4 / GPT-3.5
• Claude
• Llama
• Any custom or fine-tuned model

DeepEval does not care which model generated this — it just evaluates the text.

2️⃣ The Evaluator LLM (the judge)

This is the LLM used internally by GEval to score correctness.

By default:

• DeepEval uses OpenAI GPT-4 (or GPT-4o / GPT-4-Turbo depending on config) as the evaluator
• This model compares:
• actual_output
• expected_output
• Then produces a score between 0 and 1

This evaluator acts like a grader or reviewer, not a chatbot.

Mental Model

Think of it like this:

┌──────────────────┐
│  Your LLM App    │
│  (GPT, Claude,   │
│   Llama, etc.)   │
└────────┬─────────┘
         │ actual_output
         ▼
┌──────────────────┐
│  Evaluator LLM   │
│  (GPT-4 by       │
│   DeepEval)      │
│  "Is this        │
│   correct?"      │
└────────┬─────────┘
         │ score ≥ threshold?
         ▼
      Test Pass / Fail

Can they be the same model?

Technically yes, but not recommended.

Why?

• Bias: the model may over-approve its own answers
• Less reliable evaluation
• Reduced signal in regression testing
• Best practice:

Use a stronger or more capable model as the evaluator than the one being tested.

Can I choose the evaluator LLM?

Yes 👍 You can configure DeepEval to use:

• A specific OpenAI model
• Azure OpenAI
• Other supported providers

Example (conceptual):

GEval(
    name="Correctness",
    model="gpt-4o",
    ...
)

Why this approach works

Using an LLM to judge another LLM allows you to:

• Evaluate meaning, not keywords
• Handle paraphrasing naturally
• Scale testing across thousands of cases
• Test subjective qualities like correctness, tone, safety, and helpfulness

Final Thoughts

This small test demonstrates a big idea: LLMs should be evaluated with the same intelligence we expect from them.

By combining structured test cases with LLM-powered evaluation, DeepEval gives you a scalable and realistic way to validate AI behavior

If you’re building with LLMs, this kind of testing isn’t optional anymore — it’s essential.