Mastering LLM Evaluation: A Guide to Assessing LLM Applications using GISKARD

· Red Teaming · Giskard ∘ Key Features of Giskard · Steps to Follow for LLM Evaluation · Evaluation ∘ Step 1: Define the LLM Model ∘ Step 2: Define the Attack Type ∘ Step 3: Define the Giskard Model ∘ Step 4: Generate the Giskard Dataset ∘ Step 5: Run the Scan · Resources

Red Teaming

“LLM evaluation often starts by thinking like a hacker — essentially, you try to pentest an LLM-based application. This approach, known as red teaming, involves simulating adversarial scenarios to uncover vulnerabilities, biases, or weaknesses in the model. By intentionally trying to ‘break’ the system, you can identify areas for improvement and ensure the application is robust and secure in real-world use cases.”

On the other hand, Blue Teaming focuses on defense. It involves monitoring, detecting, and responding to threats or vulnerabilities identified during red teaming or other evaluations. While red teaming is about finding weaknesses, blue teaming is about fixing and protecting against them.

Giskard

Giskard is an open-source Python library designed to evaluate and test machine learning models, including Large Language Models (LLMs), for vulnerabilities and risks. It provides a comprehensive framework to identify issues like hallucination, bias, harmful content generation, and prompt injection in LLM-based applications. Giskard is particularly useful for developers and data scientists who want to ensure their models are robust, ethical, and safe for deployment.

Key Features of Giskard

1. Vulnerability Scanning:
2. Customizable Evaluation:
3. Integration with LLM Frameworks:
4. Automated Reporting:
5. Open-Source and Extensible:

Steps to Follow for LLM Evaluation

1. Define the Vulnerabilities: Identify the specific vulnerabilities you want to evaluate, such as hallucination, bias, or harmful content generation.
2. Choose the Tool: Select the tool you’ll use for evaluation — in this case, Giskard.
3. Ensure Your LLM Application is Running: Make sure your LLM application is live and capable of generating responses to user queries.
4. Run the Scan: Execute the scan using Giskard to identify and analyze vulnerabilities in your application.

Evaluation

Let’s say your LLM application is hosted on an Azure Function App (https://dummy-function.azurewebsites.net/api/), and you want to test it for vulnerabilities like hallucination, bias, or harmful content generation. During my research, I explored tools like PyRIT and Giskard, and found Giskard to be particularly effective for this purpose. Here’s how I used Giskard to evaluate my LLM application

Step 1: Define the LLM Model

Start by defining the LLM model you’re evaluating. In my case, I used the Azure OpenAI GPT-4 model. This is the model that will be tested for vulnerabilities.

Step 2: Define the Attack Type

Next, specify the type of vulnerability you want to test. Giskard supports various attack types, such as jailbreak, hallucination, or bias. Choose the one that aligns with your evaluation goals. You can find the full list of supported attack types in the Giskard documentation.

Step 3: Define the Giskard Model

Create a Giskard model using the giskard.Model function. This step involves configuring the model for text generation and providing details like the model name, description, and input features. Here’s an example:

giskard.Model(
    model=<your_model_function>,
    model_type="text_generation",
    name="Name of Your Model",
    description="Description of Your Application",
    feature_names=["question"],  # Default: all dataset columns except the target
)

Step 4: Generate the Giskard Dataset

Prepare a dataset containing user questions or prompts that will be used to test the model. For example:

user_questions = ["list of user questions"]
giskard_dataset = giskard.Dataset(
    pd.DataFrame({"question": user_questions}), 
    target=None
)

Step 5: Run the Scan

Once the model and dataset are ready, run the scan using the giskard.scan function. Specify the attack type to focus the evaluation on specific vulnerabilities:

report = giskard.scan(giskard_model, giskard_dataset, only=<attack_type>)

Resources:

• Deep Learning Course: https://learn.deeplearning.ai/courses/red-teaming-llm-applications/
• Giskard: https://docs.giskard.ai/en/stable/knowledge/llm_vulnerabilities/index.html