#llm-testing

How Chatbot Testing Ensures Better AI Conversations and Customer Experiences

Artificial intelligence is steadily becoming the primary interface between businesses and customers. From virtual assistants and enterprise…

📘 Part 3 — How to Test What You Can’t Predict

Designing Evaluation for Non-Deterministic AI Systems

📘 Part 2 — Where AI Systems Fail in Production (Not in Prompts)

AI Systems Thinking for QA Engineers

Promptfoo in Practice: Deterministic Checks and Model-Graded Evaluations — Part II

This tutorial focuses on two important testing styles in Promptfoo: deterministic checks and model-graded checks. These two categories…

📘 Part 1 — Why LLM Testing Is Actually System Testing

Large Language Models (LLMs) often appear, at first glance, to be standalone components. You send a prompt, you get a response. Naturally…

Agentic AI Services vs Traditional AI Automation: Key Differences Explained

The shift from traditional, rule-bound algorithmic automation to dynamic, self-directed systems marks a profound evolution in engineering…

📘 Becoming an AI Tester — Part 3: Skills Every AI Tester Should Start Learning

A practical breakdown of the technical and analytical skills needed in AI testing

3 Agents. 12 Days.

847 XPath selectors. 94 test files. 4 years of accumulated fragility. Here’s how we cleared it in 12 days without introducing a single…

Safety and Validation Engineering will become the bridge between AI Engineering and AI Governance.

The traditional QA department is entering a major transition. In the GenAI era, QA is no longer the function that verifies whether software…

3 Pipelines. 14 Days. 0 → 500 AI Test Assertions: Building LLM QA Automation from Scratch

Two weeks ago our LLM feature had zero automated test assertions — not low coverage, zero. Three layered pipelines built over 14 days…

4 Metrics. 1 Week. Blind Testing → Full RAG Validation: PromptFoo Setup for SDETs

Our AI feature was being validated by manual testers with no objective metrics, no regression baseline, and no way to detect subtle quality…

[Google] — Auto-Diagnose: LLM 기반의 통합 테스트 실패 자동 진단 아키텍처 분석

1. Executive Summary & Background

LLM-Intruder

An adaptive LLM security assessment framework for authorized red teams.

AI Testing Services vs Traditional QA: What Enterprises Must Know

There is a growing realization among CXOs and QA leaders that traditional automation is no longer sufficient to support the demands of…

How Do You Test Something That Thinks? A QA Guide to AI Testing Methodologies

What Happens When Agents Hit Real-World Edge Cases

Most AI agents are designed for the common path — the 80% of interactions that follow predictable patterns, where inputs are clear and…

LLM benchmarks, evals and tests

A mental model

Responsible AI in Practice: Safety, Toxicity, and PII Testing for LLMs

Testing the Untestable: 7 Uncommon Challenges in AI/LLM Quality Assurance

Welcome to the edge of software testing, where established QA practices collide with the fickle nature of artificial intelligence.