#llm-as-a-judge

AI

JA Janagaraj Sangamanarayanan

How I Added an Eval Layer to My Google ADK Agent

A continuation of I’ve Been Building AI Agents with LangGraph. Then I Tried Google ADK. Here’s What Surprised Me.

Jul 20, 2026 · 6 min read · 0 claps #ai-agent #ai-agent-evaluation #llm-as-a-judge

AI ART

GL Glincy Mary Jacob · GoPenAI

Architectural Playbook for Low-Cost, Scalable Agent Evaluation

Build a cost-optimized AI agent evaluation framework. Learn how to slash input token costs by 90% using semantic prompt caching, tiered…

Jul 17, 2026 · 4 min read · 58 claps #ai-agent-evaluation #llm-as-a-judge #ai-agent-optimization

AI TCH

RO Robin · AgentX.so

How to Evaluate AI Agents: Runtime, CI/CD, and Beyond

Evaluating AI agents goes far beyond checking if they give the right answers. It emphasizes that the reasoning path, how the agent…

Jun 26, 2026 · 17 min read · 0 claps #ai-agent-evaluation #agent-cicd #llm-as-a-judge

HUM AI

CH Chuan Zhang

How Do We Know a Translation Is Good? A Four-Era Tour of MT Metrics

From BLEU’s bag of n-grams to LLMs that read like a human reviewer — two decades of trying to put a number on translation quality.

Jun 22, 2026 · 9 min read · 0 claps #machine-translation #bleu #wmt

AI

NE Nero Un Chi Hin 阮智軒

Prompt Engineering & Prompt Optimization 之 Query Transformations

從 2025 年到 2026 年一直有人說 RAG 已死，但我從來不這麼覺得；於是我又打開了塵封一年多的草稿、在 2026 年中再度把 Prompt Engineering 的方法進行實驗與記錄

Jun 14, 2026 · 38 min read · 0 claps #rags #prompt-engineering #prompt-optimization

AI

AL Alan Hsu

Using LLM-as-a-Judge for AI Systems Evaluation

The Hardest Part of Building AI Systems Isn’t Generation. It’s Evaluation!

Jun 09, 2026 · 3 min read · 1 claps #llm-as-a-judge #ai-workflow #ai-evaluation-metrics

AI SCI

MA mayaandersson-writes

Calibration set size for LLM-as-judge: when 50 traces is enough and when 200 is mandatory

A practitioner’s guide to sizing the human-labeled set you use to validate an automated judge, with the kappa variance math, Wilson…

Jun 08, 2026 · 8 min read · 0 claps #llm #llm-as-a-judge #ai

AI HUM MDA TCH

CI Cisco ThousandEyes Engineering · Cisco® ThousandEyes® Engineering

(LLM-)Judges on Autopilot

By Antonino Maria Rizzo, AI Researcher; Elisabetta Ghisu, Machine Learning Engineer; Francesco Cavrini, Software Engineering Technical…

May 27, 2026 · 11 min read · 1 claps #auto-calibration #prompt-optimization #llm-as-a-judge

DSN AI SOC TCH

YI Ying Lee

De-RAG: Refactoring an RAG Project into a Faster, Leaner Multimodal System

Abstract

May 23, 2026 · 12 min read · 0 claps #rags #multimodal #gemini

AI

AM AmeerSaleem

How G-EVAL improvements vanilla LLM-as-a-judge

An overview of Chain of Thought prompting and the G-EVAL framework, with insights from their debut papers.

May 17, 2026 · 9 min read · 0 claps #g-eval #chain-of-thought #llm-as-a-judge

AI LIF

RA Radu Pana

Eval-guided iterative prompt optimization for AI-powered workout parsing

I’ve been building Featherweight, a strength training tracking app, for the past year. One of its main selling points is AI-powered…

Mar 31, 2026 · 13 min read · 6 claps #llm-evaluation #llm-as-a-judge #ground-truth

AI

KA Karan Sharma

Evaluating LLM Outputs with DeepEval and LangSmith

In continuation to post , where i just used expected results with the actual LLM output just to evaluate how the similarilty between…

Jan 02, 2026 · 6 min read · 2 claps paywalled #deepeval #llm-as-a-judge #llmeval

AI

KA Karan Sharma

Evaluating LLM Health Responses with DeepEval: A Practical Example

As Large Language Models (LLMs) are increasingly used in sensitive domains like healthcare, evaluating the quality and correctness of their…

Jan 02, 2026 · 5 min read · 0 claps paywalled #llmeval #deepeval #llm-as-a-judge

AI

CA Cahit Barkin Ozer · Softtech

Utilising LLM-as-a-Judge to Evaluate LLM-Generated Code

How to use LLM-as-a-Judge when benchmarking code LLMs?

Dec 28, 2025 · 22 min read · 7 claps #llm-as-a-judge #llm-judge #judge-llm-code-generation

AI SPR

OG Ogulcanakca

Uçtan Uca Alana Özgü Kumru Base Modeli Eğitimi: Veri Hazırlığından Üretim Ortamına Optimizasyon…

Bu çalışmada, Enerji Piyasası Düzenleme Kurumu (EPDK) mevzuatına hakim, domain-specific bir dil modeli geliştirilmesi süreçleri uçtan uca…

Nov 24, 2025 · 14 min read · 0 claps #vllm #e2e #llmops

HUM AI

YO YouShin kim

여러 심판관으로부터 효율적인 다중 턴 대화 평가자 학습하기 (Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple…

배경 및 개요

Aug 10, 2025 · 8 min read · 0 claps #llm-as-a-judge #reward-model #multiturndialogue

AI ART

AB Ab Hassanein

Measuring What Matters: Crafting Evaluation Metrics for GenAI Solutions

Why?

May 17, 2025 · 7 min read · 2 claps #llm-evaluation #generative-ai-solution #g-eval

HUM AI TCH

PE Pere Martra · Towards AI

Evaluating a RAG Solution with Giskard

Giskard is an open-source project dedicated to providing a framework for the evaluation of different machine-learning models. Here, you…

Oct 07, 2024 · 8 min read · 76 claps #large-language-models #giskard #vector-database