You Do Not Need a Server for Agent Evals

Code-first, repo-first evals cover most of the needs. Article goes in details of coding agent first approach to evals.