Why an LLM Classifier Passes Eval and Fails in Production

A balanced test set hides the imbalance production has. Score per-class on the real distribution, report macro and weighted F1 together…