LLM-as-Judge

LLM-as-judge is an evaluation method that uses a language model to grade another model’s output against a rubric, scaling assessment of open-ended work that has no single correct answer.

It is useful where exact-match grading fails — summaries, drafts, conversations — but the judge model can be biased or inconsistent, so its reliability must itself be checked.

Rigorous benchmarks validate an LLM judge against human ratings, measuring agreement before trusting its scores at scale.