Cohen’s Kappa

Cohen’s kappa is a statistic that measures agreement between two raters while correcting for the agreement that would occur by chance, yielding a value typically from 0 to 1.

A kappa near 1 indicates strong agreement beyond chance, while a value near 0 indicates agreement no better than random; it is a standard measure of inter-rater reliability.

Reporting kappa alongside an evaluation tells readers how dependable the grading was, which is part of making a benchmark defensible.