From: Inter-rater reliability in clinical assessments: do examiner pairings influence candidate ratings?
Examiners | Pair A | Pair B | Pair C | Pair D | Pair E | Pair F | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 5 | 3 | 11 | 7 | 12 | 6 | 14 | 9 | 16 | 10 | 17 | |
Good | 0.0 | 0.0 | 4 | −4 | −4 | −6 | 18 | −12 | 8 | −2 | 0 | −6 |
Average | 4.0 | 10 | −4 | −2 | 10 | −4 | 12 | −2 | 4 | 0 | 10 | −6 |
Weak | −8.0 | 6.0 | −12 | −2 | 0 | −6 | −6 | 6 | 0 | −2 | 2 | −4 |
Total change | 12 | 16 | 20 | 8 | 14 | 16 | 36 | 20 | 12 | 4 | 12 | 16 |