A german-language competency-based multisource feedback instrument for residents: development and validity evidence

Table 6 G-study and D-study on inter-rater reliability

		Estimated variance components
		Inter-resident variance	Rater variance within resident	N	G-coefficient	SEM
Run 1	G-study	0.032 (0.077)	0.021 (0.045)	11.85 (11.12)	0.60 (0.63)	0.38 (0.46)
Run 2	G-study	0.093 (0.189)	0.024 (0.040)	12.03 (11.76)	0.80 (0.83)	0.28 (0.34)
	D-study	0.093 (0.189)	0.026 (0.042)	11	0.78 (0.82)	0.29 (0.34)
			0.028 (0.047)	10	0.77 (0.80)	0.30 (0.36)
			0.032 (0.052)	9	0.75 (0.79)	0.31 (0.37)
			0.035 (0.058)	8	0.72 (0.77)	0.32 (0.39)
			0.041 (0.066)	7	0.70 (0.74)	0.34 (0.41)
			0.047 (0.078)	6	0.66 (0.71)	0.36 (0.43)

Estimated variance components for the variance associated with residents and raters and the generalisability coefficient (G-coefficient) and standard error of measurement (SEM) for the first and second run (G-study). G-coefficient and SEM as a function of the number of rater ratings (N) for the second run (D-study). Results are presented for the total score and in parentheses for the overall item 16

ISSN: 1472-6920