How to set the bar in competency-based medical education: standard setting after an Objective Structured Clinical Examination (OSCE)

Table 1 Modified Angoff method

Station	Residents	Sports Surgeon (95 % CI)	Non Sports Surgeon (95%CI)	Fellow (95 % CI)	Significance
1	Senior	0.57 (0.4-0.74)	0.64 (0.48-0.8)	0.58 (0.49-0.67)	P < 0.001
	Junior	0.37 (0.23-0.51)	0.37 (0.28-0.47)	0.33 (0.26-0.39)
2	Senior	0.56 (0.37-0.75)	0.67 (0.44-0.9)	0.58 (0.47-0.69)	P < 0.001
	Junior	0.43 (0.26-0.6)	0.39 (0.25-0.52)	0.36 (0.26-0.46)
3	Senior	0.46 (0.31-0.61)	0.57 (0.36-0.78)	0.49 (0.35-0.64)	P < 0.001
	Junior	0.35 (0.21-0.49)	0.34 (0.21-0.47)	0.29 (0.22-0.37)
4	Senior	0.58 (0.27-0.76)	0.66 (0.42-0.89)	0.5 (0.37-0.63)	P < 0.001
	Junior	0.33 (0.18-0.49)	0.34 (0.2-0.49)	0.29 (0.24-0.35)
5	Senior	0.56 (0.41-0.71)	0.71 (0.51-0.91)	0.56 (0.41-0.72)	P < 0.001
	Junior	0.38 (0.21-0.54)	0.41 (0.21-0.61)	0.36 (0.23-0.5)
6	Senior	0.50 (0.35-0.65)	0.67 (0.49-0.84)	0.5 (0.31-0.69)	P < 0.001
	Junior	0.35 (0.21-0.5)	0.42 (0.27-0.57)	0.36 (0.24-0.48)
Total	Mean Senior	0.53 (0.42-0.63)	0.65 (0.5-0.8)	0.54 (0.42-0.65)	P < 0.001
Total	Mean Junior	0.37 (0.27-0.47)	0.38 (0.27-0.49)	0.33 (0.26-0.41)

Displayed are the expected checklist percentage correct by examiner type and by resident group (junior – PGY1-3, senior PGY 4&5). For all three groups of judges (six in each group), a significant difference was seen both overall and for each station between junior and senior residents (p < 0.001). No significant difference was seen between each group of judges. PGY – postgraduate year

ISSN: 1472-6920