Skip to main content

Table 1 Summary of the differences in the methodologies used by Australian medical schools when undertaking criterion-referenced standard setting

From: Standard setting in Australian medical schools

Modifications to standard setting Modified Angoff Mixed method Modified Ebel Nedelsky Total
Conduct of the standard setting Conducted with all panel members in person 4 1 1 1 7
Conducted electronically 2 1 0 0 3
Calibration session held 3 0 0 0 3
No Calibration 3 2 1 1 7
Answers to question provided to panel members 4 2 0 1 7
No answers provided 2 0 1 0 3
Performance data shown to panel member 2 0 0 0 2
No performance data provided 4 2 1 1 8
Ratings discussed and changes allowed 2 2 1 1 6
Consensus decision 2 0 0 0 2
No discussion or change allowed 2 0 0 0 2
Analysis of standard setting data Standard set the whole exam 5 2 1 1 9
Standard set a random selection 1 0 0 0 1
Used the Mean across all panel members 3 2 1 0 6
Used the Median across all panel members 1 0 0 1 2
N/A Consensus used 2 0 0 0 2
Does not add a measurement error to the cut score 5 2 1 0 8
Measurement error added to the cut score 1 0 0 1 2
Standard setting mark used as the cut score 5 2 1 1 9
Standard setting mark used as a guide only 1 0 0 0 1
TOTAL SCHOOLS 6 2 1 1 10