Skip to main content

Table 1 Summary of the differences in the methodologies used by Australian medical schools when undertaking criterion-referenced standard setting

From: Standard setting in Australian medical schools

Modifications to standard setting

Modified Angoff

Mixed method

Modified Ebel

Nedelsky

Total

Conduct of the standard setting

Conducted with all panel members in person

4

1

1

1

7

Conducted electronically

2

1

0

0

3

Calibration session held

3

0

0

0

3

No Calibration

3

2

1

1

7

Answers to question provided to panel members

4

2

0

1

7

No answers provided

2

0

1

0

3

Performance data shown to panel member

2

0

0

0

2

No performance data provided

4

2

1

1

8

Ratings discussed and changes allowed

2

2

1

1

6

Consensus decision

2

0

0

0

2

No discussion or change allowed

2

0

0

0

2

Analysis of standard setting data

Standard set the whole exam

5

2

1

1

9

Standard set a random selection

1

0

0

0

1

Used the Mean across all panel members

3

2

1

0

6

Used the Median across all panel members

1

0

0

1

2

N/A Consensus used

2

0

0

0

2

Does not add a measurement error to the cut score

5

2

1

0

8

Measurement error added to the cut score

1

0

0

1

2

Standard setting mark used as the cut score

5

2

1

1

9

Standard setting mark used as a guide only

1

0

0

0

1

TOTAL SCHOOLS

6

2

1

1

10