Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey

Table 2 Parameter changes related to different numbers of distractors (statistical approach)

Specialty	Model	% correct (P-value)*	Discrimination*	Reliability*	Standard Reliability**
Internal Medicine, Pharmacology (55 items)	4 distractors	78.93	0.19	0.74	0.84
	A) 3 distractors, random	79.20	0.19	0.73	0.83
	B) 3 distractors, right answer	80.11	0.18	0.72	0.82
	C) 2 distractors, random	80.20	0.18	0.71	0.82
	D) 2 distractors, right answer	82.76	0.14	0.62	0.75
Surgery (54 items)	4 distractors	77.74	0.13	0.59	0.73
	A) 3 distractors, random	77.91	0.13	0.59	0.73
	B) 3 distractors, right answer	78.44	0.13	0.58	0.72
	C) 2 distractors, random	78.78	0.13	0.58	0.72
	D) 2 distractors, right answer	80.75	0.11	0.54	0.68
Social and preventive medicine (41 items)	4 distractors	76.32	0.11	0.50	0.71
	A) 3 distractors, random	76.70	0.11	0.50	0.70
	B) 3 distractors, right answer	77.62	0.10	0.46	0.68
	C) 2 distractors, random	77.61	0.10	0.45	0.67
	D) 2 distractors, right answer	80.20	0.07	0.36	0.58
Pediatrics and gynaecology (43 items)	4 distractors	75.73	0.14	0.57	0.76
	A) 3 distractors, random	76.05	0.13	0.56	0.74
	B) 3 distractors, right answer	76.99	0.12	0.52	0.72
	C) 2 distractors, random	76.97	0.12	0.54	0.73
	D) 2 distractors, right answer	79.38	0.10	0.47	0.67
Dermatology, ophthalmology, otorhinolaryngology (57 items)	4 distractors	79.69	0.22	0.77	0.86
	A) 3 distractors, random	79.92	0.21	0.76	0.85
	B) 3 distractors, right answer	80.64	0.20	0.75	0.84
	C) 2 distractors, random	80.79	0.20	0.74	0.83
	D) 2 distractors, right answer	82.91	0.16	0.68	0.78

* These indicators related to the sample of positive A-type questions only, thus they differ from the respective indicators relating to the complete examination including negative best answer and extended matching questions.
** Reliability standardized for 100 items