Table 1 Adapted questionnaire from the National Board of Medical Examiners (NBME®) guidelines
Issues related to testwiseness |
The number of correct option |
One or more distractors don’t follow grammatically from the stem |
One or more options are collectively exhaustive |
Terms such as “never” or “always” are used in options |
The correct answer is longer, more specific, or more complete than other options |
A word or phrase is included in the stem and in the correct answer |
The correct answer includes the most elements in common with the other options |
There is lack of uniformity in the options |
Some of the distractors are not plausible |
Issues related to irrelevant difficulties |
The item cannot be answered without looking at the options |
The answer to an item is “hinged” to the answer of a related item |
Negative-phrased item (“except” or “not” in the lead-in) |
Terms in the options are vague (e.g., “rarely,” “usually”) |
The stem or the options are tricky or unnecessarily complicated |
The stem or the options include unnecessary information |
The stem or the options are too complex, with more than one concept included |
Options are in an illogical order |
“None of the above” or “All of the above” are used as an option |
Others |
Orthographic or syntax errors and use of outdated terms |