Fig. 4

Technical flaws related to irrelevant difficulties. Percentage of flawed items in MIR exams between 2009 and 2013. a: The item cannot be answered without looking at the options. b: The answer to an item is “hinged” to the answer of a related item. c: Negative-phrased item (“except” or “not” in the lead-in). d: Terms in the options are vague (e.g., “rarely,” “usually”). e: The stem or the options are too complex, with more than one concept included. f: Options are in an illogical order