Fig. 3

Example of a clinical case question associated with a radiological image. Graph A displays the students’ responses, while graph B represents the distribution of students according to the probability of answering the question correctly (y-axis) based on their ability level in the exam (x-axis). “Ability” refers to the theoretical estimation of the student’s knowledge in the exam. Graph C represents the point at which this question best discriminates among the knowledge levels of the entire sample (x-axis). Both graphs belong to the Item Response Theory (IRT) using the Two-Parameter Logistic (2-PL) probability model. Incorrect questions are technically challenging to construct. In the displayed question, to obtain an incorrect response as the first option, one must rely on a subtle and minor nuance related to a short time frame (not two months, but three months). The remaining correct answers with subjective nuances not only hinder question discrimination but also, as observed in the 2PL model, lead to a situation where students who know more perform worse than those who know less (graph B). This not only introduces noise in the sample but also significantly decreases the overall discrimination of the entire exam