Fig. 2

Example of a clinical case question associated with a radiological image. Graph A displays the students’ responses, while graph B represents the distribution of students according to the probability of answering the question correctly (y-axis) based on their ability level in the exam (x-axis). “Ability” refers to the theoretical estimation of the student’s knowledge in the exam. Graph C represents the point at which this question best discriminates among the knowledge levels of the entire sample (x-axis). Both graphs belong to the Item Response Theory (IRT) using the Two-Parameter Logistic (2-PL) probability model. As a typical example of a well-discriminating multiple-choice - single select question, we can observe how it presents a short and direct statement. The question addresses a clear and precise medical concept. The answer choices consist of few characters and are devoid of subjectivity. In this way, these questions follow a “know or don’t know” format, which makes them highly effective in achieving proper discrimination