Skip to main content
Download PDF

Adding to the debate on the numbers of options for MCQs: the case for not being limited to MCQs with three, four or five options

BMC Medical Education volume 19, Article number: 354 (2019) Cite this article

Abstract

Background

There is a significant body of literature that indicates that the number of options for single-best answer multiple choice questions (MCQs) can be reduced from five to three or four without adversely affecting the quality of the questions and tests. Three or four options equates to two or three distractors respectively.

Maintext

Whilst these arguments may be true when focusing on psychometric aspects of questions, we should also focus on educational and clinical authenticity aspects of questions. I present reasons for MCQs in tests to have a variable number of options which will usually be more than three, four, or five. These include: decisions related to broad clinical scenarios cannot be limited to a small number of options; options lists should include all possible combinations of option elements; and options that are rarely chosen can provide information regarding students and/or for students.

Conclusion

Finally, given computer based delivery, longer option lists are not impractical for examinees. In the contexts that are appropriate, it is time to consider a move to adopting appropriate and variable numbers of MCQ options and not be limited to MCQs with three, four or five options.

Peer Review reports

Background

Multiple-choice questions (MCQs) are widely used in assessment within medical education and there are numerous articles comparing the number of response options and distractors [1]. Building further on this, it continues to be postulated that reducing the number of options does not lead to a reduction in assessment parameters [2, 3]. This creates a prevalent opinion as articulated in more recent reviews [4, 5] and primary research reporting that reducing the number of options does not result in significant differences in assessment parameters [6] or can lead to improvement in the parameters [7]. This body of literature suggests there are potential advantages and no disadvantages to reducing the number of MCQ options.

With this weight of evidence, why would assessment organisers not consider reducing to three or four options? In fact, to the contrary, I propose that assessment organisers consider having a variable number of options, which may mean increasing the number of options for many questions. The basis of this argument is that the evidence to reduce the number of options is based on a psychometric perspective, whereas the argument to have a variable number of options, which can include an increased number of options for many questions, is based on clinical authenticity and educational perspectives. In order to add to the debate, I proceed by way of presenting some reasons based on these perspectives.

Main text

Decisions related to broad clinical scenarios cannot be limited to a small number of options

Rarely do the questions faced by a clinicians in practice have exactly three, four or five options [8]. Although primarily developed because of concerns that a limited number of MCQ options would cue candidate to a correct response [8], longer lists of options are also perceived as being more authentic to clinical practice [9]. A single long list of options, hundreds of options long, could be used for all MCQs in an assessment [8, 10]. Equally, there is no reason the number of options has to be same for all questions in a test [11].

Extended matching type questions (EMQ) were developed such that without cueing from a short list of response options, clinical reasoning and knowledge may be assessed [12, 13]. An option list, of 5 to more than 25, is used for all questions, for a particular theme eg “What is the most likely diagnosis for a person presenting with chest pain?” [12]. An advantage of EMQs is that changes to the stem (patient scenario) can lead to a change in the correct answer from a longer list of options, thus reflecting clinical practice [14].

The number of options should not be defined by the format but the content of the question [15]. The number of options for a question should align with authentic clinical practice. There is not always the same number of options in clinical practice, so the number of response options should vary, and is likely to be more than three or four.

Options lists should include all possible combinations of option elements

Some options for an MCQ might be made up of several descriptive elements. Rather than try to select which combinations of elements should be included or not, an alternative is to increase the number of options to ensure all combinations are included.

As an example, this is a question with eight possible combinations of elements:

“A person with breathlessness has the following blood gas analysis … ..”

Which option best describes the blood gas analysis?

  1. A.

    Metabolic acidosis with a normal Aa (Alveolar-arterial) gradient

  2. B.

    Metabolic acidosis with an increased Aa gradient

  3. C.

    Metabolic alkalosis with a normal Aa gradient

  4. D.

    Metabolic alkalosis with an increased Aa gradient

  5. E.

    Respiratory acidosis with a normal Aa gradient

  6. F.

    Respiratory acidosis with an increased Aa gradient

  7. G.

    Respiratory alkalosis with a normal Aa gradient

  8. H.

    Respiratory alkalosis with an increased Aa gradient

Rather than trying to select which three, four or five options should be included or not, it is possible to have all eight. As will be discussed subsequently, clinically important incorrect answers and psychometrically important incorrect answers might be different.

Where there are two elements with two possibilities, then there are four possible options to include [3]. This will also remove the futile hunt for a fifth option, when four options provides all plausible combinations of elements.

The number of options should include all combinations of elements, rather than limiting these to a set number of options for every question. The number of options will vary with the number of elements and therefore combinations, and is best supported by a policy of variable option numbers.

Options that are rarely chosen can provide information regarding students and/or for students

Do we run the risk of losing important information if we remove rarely chosen options from MCQs? Many of the analyses upon which the recommendations to reduce the number of options are based on the assumption that incorrect responses do not have distinct intrinsic information. This is erroneous, there is significant information in incorrect responses, as there are responses that would be potentially unsafe if chosen in practice [16,17,18,19,20,21]. Panels of clinicians can consider the potential clinical impact of incorrect responses, which can lead to incorrect options being stratified for potential (un) safeness [16,17,18,19,20,21]. The most potentially unsafe responses are rarely selected [18, 20]. Rarely selected distractors are unlikely to be considered psychometrically important. Clinically important distractors are different from psychometrically important distractors [22]. Options that are rarely chosen can represent unsafe practices; it is vital to know which students are selecting these potentially unsafe responses [16,17,18,19,20,21]. Individual misconceptions can be included in feedback with the goal to direct personal learning development [16,17,18,19,20,21]. If they become apparent, cohort level misconceptions can be used with the goal to direct curriculum development. By removing rarely chosen but clinically important incorrect options representing potentially unsafe practices, we deny the opportunity for misinformed examinees to choose such options. The choice of unsafe options across multiple questions would be a concerning pattern that needs to be recognised to target learning and subsequent performance; should the pattern be repeated despite further learning opportunities, this information could be used to inform progression decisions [18, 20]. One postulated reason why examinees might continue to select unsafe options is the paucity of feedback they receive on answers that are unsafe as well as incorrect [18, 20].

The number of options in MCQs should be sufficient to include both psychometrically important distractors and clinically important distractors. As the number of each will not be the same for all content areas, their inclusion is likely to require more than three or four options, and is best supported by a policy of variable option numbers.

Computer based delivery has made longer lists of options more feasible

Assessments do need to be practical and feasible [23]. Longer lists of response options might be difficult to fit on assessment documentation or for candidates to use. As already noted, MCQs with longer lists of options do not lead to impaired performance by examinees [8, 10]. A single long list of options, hundreds of options long, could be used, and such formats have proved feasible when facilitated by computer delivery [10], though it has also been implemented in a paper-based system [8].

As long as the question meets the cover test (the correct answer can be determined without seeing the options [24]), and the options are presented in a consistent logical order (e.g. alphabetical), then long lists are not a problem. Questions not meeting these standards are most likely to be flawed irrespective of the number of options.

With computer delivery of MCQ assessments, there is no space constraint on option lists, and each option is automatically set a corresponding response tick box. Computer marking mitigates errors in reading and grading responses.

Conclusion

Now that many institutions are moving to computer delivery and marking of MCQ examinations, it is time to consider the move to adopting appropriate and variable numbers of MCQ options and not be artificially limited to MCQs with three, four or five options.

Availability of data and materials

Not applicable.

Abbreviations

Aa:

Alveolar-arterial

EMQ:

Extended matching type questions

MCQ:

Multiple choice question

References

  1. Rodriguez MC. Three options are optimal for multiple-choice items: a meta-analysis of 80 years of research. Educ Meas Issues Pract. 2005;24(2):3–13.

    Article  Google Scholar 

  2. Fozzard N, Pearson A, du Toit E, Naug H, Wen W, Peak IR. Analysis of MCQ and distractor use in a large first year health Faculty Foundation program: assessing the effects of changing from five to four options. BMC Medical Education. 2018;18(1):252.

    Article  Google Scholar 

  3. Raymond MR, Stevens C, Bucak SD. The optimal number of options for multiple-choice questions on high-stakes tests: application of a revised index for detecting nonfunctional distractors. Adv Health Sci Educ. 2019;24(1):141–50.

    Article  Google Scholar 

  4. Wilson I. What's best for multiple-choice questions: three, four or five? Clin Teach. 2014;11(7):568–70.

    Article  Google Scholar 

  5. Gierl MJ, Bulut O, Guo Q, Zhang X. Developing, analyzing, and using distractors for multiple-choice tests in education: a comprehensive review. Rev Educ Res. 2017;87(6):1082–116.

    Article  Google Scholar 

  6. Royal KD, Stockdale MR. The impact of 3-option responses to multiple-choice questions on guessing strategies and cut score determinations. Journal of Advances in Medical Education & Professionalism. 2017;5(2):84–9.

    Google Scholar 

  7. Kilgour JM, Tayyaba S. An investigation into the optimal number of distractors in single-best answer exams. Adv Health Sci Educ. 2016;21(3):571–85.

    Article  Google Scholar 

  8. Veloski JJ, Rabinowitz HK, Robeson MR, Young PR. Patients don't present with five choices: an alternative to multiple-choice tests in assessing physicians' competence. Acad Med. 1999;74(5):539–46.

    Article  Google Scholar 

  9. Huwendiek S, Reichert F, Duncker C, de Leng BA, van der Vleuten CP, Muijtjens AM, Bosse H-M, Haag M, Hoffmann GF, Tönshoff B. Electronic assessment of clinical reasoning in clerkships: a mixed-methods comparison of long-menu key-feature problems with context-rich single best answer questions. Med Teach. 2017;39(5):476–85.

    Article  Google Scholar 

  10. Schuwirth L, Cvd V, Stoffers H, Peperkamp A. Computerized long-menu questions as an alternative to open-ended questions in computerized assessment. Med Educ. 1996;30(1):50–5.

    Article  Google Scholar 

  11. Rogausch A, Hofer R, Krebs R. Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey. BMC Medical Education. 2010;10(1):85.

    Article  Google Scholar 

  12. Case SM, Swanson DB. Extended-matching items: a practical alternative to free response questions. Teaching and Learning in Medicine: An International Journal. 1993;5(2):107–15.

    Article  Google Scholar 

  13. Beullens J, Struyf E, Van Damme B. Do extended matching multiple-choice questions measure clinical reasoning? Med Educ. 2005;39(4):410–7.

    Article  Google Scholar 

  14. Samuels A. Extended matching questions and the Royal Australian and new Zealand College of Psychiatrists written examination: an overview. Australasian Psychiatry. 2006;14(1):63–6.

    Article  Google Scholar 

  15. Coderre SP, Harasym P, Mandin H, Fick G. The impact of two multiple-choice question formats on the problem-solving strategies used by novices and experts. BMC Medical Education. 2004;4(1):23.

    Article  Google Scholar 

  16. Tweed M, Wilkinson T. A randomized controlled trial comparing instructions regarding unsafe response options in a MCQ examination. Med Teach. 2009;31(1):51–4.

    Article  Google Scholar 

  17. Tweed MJ, Thompson-Fawcett M, Schwartz P, Wilkinson TJ. A confidence and safety approach to MCQ scoring. Focus on Health Professional Education: A Multi-disciplinary Journal. 2012;13(3):84–92.

    Google Scholar 

  18. Tweed M, Schwartz P, Thompson-Fawcett M, Wilkinson TJ. Determining measures of insight and foresight from responses to multiple choice questions. Med Teach. 2013;35(2):127–33.

    Article  Google Scholar 

  19. Curtis DA, Lind SL, Boscardin CK, Dellinges M. Does student confidence on multiple-choice question assessments provide useful information? Med Educ. 2013;47(6):578–84.

    Article  Google Scholar 

  20. Tweed M, Stein S, Wilkinson T, Purdie G, Smith J. Certainty and safe consequence responses provide additional information from multiple choice question assessments. BMC Medical Education. 2017;17(1):106.

    Article  Google Scholar 

  21. Rangel RH, Möller L, Sitter H, Stibane T, Strzelczyk A. Sure, or unsure? Measuring students’ confidence and the potential impact on patient safety in multiple-choice questions. Med Teach. 2017:1–6.

  22. Swanson DB, Holtzman KZ, Allbee K. Measurement characteristics of content-parallel single-best-answer and extended-matching questions in relation to number and source of options. Acad Med. 2008;83(10):S21.

    Article  Google Scholar 

  23. Crossley J, Humphris G, Jolly B. Assessing health professionals. Med Educ. 2002;36(9):800–4.

    Article  Google Scholar 

  24. Ware J, Vik T. Quality assurance of item writing: during the introduction of multiple choice questions in medicine for high stakes examinations. Med Teach. 2009;31(3):238–43.

    Article  Google Scholar 

Download references

Acknowledgements

The author acknowledges constructive comments on earlier drafts of this manuscript from the editor and reviewers (BMC Medical Education), and Tim Wilkinson (University of Otago) and Fiona Hyland (University of Otago).

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Medicine, University of Otago Wellington, Wellington, New Zealand

    Mike Tweed

Authors
  1. Mike Tweed
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The author developed, wrote all drafts this debate piece. The author read and approved the final manuscript.

Corresponding author

Correspondence to Mike Tweed.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The author declares that he has no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tweed, M. Adding to the debate on the numbers of options for MCQs: the case for not being limited to MCQs with three, four or five options. BMC Med Educ 19, 354 (2019). https://doi.org/10.1186/s12909-019-1801-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12909-019-1801-x

Keyword

BMC Medical Education

ISSN: 1472-6920

Contact us