Table 1 Data Extraction: Interventional Studies which reported an outcome
From: Barriers and facilitators to writing quality items for medical school assessments – a scoping review
Authors | Intervention Type | Setting | Population | Study Design | Outcome | Reference |
|---|---|---|---|---|---|---|
Abdulghani HM, Ahmad F, Irshad M, Khalil MS, Al-Shaikh GK, Syed S, Aldrees AA, Alrowais N, Haque S | Two day workshop | Saudi Arabia, 2014 | Single group, medical faculty | Pre-test, Post-test (Quasi experimental) | Statistically significant improvements in difficulty levels and Bloom’s cognitive levels of items post-intervention. | Abdulghani HM, Ahmad F, Irshad M, Khalil MS, Al-Shaikh GK, Syed S, et al. Faculty development programs improve the quality of Multiple Choice Questions items’ writing. Sci Rep. 2015;5:9556. |
Malau-Aduli BS, Zimitat C | Peer review | Australia, 2012 | Single group, medical faculty | Pre-test, Post-test | Statistically significant increases in functional distractors, item difficulty, discrimination index and point-biserial correlation. | Malau-Aduli BS, Zimitat C. Peer review improves the quality of MCQ examinations. Assess Eval High Educ. 2012;37(8):919–31. |
Naeem N, van der Vleuten C, Alfaris EA | Week long workshop | Pakistan, 2009 | Single group, medical and nursing faculty | Pre-test, Mid-test, Post-test (Quasi experimental) | Statistically significant increase in mean scores of items according to a quality checklist. | Naeem N, van der Vleuten C, Alfaris EA. Faculty development on item writing substantially improves item quality. Adv Health Sci Educ Theory Pract. 2012;17(3):369–76. |
Shea JA, Poniatowski PA, Day SC, Langdon LO, LaDuca A, Norcini JJ | Item modelling | USA, 1992 | Two novice item writers, single group | Post-test | Item modelling is an efficient method for producing good quality test items and is appealing to potential item writers. | Shea JA, Poniatowski PA, Day SC, Langdon LO, LaDuca A, Norcini JJ. An adaptation of item modeling for developing test-item banks. Teach Learn Med. 1992;4(1):19–24. |
Wallach PM, Crespo LM, Holtzman KZ, Galbraith RM, Swanson DB | Committee review | USA, 2002 | Single group, medical faculty | Pre-test, Post-test | Statistically significant continuous improvement in Quality Assessment Scores (QAS) with the implementation of predetermined guidelines and a committee review process. | Wallach PM, Crespo LM, Holtzman KZ, Galbraith RM, Swanson DB. Use of a committee review process to improve the quality of course examinations. Adv Health Sci Educ Theory Pract. 2006;11(1):61–8. |
Iramaneerat C | Three short workshops | Thailand, 2012 | Single group, clinical faculty | Pre-test, post-test | Statistically significant increase in post-test item discrimination. Workshops had high satisfaction ratings amongst clinical teachers. | Iramaneerat C. The impact of item writer training on item statistics of multiple-choice items for medical student examination. Siriraj Med J. 2012;64(6):178–182. |
Abdulghani HM, Irshad M, Haque S, Ahmad T, Sattar K, Khalil MS | Longitudinal item writing workshops | Saudi Arabia, 2015 | Single group, medical faculty | Longitudinal cohort study | Improvement in item quality as determined through item analysis in each successive academic year after the implementation of item writing workshops. | Abdulghani HM, Irshad M, Haque S, Ahmad T, Sattar K, Khalil MS. Effectiveness of longitudinal faculty development programs on MCQs items writing skills: A follow-up study. PLoS One. 2017;12(10):e0185895. |
Abozaid H, Park YS, Tekian A | Peer review committee | Saudi Arabia, 2013 | Single group, medical faculty | Retrospective cohort study | Statistically significant improvement in item discrimination in medicine, paediatric and surgery examinations. Significant improvement in item difficulty only for the medicine examination. | Abozaid H, Park YS, Tekian A. Peer review improves psychometric characteristics of multiple choice questions. Med Teach. 2017;39 Suppl 1:50–4. |