Table 3 Characteristics of included articles ordered by comparator and study design

Comparator: teaching as usual

Aghili et al. 2012

To evaluate whether virtual patient simulations improve clinical reasoning skills of medical students

RCT

6th years. N = 52 (29 IG, 23 CG)

Data gathering, ideas about patient management

Diagnostic test (using patient cases)

⇧ Intervention produced significantly greater improvement in data gathering and ideas about patient management compared to teaching as usual (d = 1.55)

Moderate (6)

Botezatu et al. 2010

To explore possible superior retention results with Virtual Patients versus regular learning activities, by measuring the differences between early and delayed assessment results

RCT

4^th & 6^th years. N = 49 (25 IG, 24 CG)

Data gathering, ideas about diagnoses, ideas about patient management

Virtual patient cases

⇧ Intervention produced significantly greater improvement in data gathering, ideas about diagnoses and patient management compared to teaching as usual (average effect size across 5 dimensions, d = 1.57)

Moderate (6)

Kahl et al. 2010

To explore whether the addition of systematic training in iterative hypothesis testing may add to the quality of the psychiatry course taught to fifth year medical students

RCT

5^th years. N = 72 (36 IG, 36 CG)

Ideas about diagnoses

Standardised patient (actor)

⇧ Intervention produced significantly greater improvements in ideas about diagnoses compared to teaching as usual (d = 1.17)

Moderate (7)

Kalet et al. 2007

To assess the impact of individual WISE-MD modules on clinical reasoning skills

RCT

Clinical years. N = 96 (52 IG, 44 CG)

Data gathering, ideas about patient management

Script concordance test

⇧ Intervention produced significantly greater improvement in data gathering and ideas about patient management compared to teaching as usual (d = 0.25)

Moderate (9)

Lehmann et al. 2015

Investigated the effect of Virtual Patients combined with standard simulation-based training on the acquisition of clinical decision-making skills and procedural knowledge, objective skill performance, and self-assessment

RCT

3rd & 4th years. N = 57 (30 IG, 27 CG)

Ideas about diagnoses, ideas about patient management, application of knowledge

Key feature problems

⇧ Intervention produced significantly greater improvement in ideas about diagnoses and patient management, and application of knowledge compared to teaching as usual (d = 1.91)

High (13)

Qin et al. 2022

To develop a competency-based model of practice-based learning for undergraduate radiology education

RCT

3rd years. N = 114 (57 IG, 57 CG)

Application of knowledge

Multiple-choice question examination

⇧ Intervention produced significantly greater improvement in the application of knowledge compared to teaching as usual (d = 0.63)

Moderate (5)

Plackett et al. 2020

To assess the feasibility, acceptability and potential effects of eCREST — the electronic Clinical Reasoning Educational Simulation Tool

Feasibility RCT

5th & 6th years. N = 264 (137 IG, 127 CG)

Data gathering, flexibility in thinking about diagnoses (reported separately)^a

Virtual patient case & Diagnostic Thinking Inventory (DTI)

⇧ Ability to gather essential information (data gathering; d = 0.19) significantly improved after intervention compared to teaching as usual

High (11)

⬄ There was no significant difference between groups in relevance of history taking (data gathering; d = -0.13) and flexibility in diagnoses (d = 0.20)

Kim et al. 2018

To explore how students use and benefit from virtual patient cases

Non-randomised trial

3^rd years. N = 255 (129 IG, 126 CG)

Ideas about diagnoses

Clinical rating at end of clerkship by faculty

⬄ Ideas about diagnoses did not significantly improve compared to teaching as usual (voluntary access to cases) (d = 0.09)

Moderate (8)

Raupach et al. 2021

To investigate the effectiveness of a digital simulation of an emergency ward regarding appropriate clinical decision-making

Non-randomised trial

4th years. N = 100 (58 IG, 42 CG)

Data gathering, ideas about diagnoses, ideas about patient management (reported separately)

Virtual patient cases

⇧ Intervention produced significantly greater improvement in diagnostic accuracy (ideas about diagnoses for 2/3 cases; d = 0.81) and patient management (d = 0.81), compared to teaching as usual

Moderate (5)

⬄ Intervention did not significantly improve data gathering, compared to teaching as usual (d = 0.03)

Comparator: tutorial covering the same case

Devitt & Palmer 1998

To evaluate the intervention by assessing whether it expanded students’ knowledge base, improving data-handling abilities and clinical problem-solving skills

RCT

5^th years. N = 71 (46 IG, 25 CG)

Problem-solving skills

Multi-step clinical problem (patient case)

⬄ Intervention produced non-significantly greater improvement in problem-solving skills compared to tutorial (d = 0.50)

Moderate (6)

Raupach et al. 2009

To explore whether students completing a web based collaborative teaching module show higher performance in a test aimed at clinical reasoning skills than students discussing the same clinical case in a traditional teaching session

RCT

4^th years. N = 143 (72 IG, 71 CG)

Data gathering, ideas about diagnoses, ideas about patient management

Key feature problems

⬄ Intervention did not significantly improve data gathering, ideas about diagnoses and patient management compared to tutorial (d = 0.03)

High (10)

Sobocan et al. 2017

To determine the educational effects of substituting p-PBL sessions with VP on undergraduate medical students in their internal medicine course

RCT

3^rd years. N = 34 (17 IG, 17 CG)

Application of knowledge and flexibility in thinking

DTI

⬄ Intervention did not significantly improve application of knowledge and flexibility in thinking compared to tutorial (d = 0.25)

Moderate (7)

Middeke et al. 2018

To compare a Serious Game, the virtual A&E department ‘EMERGE’ to small-group problem-based learning (PBL) regarding student learning outcome on clinical reasoning in the short term

Non-randomised trial

5^th years, N = 112 (78 IG, 34 CG)

Data gathering, ideas about diagnoses, ideas about patient management (reported separately)

Key feature problems & virtual patient cases

⇧ Intervention produced significantly better clinical reasoning skills compared to tutorial (d = 0.47) when measured on key features test and for some domains measured by the virtual patient cases – final diagnosis (ideas about diagnoses),

Moderate (6)

therapeutic interventions (ideas about patient management), physical examination, instrumental examination (data gathering)

⬄ There was no significant difference between groups in history taking (data gathering), laboratory orders and patient transfer (ideas about patient management)

Comparator: N/A

Chon et al. 2019

To test the effect of a serious game simulating an emergency department (“EMERGE”) on students’ declarative and procedural knowledge

Single group pre & post comparison

Clinical years. N = 140

Data gathering, ideas about diagnoses, ideas about patient management, (reported separately)

Patient case

⇧ Diagnostic questions (data gathering; d = 0.77), choosing the correct order of diagnostic procedures (ideas about diagnoses; d = 0.65) and treatment suggestions (ideas about patient management; d = 0.82) improved after using intervention

Moderate (5)

⬄ There was no significant difference between groups in diagnostic accuracy (ideas about diagnoses; d = 0.08)

Dekhtyar et al. 2021

To test the hypothesis that the Symptom to Diagnosis diagnostic reasoning approach videos paired with practice virtual patient encounter simulations could improve the diagnostic accuracy in medical students as evidenced by their ability to diagnose new simulated cases with diagnoses not previously encountered

Single group pre & post comparison

2nd & 3rd years. N = 285

Data gathering, ideas about diagnoses (reported separately)

Virtual patient cases

⇧ History taking efficiency (data gathering; d = 0.47), history taking completeness (data gathering d = 0.32); efficiency of differential diagnosis (ideas about diagnoses; d = 1.16) and completeness of differential diagnosis (ideas about diagnosis; d = 0.93) improved after using intervention

Low (3)

Isaza-Restrepo et al. 2018

To present evidence regarding the effectiveness of a low-fidelity simulator: Virtual Patient

Single group pre & post comparison

1st-5th years. N = 20

Data gathering, ideas about diagnoses, ideas about patient management

Standardised patient (actor)

⇧ Data gathering, ideas about diagnoses and patient management, and presentation of a case significantly improved after using intervention (average effect size across 5 dimensions from 3 evaluators, d = 1.41)

Moderate (6)

Kleinert et al. 2015

To examine whether the use of ALICE has positive impact on clinical reasoning and is a suitable tool for supporting the clinical teacher

Single group pre & post comparison

3rd years. N = 62

Ideas about diagnoses, ideas about patient management

Patient cases

⇧ Ideas about diagnoses and patient management significantly improved after using intervention (d = 0.92)

Low (3)

Watari et al. 2020

To clarify the effectiveness of VPSs for improving clinical reasoning skills among medical students, and to compare improvements in knowledge or clinical reasoning skills relevant to specific clinical scenarios

Single group pre & post comparison

4th years. N = 169

Data gathering, ideas about diagnoses, ideas about patient management

Multiple-choice question quiz (using patient cases)

⇧ Data gathering, ideas about diagnoses and patient management significantly improved after using intervention (d = 1.39)

Low (3)

Wu et al. 2014

To examine the effectiveness of a computer-based cognitive representation approach in supporting the learning of clinical reasoning

Single group pre & post comparison

3rd-5th years. N = 50

Problem-solving

Concept maps

⇧ Problem-solving significantly improved after using intervention (d = 1.17)

Moderate (5)