Table 1 Selected studies

Burden [66]

CVC

Level 4, 5

ML: Didactic lecture and simulation practice with feedback

4 h

N =?

Insertion in patients supervised by senior staff

Historical controls

N =?

Annual savings from decrease in infection $540,000

Cost-benefit from actual financial data adds strength to conclusion. CBE as part of bundle, pre-post CBE-setup. Other interventions than only ML-course

Cohen [67]

CVC

Level 4, 5

ML: Lecture and simulation training with feedback

4 h

N = 69

Traditionally trained, five supervised insertions

Historical controls

N =?

Annual savings from decrease in infection $700,000

Even including one-time investments, still resulted in positive Cost-benefit

Sherertz [68]

CVC

Level 3, 4, 5

CBE: Lecture, series of hands-on stations, one CVC.

3 h

N = 406

Conventional bedside and didactic instruction Historical controls

N = 140

Cost savings from decrease in infection 63,000-$800,000

CVC Infection control course, large groups, other relevant procedures taught

Evans [76]

CVC

Level 3, 4

ML: Added lecture, video presentations, observed simulated hands-on

1–8 attempts

N = 90

“see one, do one, teach one.”

Five supervised insertions on patients

Concurrent, randomized controls

N = 95

Significantly higher first pass success rate in clinical setting

Ultra sound guided, Very low complications, pre and post-intervention

Smith [81]

CVC

Level 3, 4

CBE: Added case based didactic discussion, hands on simulation training

2 h

N = 25(20)

Supervised performance on patients with immediate feedback

Concurrent, randomized controls

N = 27(8)

Intervention significant better knowledge and comfort in post-test, no difference to controls at 3 months. No difference in complications, nor needle passes

Skills decline over 3 months as argument for renewed skills training.

Khouli [69]

CVC

Level 4

CBE: Video and debriefing of hands on simulation training

? duration

N = 24

Historical:

Apprenticeship model “see one, do one, teach one.”

Concurrent: Video only

Historical and Concurrent, randomized controls

N = 23

Significantly lower infection rate in interventional department than in the control group and historically

Strength from RCT-setup and well-defined control group training. Comparison to other enhanced training to account for Hawthorne effect

Miranda [93]

CVC

Level 3, 4

CBE: Presentation, observed and supervised hands on simulated training

2.5 h

N = 40

Usual ward orientation

Firm based allocation

N = 110

Significantly larger increase in knowledge in intervention group, no difference in success rate.

No change in behavior or patient care, despite practical intervention. Infrequent insertion rate perhaps responsible for non-sustainable results.

Udani [77]

Spinal anesthesia

Level 3, 4

ML: additional training with deliberate practice and immediate feedback

? duration

N = 10

Base curriculum of written teaching materials and 15-min video

Randomized, concurrent controls

N = 11

Significantly better checklist scores post-training, higher failure rate in intervention group

Randomized and well-described control group training. Immediate transfer of simulated best clinical practice skills to real patients, enhances patient safety in early procedural training

Britt [70]

CVC

3, 4

ML: additional hand-on demonstration and performance

? duration

N = 13

Standard lecture followed by supervised training on patients

Randomized, concurrent controls

N = 21

Nonsignificant lower complication rate in intervention group, no effect on infection rate

Randomized. Just short of statistical significance for level 4-measures, population too small.

Barsuk [78]

CVC

Level 3, 4

ML: lecture, ultrasound and simulator training with feedback

4–5 h

N = 28

Traditionally trained, five supervised insertions

Historical controls

N = 13

Intervention group needed fewer needle passes in clinical performance

Only self-reported data on needle passes and self-confidence, introduces possible reporting bias.

Barsuk [71]

CVC

Level 4

ML: lecture, ultrasound and simulator training with feedback

4–5 h

N = 92

Lecture series, no formal training Historical controls in same ICU and concurrent controls in other ICU

N =?

Significantly lower infection rate in intervention group compared with historically and control group

Historical controls, no control for Hawthorne effect of altered behavior not stemming from the practical hands-on training.

Peltan [94]

CVC

Level 3, 4

ML: added supervised practice on simulator

1–2 h additional

N = 36

Lecture, interactive online module, familiarization to CVC equipment, instruction at all procedures

Randomized, concurrent controls

N = 37

Significant improvement in adherence to procedural protocol, no difference in other clinical performance

Strength from randomization. Direct observation enhances reliability of results. Equal clinical performance raises questions of the appropriateness of procedural protocol for improving outcome.

Barsuk [72]

CVC

Level 3, 4

ML: lecture, ultrasound and simulator training with feedback

4–5 h

N = 51

Lectures and by observing more experienced physicians performing CVC

Historical controls

N =?

Significant decrease in infection rate post-intervention in different hospital setting.

Enhances generalizability for results of the intervention, highlights the effort needed for implementation to succeed and the vulnerability of the intervention.

Sekiguchi [73]

CVC

Level 3, 4

CBE: Interactive video, hands on training

105 min

N = 56

Supervision of 10 subclavian, 10 internal jugular and 5 femoral vein insertions or 10 ultrasound guided procedures

Historical controls

N =?

Significant post-interventional decrease in complications, interns as fellows and attending physicians

Coinciding with increase in Ultrasound Guided insertion, which in itself decreases risk of mechanical error, possible confounder.

Hoskote [74]

CVC

Level 4

CBE:

Debriefing on simulated pre-test, hands on training and repetitive simulated practice

? duration

N = 60

Not stated

Historical controls

N =?

Change in policy owing to decrease in infection rate to benchmark level

Good example of organizational change due to enhances in procedural safety following CBE training program

Koh [83]

CVC

Level 3, 4

CBE: lecture, video demonstration, simulation

? duration, 5 CVCs

N = 32

No controls

Learning curve of 7 CVCs performed before acceptable complication and success rate reached

Not directly related to the training course, but interesting to establish learning curve

Martin [75]

CVC

Level 3, 4

CBE: Didactic sessions, supervised skills training on cadaver, videotaped and reviewed for repeated sessions

? duration

N =?

Advanced cardiac life support and advanced trauma and life support courses

Historical controls

N =?

Significant decrease in pneumothorax at 3 months, non-significant at 1 year

Although pre-graduate intervention, the results are postgraduate. Argument for effect of early training despite many procedures trained at the same time

Smith [109]

Fiberoptic intubation

Level 4

ML: Written instruction, simulated then clinical supervised training

Min 1 h

N = 12

No controls

95% completed within benchmark duration

Learning curve interesting for expected skills development in training.

Barsuk [99]

CVC

Level 4

ML: lecture, ultrasound and simulator training with feedback

0 h

N = 102

Lecture series, no formal training

Historical controls

N =?

Trickle-down effect of pre-test increase after first years of mastery learning course

Potential for additional effect of a training program, what kind of learning is transferred passively.

Friedman [95]

EDC

Level 3

CBE: Lectures on EDC insertion. High fidelity EDC-model

60 min, 15 insertions

N = 12

CBE: Lectures on EDC insertion. Low-fidelity banana model

Randomized, concurrent controls

N = 12

No difference between hi- and lo-fi intervention, higher score by experience

Effects from inexpensive models comparable to more expensive could reduce costs of training, leading to higher cost-benefit

Scavone [80]

General Anesthesia

Level 3

CBE: General anesthesia for emergency cesarean delivery

? duration

N =?

Lecture and General anesthesia scenario, unrelated to obstetric emergency

Randomized, concurrent controls

N =?

No difference in time to incision or confidence, I-group better score in repeated simulation

Adherence to scoring system perhaps enhanced safety, but did not lead to earlier operation, which would be a desirable outcome in real world.

Gaies [85]

Bag mask ventilation

Level 3

CBE: didactic session, observation and hands on, supervised practice

? duration

N = 18

Observing more experienced clinicians

Block randomized controls

N = 20

Significant skills decline in both groups in final test

Rapid decline in skills after early skills training, rarely performed procedure

Kulcsar [98]

Spinal Anesthesia

Level 3

CBE: Same teaching, but by simulator with haptic feedback

110 min

N = 14

Practical procedural subparts teaching, using an orange

Randomized, concurrent controls

N = 13

Non-significantly increased scores on clinical performance.

Less than half were tested clinically, very small study groups. Short follow-up 3 weeks.

Barsuk [96]

CVC

Level 3

ML: lecture, ultrasound and simulator training with feedback

4–5 h

N = 76

Lecture series, no formal training

Historical Controls

N = 27

No difference in quality indicators in clinical performance between groups

Only self-reported data of complications, risk of reporting bias in the intervention group.

Chan [101]

CVC

Level 3

CBE: Instruction and demonstration in parts, followed by practice

61/50 min average

N = 11 (part)

Instruction and demonstration in whole procedure, followed by practice

Randomized, concurrent controls

N = 8

Only Part Task significant better in Global Rating Scale at one-month retention, rest no difference.

Interesting that the difference was found in the overall global rating scale and not in the check lists for parts of the procedure, when comparing whole to part-task instruction

Friedman [79]

EDC

Level 3

CBE: Added 17 min demonstration video on aseptic technique

75 min

N = 18

Lecture on aseptic technique

Historical controls

N = 11

Significantly better scores at all intervals and in overall score of skills retention

Unclear to the extent of difference in training, only a new video or the subsequent clinical supervision as well?

Ortner [84]

General Anesthesia

Level 3

CBE: Full-scale general anesthesia scenario, supervised and debriefed

? duration

N = 24

Traditionally trained

attending physicians as benchmark performers

N = 6

Trainees reached benchmark level of attending physicians immediately and at 8 months

The short course seems as effective as experience in sustainable skills for a multidisciplinary procedure

Finan [97]

Airway management

Level 3

CBE: Didactic component, demonstration and supervised hands-on simulator training

2 h

N = 13

Standard course, one of more skills training sessions and subsequent clinical experience Historical controls

N =?

Significant lower clinical success rate and return to baseline skills after immediate effect

Kirkpatrick level 1 and 2 reached but could not be transferred into clinical practice. Cause? Fidelity, simulation not encompassing the variability of real life?

Millington [82]

CVC

Level 3

CBE: Multimedia educational material, demonstration followed by hands-on training

2 h

N = 30, 16 in retention phase

No controls

Significant increase in retention of knowledge, immediate post-training increase in skills also

Retention test of skills would have been preferable to knowledge retention as an effect measure since other studies have shown retention discrepancies between the two.

Garood [86]

CVC

Level 3

CBE: One of more skills training stations in one day course. Small group training

One day

N = 41

No controls

Immediate confidence increase, significant decrease at 3 months

Self-reported confidence in clinical encounters is a weak measure of learning effect, subject to reporting bias.

Lenchus [100]

CVC

Level 3

CBE: Video instruction, discussion, instruction on ultrasound guidance, demonstration, individual practice

160 min

N = 60

No controls

Significant improvement in clinical performance score

Very short training time, but until competency? Better adherence to checklist = better procedural performance or clinical outcome.

Lenchus [103]

CVC

Level 3

CBE: Video instruction, discussion, instruction on ultrasound guidance, demonstration, individual practice

4 h

N = 60

No controls

Significant immediate improvement in knowledge and procedural checklist score

Same setting as above, unclear if the post-instruction score was on the first real patient performance.

Thomas [87]

CVC

Level 3

ML: Instructional video, supervised hands-on training

60–90 min

N = 26

No controls

Confidence significantly improved at 3 months, clinical scores deteriorated.

Argument for mandatory retesting and training, as residents own perception of skills was incongruent with actual skills performance after three months.

Barsuk [88]

CVC

Level 3

ML: lecture, ultrasound and simulator training with feedback

4 h

N = 49

No controls

Significant decline in skills test at 6 and 12 months after initial improvement.

Another powerful argument for repeated testing and remedial training, skills decay over time if not.

Laack [89]

CVC

Level 3

ML: interactive learning stations of part tasks, supervised

4 h

N = 26

No controls

Significant skill decay after 3 months

Remedial training argument for maintenance of initially acquired skills

Siddiqui [90]

EDC

Level 3

ML: Lecture, video, hands-on training on lo-fi model

Duration > 45 min

N = 21

No controls

Retention score consistently over benchmark

Strong argument for hands-on training, also for aseptic technique

Diederich [91]

CVC

Level 3

ML: Low-fidelity mannequin trainer, instructional video, partwise instruction and hands-on training with immediate feedback

? duration

N = 20 (Low-fidelity)

ML: High-fidelity mannequin trainer, instructional video, partwise instruction and hands-on training with immediate feedback

N = 20 (High-fidelity)

Both groups performed above the minimum passing score at 4 weeks retention test

Strength from randomization and from well-defined ML-interventions in both groups. Possible cost-saving potential from low-fi non-inferiority. Short follow-up (4 weeks).

Cartier [92]

CVC

Level 3

CBE: Instructional video and hands-on, videotaped simulations, supervised by peers

? duration

N = 37, 18 sustainability tested

No controls

Significant skills and knowledge increase from pre-training to post training and subsequent decline to > 2 years sustainability test.

Possible Hawthorne-effect from one cohort pre-post testing. Large dropout to sustainability. Still effect of training although diminishing after 2 years as argument for remedial training at interval shorter intervals than 2 years.