Effectiveness of simulation-based nursing education depending on fidelity: a meta-analysis

Clinical education in nursing aims to integrate theoretical knowledge from books into practical knowledge in real-life situations and to help students develop their problem-solving skills. Due to rapid changes in clinical placements, patient safety issues, and ethical concerns, students’ direct experience with patient care and opportunities to handle problem-based clinical situations have been diminished. Simulation-based clinical education is a useful pedagogical approach that provides nursing students with opportunities to practice their clinical and decision-making skills through varied real-life situational experiences, without compromising the patient’s well-being.

Simulation-based clinical education in nursing refers to a variety of activities using patient simulators, including devices, trained persons, lifelike virtual environments, and role-playing, not just handling mannequins [1]. With realistic clinical scenarios, simulation-based educational interventions in nursing can train novice as well as experienced nurses, helping them develop effective non-technical skills, practice rare emergency situations, and providing a variety of authentic life-threatening situations. The advantages of simulation-based educational interventions include the ability to provide immediate feedback, repetitive practice learning, the integration of simulation into the curriculum, the ability to adjust the difficulty level, opportunities to individualize learning, and the adaptability to diverse types of learning strategies [1].

Simulation can be described as a continuum ranging from low-fidelity simulation (LFS) to high-fidelity simulation (HFS) [2]. Various simulation methods can be adapted according to specific learning outcomes and educational levels. Dieckmann [3] warns against placing too much emphasis on having optimal equipment and surroundings that realistically replicate the clinical setting. The required learning outcomes must govern the choice of simulation method [4].

A number of research studies in nursing have evaluated the effectiveness of simulation-based educational interventions [5]. However, the reported effectiveness has varied according to the fidelity level of the simulators and the outcome variables. Issenberg et al. [1] found that HFS was effective for learning in medicine. However, their review was limited to HFS, medical education, and learner outcome variables, and did not compare simulation methods. Therefore, a meta-analysis synthesizing the results of these studies is needed to provide important insights into the level of simulation fidelity that is most effective for educational use.

The aims of this study were to determine the effect size of a simulation’s impact on nursing education and compare effect sizes according to the fidelity level of the simulators used.

This study was planned and conducted in adherence to PRISMA standards [6] of quality for reporting meta-analysis. We also considered the PRISMA criteria based on the PRISMA 2009 checklist in reporting each section, such as introduction, methods, results, and discussion.

Study selection

Studies published between January 1995 and July 2013 were identified by conducting an electronic search of the following databases: EBSCO, Medline, ScienceDirect, ERIC, RISS, and the National Assembly Library of Korea database. The literature search was limited to articles published in English or Korean and was conducted using combinations of the keyword phrases nursing, simulation, human patient, and simulator. A total of 2279 potential studies were identified. Titles and abstracts were reviewed for eligibility.

Relevant studies were screened for inclusion based on the following criteria: 1) the study aimed to evaluate the effectiveness of simulation-based education for nursing students, and 2) an experimental or quasi-experimental design was used. We excluded articles that did not report a control group or that tested the effectiveness of computer-based virtual patients. For abstracts that did not provide sufficient information to determine eligibility, full-length articles were retrieved. Disagreement on the inclusion or exclusion of articles was resolved by consensus. Of the potentially relevant 2279 articles, screening of the title and abstracts resulted in 317 relevant studies. After a review of these articles, 96 studies were retained and three articles included additionally via hand search. These 99 full-text articles were reviewed systematically to confirm their eligibility (Fig. 1).

The present study provided meta-analytical data for evidence-based education through a comprehensive analysis of simulation-based nursing education with diverse backgrounds and characteristics. Compared with our previous article “Effectiveness of patient simulation in nursing education: meta-analysis” [14], the current study included an additional electronic search of Korean databases such as RISS and the National Assembly Library of Korea database. Through this process, 20 Korean papers were included additionally and half of papers were Korean. This could cause different result compared to previous one. In addition to including a reaction outcome according to fidelity levels, effect sizes based on outcomes and fidelity level were identified. A systematic search of the literature resulted in 40 published studies that were eligible for inclusion in this meta-analysis. These primary studies provided evidence of the effects of simulation-based nursing education in various evaluation and learning environments.

Random assignment studies accounted for 62.5 % of the studies included. This represents a noticeable increase in randomized research designs, which made up less than 30 % of studies in a systematic review conducted 10 years ago on HSF in medical education [1]. That review found that HFSs were used in 50 % of studies, and 25 % used SPs, which is similar to the findings of the study by Kim, Park, and Shin [15]. This confirms the relatively high usage of HFSs and SP in nursing education.

The medium-to-large effect size (0.70) suggests that simulation-based nursing education is effective. This is consistent with the findings of a study on health professional education [16], which reported that technology-enhanced simulation training produced moderate to large effects.

Regarding simulator fidelity level, HFS (0.86), MFS (1.03), and SP (0.86) displayed larger effect sizes compared to LFS or hybrid simulation. This result supports the findings of a previous meta-analysis of simulation in health professions, showing that HSF offers benefits over LFS [17]. However, these findings should be interpreted with caution. Recent studies suggest that the degree of realism required of a simulation is a function of the learning task and context, and can therefore vary widely for different areas of educational outcomes [17].

In the reaction domain, which includes satisfaction and learning attitudes, HFS had a larger effect size than LFS. Satisfaction levels are high among students participating in simulation learning that utilizes human simulators or SP [18]. Considering that problem-based learning (PBL) lessons were found to enhance student attitudes more than traditional lectures [19], student participation and actual activity appear to have positive effects on satisfaction and learning attitudes.

In the sub-group analysis for learning outcome according to fidelity level, the effect size was the largest for psychomotor outcome, followed by affective and cognitive outcomes. This result differs somewhat from the meta-analysis on the effects of PBL [19], in which effect sizes were the largest for psychomotor outcomes, followed by the cognitive and affective domains. This difference is interpreted as reflecting PBL’s emphasis on reasoning based on problems and cases, compared to the actual clinical practice emphasized in simulation-based learning.

Specifically, the effect size of cognitive outcome was the largest for HFS (0.50), while the order for affective outcome was HFS (0.80), followed by SP (0.73). In the psychomotor domain, the order was MFS (1.76), SP (1.27), and HFS (1.03). These results demonstrate that HFS and SP are effective in producing cognitive and affective outcomes; however, to achieve psychomotor learning outcomes, technical training using MFS would be more helpful, which concurs with the lack of positive association between fidelity and process skills [17].

However, the present study has the limitation of not considering learning-related factors in the analyses based on the fidelity level of simulators. Even though debriefing has become more crucial in simulation-based learning and the methods have diversified over the years, a few selected studies do not indicate the methods of debriefing they had used, making it difficult to categorize and discuss the effects of each debriefing method. This may be because it is customary to omit debriefing while learning from low fidelity simulations, especially for training simple nursing skills. As such, the present study has the limitation of not considering learning-related factors from debriefing at each fidelity level of simulators, including reflection, feedback, and a range of debriefing methods (self-debriefing, multimedia debriefing, and/or in-simulation instructor facilitated debriefing). In addition, we did not include studies published in languages other than English or Korean.

Despite such limitations, this study demonstrated that simulation-based nursing education has an educational effect, with particularly strong effects in the psychomotor domain. Since the effects are not proportional to fidelity level, educational interventions should be broad enough to satisfy educational goals, all of which are supported by the results presented above. In addition, a recent study reported that debriefing was the most important factor in simulation, with positive effects from self-debriefing and video-facilitated instructor debriefing [20]. Based on these findings, the clinical reflection process needs to be improved to increase the learning effects in the cognitive domain.

Our results indicated that simulation-based nursing educational interventions were effective with particularly large effects in the psychomotor domain. In addition, the effect of simulation-based nursing education was not proportional to fidelity level. Therefore, it is important to use an appropriate level of simulation to meet all of the educational goals and outcomes.