The efficacy of simulation-based learning versus non-simulation-based learning in endocrinology education: a systematic review and meta-analysis

Abstract

Background

Simulation-Based Learning (SBL) is increasingly adopted in medical education across various specialties, employing realistic simulations to significantly enhance learning experiences. However, a comprehensive evaluation of its effectiveness specifically in endocrinology has not yet been conducted. The study aims to systematically review and meta-analyze the impact of SBL versus Non-Simulation-Based Learning (NSBL) on knowledge acquisition, skills, satisfaction, and interest in learning among endocrinology trainees.

Methods

This systematic review and meta-analysis adhered to the PRISMA guidelines, searching PubMed, Web of Science, Embase, Cochrane library, China National Knowledge Infrastructure (CNKI), Wanfang Data, Weipu, and Chinese Biomedical Database (CBM) until March 2024. We included randomized controlled trials comparing SBL to NSBL in endocrinology education. The quality evaluation relied on the Cochrane risk-of-bias assessment tool. The main results included evaluations from both theoretical and practical assessments. Additional measures consisted of assessing satisfaction and interest in learning.

Results

We identified 22 studies suitable for systematic review and 21 for meta-analysis, involving a total of 2517 participants. SBL greatly enhanced theoretical knowledge [standardized mean difference (SMD) = 1.00, 95% confidence interval (CI): 0.68–1.32, P < 0.00001, I² = 89%] and practical skills (SMD = 1.56, 95% CI: 1.11–2.01, P < 0.00001, I² = 93%) compared to NSBL. Additionally, SBL was associated with higher satisfaction and greater interest in learning. No significant publication bias was detected, and sensitivity analysis confirmed the stability of these findings.

Conclusions

SBL significantly enhances knowledge, skills, satisfaction, and interest in learning within endocrinology education compared to NSBL. These findings support the integration of high-quality SBL into endocrinology curricula to improve educational outcomes. Future research should explore the lasting effects of SBL on knowledge retention and clinical practice, as well as to evaluate its cost-effectiveness and compatibility with various educational tools in diverse settings.

Background

Endocrine professionals are confronted with increasing complexities and challenges in their field, underscoring the critical importance of providing education and guidance [1]. For many years, lecture-based instruction has been seen as the conventional approach to medical education for teaching theoretical concepts, in addition to alternative methods like Case-Based Learning (CBL) and Problem-Based Learning (PBL) [2]. While these approaches in endocrinology are valuable, they possess inherent limitations, particularly concerning the depth of practical experience. Moreover, the time honored concept of “see one, do one, teach one” apprenticeship model for skill training is no longer tenable, primarily due to considerations regarding patient safety [3].

Simulation-Based Learning (SBL) has emerged as a promising educational approach in medical education, offering immersive experiences that serve as an alternative to traditional methods. Essentially, simulations provide learners with hands-on, experiential learning through carefully designed scenarios, rather than working with real patients in a clinical setting [4]. In practice, simulation methods utilize a variety of tools, including virtual reality software, realistic mannequins, plastic models, animal samples, and human cadavers [5].

Previous studies have shown that SBL is effective in various medical specialties, including anesthesiology, obstetrics and nursing education [6,7,8]. However, its efficacy in the field of endocrinology remains debated. Several studies have reported positive outcomes, highlighting its potential to enhance knowledge, decision-making, and confidence, all of which are critical competencies in endocrinology [9,10,11]. Nevertheless, results have varied depending on the specific intervention and outcome measure used. Therefore, the primary goal of this systematic review and meta-analysis is to evaluate the effectiveness of SBL compared to Non-Simulation-Based learning (NSBL) in enhancing the knowledge, skills, satisfaction, and interest in learning among endocrinology trainees, through a comprehensive analysis of existing research studies.

Methods

The systematic review and meta-analysis followed PRISMA guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) during its design, execution, and reporting [12]. The protocol for this study can be found on PROSPERO (CRD42024528630).

Search strategy

We conducted searches in PubMed, Web of Science, Embase, Cochrane Library, China National Knowledge Infrastructure (CNKI), Wanfang Data, Weipu, and the Chinese Biomedical Database (CBM). During the search, only English and Chinese publications were considered, with an end date of March 2024. A mix of MeSH terms and free text keywords was utilized for the retrieval process. The complete electronic search methods can be found in the supplementary document (additional file 1). Furthermore, we hand-searched previously published reviews and Google Scholar for additional eligible studies.

Inclusion and exclusion criteria

The following criteria must be met for inclusion. (1) The study population should target medical students, residents, and staff members associated with endocrinology. (2) The interventions implemented should focus on SBL techniques within the realm of endocrinology. This encompasses all types of simulators used to emulate clinical scenarios for training or assessment purposes, including computer-based models, manikins, standardized patients, and high-fidelity simulators, whether employed in isolation or in conjunction with conventional instructional approaches. (3) The comparator, defined as NSBL, involves traditional learning techniques. (4) Studies must report primary or secondary outcomes that are pertinent to the research being conducted. (5) The study must follow the guidelines for a randomized controlled trial (RCT) in its design.

Exclusion criteria for this study include: (1) The integration of simulation-based interventions with innovative educational methods that are not related to simulation, in order to mitigate potential biases; (2) Studies lacking full-text accessibility; (3) Conference abstracts, case reports, editorials, opinion pieces, reviews, systematic reviews, and study protocols.

Study selection and data extraction

Following the predetermined search strategy, all relevant articles were retrieved. EndNote X9 software (Clarivate Analytics, Philadelphia, PA, USA) was used to eliminate duplicate articles at the beginning. Two authors (ZYW and YLH) separately assessed the titles and abstracts of the remaining papers according to the inclusion and exclusion criteria. Discrepancies between the authors were resolved by seeking input from a third researcher (LL). The full texts of potentially eligible articles were then obtained and reviewed in a similar manner to ultimately determine the studies to be included.

The authors’ information, publication year, participant characteristics, interventions, comparisons, and outcomes were systematically extracted. Primary outcomes consisted of scores from theoretical knowledge and skills assessments, while secondary outcomes included measures of teaching satisfaction and learning interest.

Assessment of risk of bias

The methodological quality of the selected studies was independently assessed by two authors (ZYW and LL) based on the Cochrane risk-of-bias tool [13]. The evaluation components included the randomization process, allocation concealment, masking of participants and staff, masking of outcome assessment, incomplete outcome data, selective reporting, and other potential sources of bias. Each item was categorized as having either low, unclear, or high risk levels. Quality assessment of trials focused on the risk of bias related to randomization and allocation concealment. Trials were considered high quality if both factors had a low risk of bias, along with all other items having a low or unclear risk of bias. Trials with a high risk of bias in either randomization or allocation concealment were classified as low quality, regardless of other factors. Trials that did not meet the criteria for high or low risk of bias were classified as moderate quality.

Statistical analysis

Review Manager 5.4 software (Cochrane Collaboration, Oxford, UK) was utilized to synthesize the data. Continuous variables were reported as standardized mean difference (SMD) with 95% confidence interval (CI). Binary variables were expressed as risk ratio (RR) with corresponding 95% CI. Statistical significance was determined by a p-value of less than 0.05. Study heterogeneity was evaluated using the I² test. In cases where heterogeneity was not significant (I² < 50%), a fixed-effect model was applied. Conversely, a random-effect model was utilized, along with subgroup analyses to investigate potential sources of heterogeneity based on participant demographics, study regions, study quality and simulation technique. To assess publication bias, funnel plots were visually inspected in Review Manager 5.4 and quantitatively analyzed using Egger’s regression and Begg’s rank correlation tests in Stata 12.0 software (StataCorp LP, College Station, TX, USA) for outcomes with at least 10 included studies. Additionally, a sensitivity analysis was conducted using the leave-one-out technique to evaluate the robustness of the combined effect estimates. To further ensure the reliability of our findings, we also tested the results from the fixed-effects model with a random-effects model, as the latter provides more conservative summary estimates.

Fig. 1

PRISMA diagram of the process for selecting studies

Results

Study selection and characteristics

A total of 2,926 studies were initially identified through database searches, with an additional 2 eligible studies found through screening on Google Scholar. After deduplication, 2,428 articles underwent citation screening, leading to the exclusion of 2,341 studies based on titles and abstracts. Subsequently, 87 full-text manuscripts were retrieved for further evaluation. After conducting full-text screening, 22 articles [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35] were found to meet the criteria for inclusion in the systematic review, with 21 articles [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33, 35] considered appropriate for meta-analysis. Comprehensive justifications for excluding studies are available in the PRISMA diagram (Fig. 1).

Table 1 Characteristics of the included studies

The characteristics of the included studies are summarized in Table 1. These studies involved a total of 2,517 individuals, with 1,251 assigned to the SBL cohort and 1,266 to the NSBL cohort. All studies were published between 2003 and 2024, with 5 studies [14, 15, 17, 26, 27] in English and the remaining 17 studies [16, 18,19,20,21,22,23,24,25, 28,29,30,31,32,33,34,35] in Chinese. The study samples ranged from 20 to 200 individuals and consisted of undergraduates, interns, residents, and primary care physicians. Among these studies, 10 studies [15, 16, 19, 20, 22, 25, 30,31,32, 34] used standardized patients, 6 [14, 17, 18, 27, 28, 35] employed computer-based simulations, and the remaining 6 [21, 23, 24, 26, 29, 33] utilized scenario-based simulation methods for educational purposes. The instructional content covered endocrine physiology [26, 35], pathophysiology [26, 35], and common endocrine disorders such as diabetes [15,16,17,18, 25, 27, 29, 30, 33, 34], thyroid diseases [14, 18, 23,24,25, 28, 34], adrenal disorders [16, 34], and pituitary disorders [16, 31, 34]. Among these studies, 9 [15, 17, 23, 24, 27, 28, 30, 31, 33] focused on a single topic, while 13 [14, 16, 18,19,20,21,22, 25, 26, 29, 32, 34, 35] covered two or more instructional contents. A total of 17 studies [14, 16,17,18,19, 21,22,23,24,25,26,27,28,29,30,31, 33] assessed knowledge levels, while 14 studies [15, 16, 19,20,21,22,23,24, 28,29,30, 32, 33, 35] evaluated skills. Additionally, 8 [16,17,18, 22, 28, 29, 31, 35] studies reported on satisfaction, and 9 [16, 19, 21, 23, 24, 28, 29, 31, 35] studies reported on interest.

Assessment of risk bias

The bias risk assessment of the included RCTs is summarized in Figs. 2 and 3. Nine studies detailing methods for generating random sequences were deemed to have a low risk of bias, using methods such as random number tables, random number generators, random draws, and third-party online randomization tools. Three studies that assigned participants by internship date, departmental rotation order, or class cohort were judged as high risk. The remaining 10 studies did not offer pertinent details and were judged as having an unclear risk of bias. Only 1 study was categorized as low risk in terms of describing the method of allocation concealment. Regarding performance bias, 13 studies were assessed as low risk either because participants and implementers were unaware of the group assignments, or because detailed educational specifications for both the experimental and control groups were provided to mitigate potential biases due to human factors. In the context of detection bias, 11 studies that promoted objectivity, consistency, or anonymous evaluation were deemed low risk. Two studies discussed attrition, and all studies were assessed as having a low risk of reporting bias and other bias. To summarize, one study [26] was rated as high quality, while three studies [21, 24, 34] were rated as low quality, with the rest falling into the category of moderate quality.

Fig. 2

The risk of bias summary represents the review authors’ evaluations of each risk of bias element for every study included. Green indicates a low risk of bias, while yellow indicates unclear risk of bias

Fig. 3

The risk of bias graph shows the review authors’ evaluations of each bias item as percentages across all studies included

Theory test and skill test

Seventeen studies [14, 16,17,18,19, 21,22,23,24,25,26,27,28,29,30,31, 33] reported test scores of theoretical knowledge (811 subjects in the SBL cohort and 830 subjects in the NSBL cohort), demonstrating a significant advantage of SBL over NSBL (SMD = 1.00; 95% CI: 0.68–1.32, P < 0.00001, I² = 89%) (Fig. 4). Additionally, 14 studies [15, 16, 19,20,21,22,23,24, 28,29,30, 32, 33, 35] reported post-intervention skill test scores (801 subjects in the SBL cohort and 793 subjects in the NSBL cohort), with the pooled results indicating the superiority of SBL over NSBL (SMD = 1.56; 95% CI: 1.11–2.01, P < 0.00001, I² = 93%) (Fig. 5). In terms of publication bias, the funnel plot of knowledge and skill scores is depicted in the supplementary document (additional file 2). Neither Egger’s test (theory test: P = 0.051; skill test: P = 0.662) nor Begg’s test (theory test: P = 0.108; skill test: P = 0.125) revealed significant publication bias.

Fig. 4

Forest plot displaying results from randomized controlled trials assessing the efficacy of SBL on theoretical knowledge

Fig. 5

Forest plot displaying results from randomized controlled trials assessing the efficacy of SBL on skill performance

Subgroup analysis

Given the observed significant heterogeneity in both theoretical and skill tests, further analyses were carried out to investigate potential factors contributing to the variation. By employing sensitivity analysis through the one-by-one elimination method, the I² values fluctuated between 87% and 90% for theoretical assessments and between 90% and 94% for skill evaluations. It is noteworthy that the findings of the meta-analysis remained stable despite these fluctuations.

Following this, subgroup analyses were conducted based on study quality (low-quality studies vs. medium to high-quality studies), participant status (undergraduate vs. postgraduate), study region (China vs. other regions) and simulation technique (computer-based simulation vs. scenario-based simulation vs. standardized patient). The results are presented in Table 2. However, the aforementioned four factors did not contribute to the observed heterogeneity.

Table 2 Results of subgroup analysis

Satisfaction evaluation and learning interest

Eight studies [16,17,18, 22, 28, 29, 31, 35] reported evaluations of satisfaction with teaching methods, with six studies [16, 17, 22, 29, 31, 35] presenting count data (RR = 1.43; 95% CI: 1.28–1.60, P < 0.00001, I² = 0%) (additional file 3) and two studies [18, 28] presenting continuous data (SMD = 0.51; 95% CI: 0.28–0.74, P < 0.0001, I² = 16%) (additional file 3). The combined results consistently indicated higher satisfaction among participants with SBL methods.

Similarly, nine studies [16, 19, 21, 23, 24, 28, 29, 31, 35] assessed participants’ interest in learning, with six studies [16, 19, 23, 29, 31, 35] reporting count data (RR = 1.56; 95% CI: 1.22–1.99, P = 0.0004, I² = 77%) (additional file 4) and three studies [21, 24, 28] reporting continuous data (SMD = 1.39; 95% CI: 1.11–1.67, P < 0.00001, I² = 0%) (additional file 4). The results demonstrated that SBL was more effective in stimulating learning interest compared to NSBL.

Sensitivity analysis using the one-by-one elimination method showed no significant change in the combined results, confirming the stability of the conclusions. Revalidating the results originally synthesized using a fixed-effects model with a random-effects model showed only minor differences in the pooled effect sizes and confidence intervals, or no change at all. This further ensures the robustness of our conclusions.

Discussion

Our results suggest the potential advantages of SBL over NSBL across various educational outcomes in endocrinology. Notably, SBL significantly improves the theoretical knowledge and practical skills of medical professionals. This is consistent with the immersive and experiential nature of SBL, which fosters an interactive and engaging environment that more accurately mirrors clinical scenarios. By providing learners with practical experiences, SBL enables the acquisition of skills and knowledge that traditional learning methods struggle to replicate.

The simulation techniques utilized in the analyzed studies were categorized into three primary types: computer-based simulation, scenario-based simulation, and standardized patient, each possessing distinct attributes that contribute to their efficacy in endocrinology education. Despite sharing common features such as interactive design, timely feedback, and the establishment of a safe learning environment for developing intricate medical skills, notable disparities exist among the three methods. Computer-based simulations employ software applications that can be utilized on local machines or web-based platforms, providing significant adaptability and scalability for repetitive practice and self-evaluation across diverse domains of knowledge and expertise [36, 37]. Scenario-based simulations utilize prearranged scenarios and settings, frequently integrating role-playing and tangible equipment, and occasionally incorporating narrative elements to enrich the learning experience [38]. This approach effectively transforms theoretical medical principles into practical situations, thereby enhancing learner involvement and comprehension. Standardized patients involve trained actors or volunteers who simulate real patient scenarios, providing highly realistic clinical interactions that are beneficial for developing communication and clinical reasoning skills [39]. Understanding these distinctions is essential for maximizing the effectiveness of simulation teaching strategies and achieving optimal educational outcomes.

The research conducted by Diehl et al. employed electronic simulation gaming as a method to instruct primary care physicians on insulin therapy, yielding notable improvements in participants’ abilities following the intervention [17]. This aligns with our findings, supporting the notion that SBL fosters greater learning and skill development, thus establishing a robust foundation for clinical practice. Furthermore, the flexibility of the game and favorable user responses suggest its potential as a feasible choice for widespread continuing medical education initiatives.

Building on a purely simulation-based foundation, the Double-S method, an innovative example introduced by Hassanzadeh Rad et al., illustrates how simulations can be integrated into medical curricula to enhance the understanding of complex physiological concepts, such as congenital hypothyroidism [26]. Combining simulation with storytelling, the Double-S method offers a richer and more multidimensional learning experience. By metaphorically representing the thyroid gland as a kitchen and narrating the process of thyroid hormone production, this method transforms abstract physiological processes into tangible and relatable scenarios. The success of the Double-S method may provide a new direction for future endocrinology education, enhancing the effectiveness and appeal of training by merging simulation and storytelling. This improvement serves as a vital link in specialized areas such as endocrinology, where a thorough understanding of physiological and biochemical principles is imperative.

Endocrinology involves complex decision-making because of the intricate nature of hormone-related disorders [40]. According to Brown et al., utilizing standardized patients provides a consistent environment for students, allowing them to safely interact with and manage newly diagnosed diabetes patients under supervision [15]. SBL offers immediate feedback, error analysis, and practice in a risk-free, controlled setting. This approach is essential for developing the advanced decision-making skills required in endocrinology.

Although SBL clearly enhances immediate learning outcomes, its impact on long-term skill and knowledge retention remains uncertain. This observation aligns with findings from other disciplines, which suggest that while simulations improve immediate skill acquisition, they show inconsistent effects on long-term maintenance without additional reinforcement or repeated training [41]. Integrating supplementary learning components and regular reviews through simulations can help sustain and enhance these skills over time [42, 43]. This is particularly important in endocrinology, where the complexity of the field demands sustained competency and continuous skill refreshment.

Furthermore, studies report significant improvements in learner satisfaction and interest, suggesting the motivational benefits of SBL, which can be attributed to its interactivity, hands-on experience, and engaging nature. These elements may engage learners more effectively than traditional teaching methods. Some studies suggest that SBL enhances clinical reasoning and teamwork skills, which are crucial competencies for endocrinology professionals [18, 28, 29]. Although these capabilities were not directly measured in this review, they are implied by the observed increases in knowledge, satisfaction and interest.

A primary strength of our study is the comprehensive search strategy and strict adherence to PRISMA guidelines, which ensure the rigor and reliability of our findings. However, we must acknowledge the substantial heterogeneity in the results, likely attributed to several factors. These include variations in educational designs and teaching contents, participant characteristics such as instructor expertise and learner abilities, and the specific simulation technologies employed. Additionally, discrepancies in the NSBL techniques used as control measures in the analyzed studies may contribute to this heterogeneity. Notably, two of the studies did not provide a detailed account of the NSBL interventions utilized, complicating the comparison between the SBL and NSBL groups. Despite conducting subgroup analyses by study quality, participant status, study region and simulation technique, the sources of heterogeneity remain unidentified. This suggests the presence of other unmeasured variables or methodological differences, warranting further investigation. The high risk of bias, particularly in random sequence generation and allocation concealment in some studies, may limit the generalizability of our findings. Future research should address these issues with more robust randomization and standardized control measures.

As endocrinology continues to evolve and face new challenges, educational methodologies should adapt to provide the most effective training. SBL may represents a promising advancement in this direction, serving as a valuable supplement to traditional learning methods. To effectively utilize SBL, it is essential to consider factors such as simulation fidelity, integration with conventional instructional approaches, and the development of modules that cater to various learning styles and requirements. A study by Junghee et al. indicates that high-fidelity simulations, which offer realistic clinical scenarios, accurate anatomical models, personalized learning, and immediate feedback, are more effective [44]. This suggests that investing in high-quality simulation resources could enhance learning outcomes. Future research should identify the optimal mix of simulation-based and traditional learning. Analyzing the compatibility between different teaching contents and simulation techniques is also crucial, as it helps determine the most effective use of simulations for various subjects. Given the substantial resources required for high-fidelity simulations, investigating their cost-effectiveness in endocrinology training across different settings is necessary. Additionally, more research is needed to explore how simulation-based education influences learning, to standardize simulation training, and assess SBL’s long-term impact on clinical practice. It is also vital to identify strategies that enhance the sustainability of the skills and knowledge acquired.

Conclusions

This systematic review and meta-analysis demonstrates that SBL surpasses NSBL in enhancing knowledge, skills, satisfaction, and interest within endocrinology education. The current evidence supports the integration of SBL as an effective educational tool in endocrinology. We recommend that educational institutions and governing bodies consider incorporating high-quality SBL courses into the endocrinology curriculum. This integration with traditional methods should be approached thoughtfully, considering specific contexts and individual learner needs, ensuring high simulation fidelity and ongoing adjustments based on educational research and technological advancements. It is crucial to conduct further research to optimize these elements to enhance the efficacy and sustainability of SBL in medical education.