Skip to main content
Download PDF

Integrated virtual simulation and face-to-face simulation for clinical judgment training among undergraduate nursing students: a mixed-methods study

BMC Medical Education volume 24, Article number: 32 (2024) Cite this article

Abstract

Background

Virtual simulation and face-to-face simulation are effective for clinical judgment training. Rare studies have tried to improve clinical judgment ability by applying virtual simulation and face-to-face simulation together. This study aimed to evaluate the effect of an integrated non-immersive virtual simulation and high-fidelity face-to-face simulation program on enhancing nursing students’ clinical judgment ability and understanding of nursing students’ experiences of the combined simulation.

Methods

A sequential exploratory mixed-methods study was conducted in a nursing simulation center of a university in Central China. Third-year nursing students (n = 122) taking clinical training in ICUs were subsequentially assigned to the integrated non-immersive virtual simulation and high-fidelity face-to-face simulation program arm (n = 61) or the face-to-face simulation-only arm (n = 61) according to the order in which they entered in ICU training. Clinical judgment ability was measured by the Lasater Clinical Judgment Rubric (LCJR). Focus group interviews were conducted to gather qualitative data.

Results

Students in both arms demonstrated significant improvement in clinical judgment ability scores after simulation, and students in the integrated arm reported more improvement than students in the face-to-face simulation-only arm. The qualitative quotes provided a context for the quantitative improvement measured by the LJCR in the integrated arm. Most of the quantitative findings were confirmed by qualitative findings, including the domains and items in the LJCR. The findings verified and favored the effect of the combination of non-immersive virtual simulation and high-fidelity face-to-face simulation integrated program on enhancing nursing students’ clinical judgment ability.

Conclusions

The integrated virtual simulation and face-to-face simulation program was feasible and enhanced nursing students’ self-reported clinical judgment ability. This integrated non-immersive virtual simulation and high-fidelity face-to-face simulation program may benefit nursing students and newly graduated nurses in the ICU more than face-to-face simulation only.

Peer Review reports

Background

Clinical judgment ability refers to applying knowledge and professional skills to a clinical situation to solve problems and is a core competency that nursing students should possess before transited to registered nurses [1, 2]. With the high acuity of patients’ conditions and various public health emergencies, the complicated care environment requires nurses to employ good judgment ability to maintain patients’ safety and improve patients’ outcomes, which requires rapid assessment and judgment of patients so that patients receive timely and effective care [3, 4]. Nursing students take advantage of clinical training in various clinical settings, including intensive care units (ICUs), to enhance their clinical judgment ability [5]. Massive lectures and limited clinical placement opportunities impede nursing students' immersion in adequate clinical experiences [6], thus hindering the training of clinical judgment ability among nursing students.

With advances in technology, new approaches to nursing education are being used in nursing curricula, such as simulation and virtual simulation, to improve the clinical judgment skills of nursing students [7, 8]. Several benefits of high-fidelity simulation have already been established, including enhanced performance during simulated resuscitation and improved triage skills [9, 10], enhanced nursing students' communication, teamwork, and ability to manage complex situations [11, 12]. A virtual simulation is an innovative approach to providing virtual clinical experiences through a digital platform where learners can complete specific tasks in various potential environments, use the information to provide assessment and care, make clinical decisions, and observe the outcomes of actions [13, 14]. Virtual simulation allows nursing students to familiarize themselves first with problems encountered in simulation by providing flexible and repetitive exercises [15]. Although previous studies have the effect of simulation-based intensive care nursing training on enhancing nursing students’ performance [9, 12], and the impact of virtual simulation training on improving clinical judgment ability in newly graduated nurses [16], rare studies have tried to improve clinical judgment ability by integrating virtual simulation and face-to-face simulation together among nursing students during the clinical training in ICUs.

The present study aimed to verify the effect of a non-immersive virtual simulation and high-fidelity face-to-face simulation integrated program and compared the effect between the intergrade program (Integrated) and the face-to-face stimulation only (Simulation) on enhancing the clinical judgment ability of third-year nursing students during clinical training in ICUs. One of the vSim for Nursing scenarios (Version 1, Wolters Kluwer, Philadelphia, PA, USA) was used in the virtual simulation (acute pulmonary embolism); this scenario (acute pulmonary embolism) was also adopted into high-fidelity face-to-face simulation scenario in the current study following the standards of best practice for simulation [17]. The hypotheses included: a) both Integrated program and Simulation could improve students’ clinical judgment ability, b) students in the Integrated arm improve more than students in the Simulation arm, and 3) the simulation was acceptable among nursing students. This study also explored students’ experiences of the integrated program to provide context for the effects of virtual simulation.

Methods

Study design

An explanatory sequential mixed methods design (QUAN → qual) [18] was employed in the present study. This design consisted of quantitative analysis and qualitative description to enrich and provide context for the quantitative findings. The guideline for reported mixed-methods studies [19] was followed (Supplementary file 1). A two-group, non-randomized, pre-and post-test quasi-experiment design was conducted in the quantitative strand (Fig. 1). Focus group interviews were conducted sequentially in the qualitative strand. Inform consent was signed by each of the participants following ethics approval.

Fig. 1
figure 1

Flow chart of the study. Note: Integrated, the non-immersive virtual simulation and high-fidelity face-to-face simulation integrated program arm

Full size image

Quantitative phase

Setting

This study was conducted in a simulation center in central China. The Bachelor of Science in Nursing program is a four-year synchronous lecture and clinical training program. Simulation has been embedded in most nursing curricula, including health assessment, fundamentals of nursing, and clinical nursing (i.e., medical-surgical nursing, pediatric nursing, intensive care nursing). Undergraduate nursing students take some lectures and take turns completing assigned clinical training in each academic semester. The four-week clinical training in ICU was appointed in the third year, and students take turns to complete the ICU training in Fall and Spring semesters.

The clinical training in ICU

The clinical training in ICU in the third year consisted of lectures, simulations, and clinical placement in each of the four weeks. During each week, students took a pre-test on Day 1, and a lecture was then delivered by a certificated simulation lecturer (JC). Students took turns to run face-to-face simulation scenarios on Day 2 led by a certificated simulation lecturer (JC), the students also spent three hours in the skill lab to practice related nursing skills, including but not limited to blood draw, venipuncture, and oral suction on Day 2. Students were then placed in ICU on Day 3 with an assigned RN and patients; a debriefing was conducted at the end of the day. Students were taking other courses on Day 4 and 5.

Participants

Two cohorts of third-year nursing students (Cohort 2013 and Cohort 2014) taking their four-week training in the ICU from September 2015 through June 2017 were invited to join this study. After acquiring informed consent, one hundred twenty-two students were divided into sixteen groups, with eight groups in each cohort (five to eight students in each group, supplementary Table 1). Students were subsequently assigned to the Integrated and Simulation arms (supplementary Table 1) according to the order in which they entered the ICU internship. Eight groups of students (n = 61) were assigned to the Integrated arm, and the other eight groups of students were in the Simulation arm (n = 61).

The procedure

Figure 2 illustrates the detailed procedure on the timeline in the quantitative phase. All students completed a pre-knowledge test of acute pulmonary embolism and self-assessment of clinical judgment ability on Day 1 in the third week of the ICU training before the lecture. Only students in the Integrated arm were additionally invited to interact with the vSim for Nursing scenario as a flipped-learning strategy after the pre-test on Day 1. The face-to-face simulation section as a standard component of the ICU training was provided to students in both the Integrated and Simulation arms on Day 2. After the face-to-face simulation on Day 2, a post-knowledge test of acute pulmonary embolism, self-assessment of clinical judgment ability, and evaluation of the simulation were conducted among all students.

Fig. 2
figure 2

Data collection procedure. Note: Integrated, the non-immersive virtual simulation and high-fidelity face-to-face simulation integrated program arm

Full size image

The vSim

Students in the Integrated arm were provided a user ID and password to access the vSim for Nursing scenario after the pre-test. The scenario, “acute pulmonary embolism,” from vSim for Nursing (Version 1, Wolters Kluwer, Philadelphia, PA, USA) was chosen for the students to complete during the third week of the ICU training. The vSim for Nursing scenario and simulation scenario focus on the initial identification of the signs and symptoms of acute pulmonary embolism post-surgery. The nursing process, including timely assessment, communication with physicians, drug administration, post-assessment, and health education, was also the critical point of this scenario. Students were encouraged to try as many times as they liked and were asked to provide a screenshot to show that they achieved a score of at least 90 out of 100 before the face-to-face simulation on Day 2. Students were told to refrain from discussing the vSim for Nursing nor simulation with each other out of the ICU internship.

The face-to-face simulation

Certificated simulation lecturers adopted one scenario (acute pulmonary embolism) from vSim for Nursing (Version 1, Wolters Kluwer, Philadelphia, PA, USA) into a simulation scenario following the standards of best practice for simulation [17]. The lecture team developed the other three face-to-face simulation scenarios following the same standards of best practice, including cases with septic shock, lung cancer, and trauma. Each of the four scenarios was delivered in the four weeks of the ICU training. The simulation scenario was repeated three times, each lasting for one hour, including 15 min of introduction and pre-discussion, 15 min of running the scenario, and 30 min of debriefing after the simulation following our previous protocols [9, 10]. Two or three students (5 to 8 students in each group, supplemental Table 1) took turns to immerse in the simulation scenario.

Measures

Clinical judgment ability

The primary outcome variable, students’ clinical judgment ability, was assessed by the 11-item Lasater Clinical Judgment Rubric (LCJR) [16]. The 11-item LCJR includes effective discovery processes (3 items), valid interpretation of the material (2 items), effective feedback was given (4 items), and effective reflection (2 items). The total score of the LCJR ranges from 11 to 44, with each item rating on a 4-point Likert scale; a higher LCJR score indicates higher self-reported clinical judgment ability. The Cronbach’s alpha of LCJR was 0.953 in the current study.

Simulation Design Scale (SDS)

There are 20 items and five domains in this Likert five-point scale, with 5 items in objectives/information, 4 in student support, 5 in problem-solving, 2 in guided reflection and feedback, and 2 in fidelity [20]. The total mean score of SDS ranges from 1 to 5, with each item ranging from 1 to 5 (1 = strongly disagree, 5 = strongly agree). All mean scores greater than 3.5 were considered relatively high for simulation design. The Cronbach’s alpha of the SCS was 0.902 in the current study.

Educational practices in simulation scale

There are 16 items and four domains in this Likert five-point scale, with 10 items in active learning, 2 in collaboration, 2 in diverse ways of learning, and 2 in high expectations [21]. The total mean score of this scale ranges from 1 to 5, with each item ranging from 1 to 5 (1 = strongly disagree, 5 = strongly agree). A mean score greater than 4 indicates a high perspective of education practice in simulation. The Cronbach’s alpha was 0.902 in the current study.

Student satisfaction and self-confidence in learning

There are 13 items and two domains in this Likert five-point scale, with 5 items in satisfaction and 8 in self-confidence in learning [21]. The total mean score of this scale ranges from 1 to 5, with each item ranging from 1 to 5 (1 = strongly disagree, 5 = strongly agree). A higher mean score indicates a higher level of satisfaction and self-confidence in learning. The Cronbach’s alpha was 0.889 in this study.

Sample size calculation

The enhanced clinical judgment ability measured by the LCJR was the primary outcome. The sample size for each arm was calculated for a power and a false discovery rate set at 0.8 and 0.05, respectively. The sample size calculations were performed using G*Power 3.1 [22, 23]. For the first hypothesis, we assumed that Simulation has a median effect (Cohen’s d = 0.5) [24] on improving clinical judgment ability among nursing students, thus a significant improvement in scores of self-reported clinical judgment ability from pre-test to post-test; thirty-four participants in the Stimulation arm were estimated to test the difference in this assumption. We also hypothesized that the Integrated arm improved more than students in the Simulation arm; fifty-one participants in each arm were estimated to test the difference in this assumption with an effect size of 0.50 (Cohen’s d) [24]. In summary, at least 102 subjects should be recruited (51 in each arm) to achieve a power of 0.8 at a significance level of 0.05.

Data analysis

IBM SPSS Statistics version 25 (IBM Inc., Armonk, NY, USA) was used for data analysis. The frequency, percentage, mean, and standard difference were used for the statistical description of pre-and post-test scores. Pre- and post-test LCJR scores in each arm were compared using a paired t-test to verify the effect of the Integrated program and Simulation on improving students’ clinical judgment ability. The difference between the pre-test LCJR scores, and the post-test LCJR scores from the Integrated and the Simulation arms was compared using an independent t-test to explore if students in the Integrated arm improved more on clinical judgment ability than students in the Simulation arm. The difference between the evaluation for the simulation study from the Integrated and the Simulation arms was compared using an independent t-test to investigate if the students in the two arms perceived differently of the face-to-face stimulation.

Qualitative phase

The qualitative phase used focus group interviews to collect students' data. The focus group aimed to a) explore novel findings that emerge in the analysis of the quantitative data, b) identify new areas of inquiry in addition to the quantitative assessments, and c) assess the validity of quantitative findings by presenting them for interpretation to study participants in qualitative interviews. Students in the Integrated arm were invited to join the focus group in the qualitative phase after the post-test in a private room. The interview protocol was grounded in the quantitative results from the first phase of the study (Supplementary file 2). To maintain a heightened level of awareness, a journal of personal feeling reflection, and contemplation was kept throughout the study. Eight focus group interviews were conducted, and each interview was audiotaped and transcribed verbatim by a research assistant. Semi-structured interview data analysis followed the process: (a) preparing the data for analysis, (b) conducting content analysis [18], and (c) transforming the textual data into numerical data if possible [25]. This process facilitated the integration of quantitative and qualitative data in a mixed-methods design.

Integrating the quantitative and qualitative findings

The results of the quantitative and qualitative phases were integrated while comparing the results of the entire study, as well as in the dissection of the study (Fig. 1 for a diagram of the explanatory sequential mixed methods design). Themes extracted from the qualitative phase were compared and contrasted with the subjective performance and clinical judgment measurements in the quantitative phase [18]. Similarity and differences across and with quantitative and qualitative data were presented in tables.

Results

Demographic information of the included students

Among the 122 students invited, 105 were females, and 17 were males, aged 19 to 24 (20.65 ± 0.832) years old. All the students invited agreed to participate, on one dropped out. There were no significant differences between the two groups regarding gender and age (p = 0.562).

Quantitative strand findings

The effect of integrated program and simulation on clinical judgment among nursing students

The scores of clinical judgment ability measured by the LCJR before and after the simulation were presented in Table 1. There was no significant difference regarding the pre-test scores acquired from students in the Integrated and Simulation arms. Both the Integrated and Simulation arms showed significant improvement after the simulation (all p < 0.01). Regarding the difference between the Integrated arm and the Simulation arm at post-test, scores in the Integrated arm were significantly higher in five items of two domains (all p < 0.05), including effective discovery process and valid interpretation of the material. No significant difference was observed in the other six items of the other two domains (effective feedback was given and effective reflection).

Table 1 Comparison of nursing students' scores in each item and domain of the Lasater Clinical Judgment Rubric (LCJR) between pre- and post-test in two arms (n = 122)
Full size table

The mean difference of scores between pre and post-test in each item and domain of the LCJR in the two arms were also compared (Table 2). Scores in the Integrated arm significantly improved more than those in the Simulation arm in seven of the 11 items, and three of the four domains of the LCJR, including effective discovery processes (t = 4.82, p < 0.05), valid interpretation of the material (t = 5.01, p < 0.05), effective reflection (t = 2.10, p < 0.05).

Table 2 Comparison of the mean difference of participants’ scores in each item and domain of the Lasater Clinical Judgment Rubric (LCJR) between pre and post-test in two arms (n = 122)
Full size table

Nursing students’ feedback on simulation design

Table 3 presents students’ feedback on the simulation in the two arms. All domains in the SDS were rated relatively high both in the Integrated and the Simulation arms, and the difference between these two arms was not significant. However, compared to the Simulation arm, the Integrated arm has higher mean scores in the domains of fidelity (4.09 (SD = 0.588) vs. 3.81 (SD = 0.881), p = 0.042).

Table 3 Students’ perspective of the simulation (n = 122)
Full size table

Mean scores of educational practice in simulation and each domain were greater than 4 in both arms. Collaboration was the highest, followed by diverse ways of learning, high expectations, and active learning. There was no significant difference between these two arms regarding mean scores of educational practice in simulation and each domain.

Mean scores in satisfaction and self-confidence of students were also greater than 4, with higher scores in the satisfaction domain than in self-confidence in both arms. Regarding scores in satisfaction and self-confidence of students in the two arms, the difference was not significant.

Qualitative strand findings

The findings of focus group interviews supported the improvement of clinical judgment and helped interpret quantitative data related to these improvements. The vSim for Nursing allowed students to reflect on their decisions in a safe and self-controlled learning environment. Themes and quotes consistent with the domains of the LJCR are listed below and in Table 4. Other advantages, inadequacies, and recommendations were also summarized (Table 4).

Table 4 Qualitative findings from focus group interviews in the Integrated arm (n = 61)
Full size table

Effect on improving clinical judgment

Effective discovery processes

Through step-by-step feedback and timely reminders in vSim practice, students can identify the critical learning points and important questions they must know.

“I think the tone and scene in vSim are authentic; it (vSim) can lead you to believe what to pay attention to when dealing with patients and their diseases. You do not know what to observe initially, but later you know what to discover because the feedback from vSim will lead you to step by step.

“vSim allows us to pay attention to the information not covered in the simulation, such as calculating the dosage of drugs and the time interval between drugs. We usually don't pay much attention to the doctor's orders when we do the simulation, so I think combining vSim and simulation is better than just doing a simulation”.

Valid interpretation of the material

Students believed they could explain and sort out the materials most relevant to patients and evaluate and rank the information when dealing with complex scenarios.

“vSim is a bit strict, but you can know the rules as you need to inquire about allergies before using heparin and the rules of checking, and then we'll apply this to the actual (drug administration) process. However, simulation can also be more flexible. For example, in vSim, when we evaluate pain, we will be with the patients in the private room when we ask questions here and there (stay with the patient). Still, in the actual process, it will be very chaotic (chaotic environment in clinical wards), so the two can be combined with and complementary”.

Effective feedback was given

Students could adjust the treatment regimen on time-based on the information collected by themselves and the feedback from patients and compare them with the knowledge and experience they acquired from previous learning to develop an intervention program.

“vSim can give effective feedback quickly. Let you know how to improve; there are many opportunities; there is no real tension; quickly master its process.”

“…..do vSim you feel like a combination of examination and game, there is a sense of painting, both image memory, but also deepen the body memory”.

Effective reflection

Students could benefit from the timely reflection provided by the vSim.

“There is no such thing as practicing on real people; we can repeat many times. Suppose there is a violation of nursing principles or some ideas. In that case, there will be reminders on it, as well as sources, that is, we can systematically understand the relevant knowledge and target points of a disease”.

Other advantages

The subjective initiative of nursing students was improved

The vSim enhanced students' learning interests.

“Read the book and then do vSim, and you will sometimes find that the knowledge is incomplete. When I get stuck in my mind, I will be more impressed if I turn to a book”.

The teaching form of the course is novel and flexible

The vSim allowed students to choose the time and place freely. The combination of vSim and simulation led students to think actively, stimulated their interest in learning, and made the boring skill training more vivid.

“I think vSim can save time because we can practice vSim in the dormitory; we do not need to gather in the classroom; sometimes it’s hard to find an available time slot for everyone in the group.”

“vSim provides us with such a platform. Without vSim, our main experience might come from clinical and classroom teaching. Now with vSim, we can practice alone without the teacher or in the clinic, which is a relatively easy experience”.

Inadequacies and recommendation

Students mentioned that there was no real communication and interaction during vSim practice.

“In the virtual simulation process, there is no communication with patients; there is no real interaction, can not respond naturally like people. Most of the time, as a reminder, the answer is always a state of emergency.”

Themes of recommendations to improve the design to combine vSim and simulation also emerged, such as improving the fidelity and creating more scenarios.

“The design of vSim itself needs to be closer to the clinical setting, to increase the auscultation, palpation and abnormal signs measurement function through computer technology.”

“Only one scenario may make you feel bored. I think more vSim scenarios should be developed.”

Mixed methods findings

The mixed-methods findings integrated quantitative and qualitative feedback of clinical judgment among nursing students (Table 5). The qualitative quotes provided a context for the quantitative improvement measured by the LJCR. Most of the quantitative findings were confirmed by qualitative findings, including the domains and items in the LJCR. The qualitative quotes expanded quantitative improvement in the domain of “valid interpretation of the material”.

Table 5 Integration of clinical judgment improvement from quantitative and qualitative data among nursing students in the Integrated arm (n = 61)
Full size table

Discussion

The feasibility and effect of a virtual simulation and simulation for improving nursing students’ self-reported clinical judgment ability were established in this mixed-methods study. The findings in the quantitative strand supported all three hypotheses by comparing and verifying the effect of a combination of virtual simulation and face-to-face simulation. The focus group interviews in the qualitative strand yielded different but complementary data, confirming findings in the quantitative strand and providing a context for the quantitative findings.

Improvement after simulation in both arms supported the first hypothesis that both the Integrated program and simulation could enhance nursing students’ self-reported clinical judgment ability (all p < 0.01, Table 1), which is in line with previous studies [10, 26,27,28]. Clinical judgment is the process by which the nurse gathers information, identifies vital information, identifies problems, sets improvement goals, plans and implements interventions, evaluates results, and learns to reflect on them. Participation in simulation requires nursing students to be involved in the process and employ their skills to solve the problems, confirmed by previous studies [9, 10].

Students in the Integrated arm improved more than students in the Simulation arm in scores of effective discovery processes, valid interpretation of the material, and effective reflection (all p < 0.05, Table 2), favored the Integrated program and verified our second hypothesis. Table 2 shows the average differences in LCJR scores between pre-and post-test among the two arms, demonstrating that students in the combination of vSim for Nursing and simulation arm reported significantly more improvement in clinical judgment ability than those in the simulation-only arm. These results were consistent with the interview results (see Table 5). Students mentioned in the interview that the combination of vSim for Nursing and simulation could improve communication and operation skills, achieve effective intervention, and improve self-confidence. Evidence showed that students' confidence levels improved with repeated simulation [29]. In addition, previous study shows that students tend to show tension, anxiety, and fear when being assessed and tested in a simulated environment due to the expectation of achieving near-perfect execution [8]. Repeated training in virtual simulation may relieve students’ stress before they immerse in the simulation. However, we did not record the times each student tried the vSim scenario as well as the final score of the vSim scenario, since students in the Integrated arm were provided ample opportunity to get familiar with the vSim scenario, an individual style of learning was also the benefit of the vSim.

Our qualitative findings provided context for the improvement of clinical judgment ability in the Integrated arm. The qualitative data supported the significant improvement in effective discovery processes in the Integrated arm found in the quantitative strand, consisting of focused observations, recognizing deviations, and information seeking (Table 5). Students in the interview session also mentioned that repeatedly conducting vSim for Nursing was like experiencing multiple simulations that enabled them to continuously summarize experiences and rules, supported by a previous study [30]. The vSim for Nursing plays a suggestive role in subsequent simulation, allowing students to collect targeted information, discover outliers, and accurately evaluate the implementation of practical measures, ultimately gradually translating knowledge into clinical thinking and clinical judgment [31].

Mixed-methods findings also supported our hypotheses. The significant improvement of “valid interpretation of the material” and “effective reflection,” especially the prioritizing data, was also mentioned in the interview (Table 5). The sequential evaluation and operation of vSim for Nursing include standardized diagnosis and treatment procedures, and the severity of each error is graded and marked, which enables students to think, distinguish between priorities in the processing process, pay attention to the principle of the problem, and to respond orderly and calmly. In the vSim for Nursing scenario, students interpreted the collected information for further treatment. Thus, virtual simulation helped students think, understand, and apply their previous knowledge [32]. The vSim for Nursing animation is vivid so that students feel different from the classroom teaching [33]. Through the immersive learning of vSim for Nursing, students deepen their multidimensional understanding [34]. Identical to previous findings, this study allowed students to perform virtual exercises repeatedly on-demand before simulation until they were satisfied with their performance [35].

Even though students in the two arms rated different scores in support and fidelity in simulation design, they all rated relatively high when considering their perspective of the simulation, which supported the acceptability of the simulation scenario adopted from the vSim for Nursing, and also supported our third hypothesis. Virtual simulation could be an alternative to improve the fidelity since virtual-reality simulation has the advantage of an entire virtual 3D Platform to present live-action training as closely as possible [31, 36]. Students admitted that the learning objectives were met, they were satisfied with the simulation, and they rated pretty high for simulation design, education practice in simulation, and self-confidence in learning. The vSim for Nursing coupling simulation education for ICU internship bolstered nursing students’ clinical judgment ability by facilitating the translation of knowledge to practice. Maximizing the opportunity for students to participate in clinical judgment that cannot be engaged in clinical practice and to experience clinical case management by engaging in virtual simulation may be an addition to classroom lectures to improve learning outcomes [37].

However, the post-simulation improvement of “effective feedback was given” between the Integrated and the Simulation was not significantly different (Table 2). The main reasons could be that the vSim for Nursing lacks a flexible communication environment, a design to develop students' hands-on ability and real-time observable changes in patients’ conditions. In addition, the vSim was independently practiced by students without a faculty member; the feedback was generated by the system and may have lacked depth. The other reason could be the short training time since the simulation only lasted 1 h.

Implications

The findings in the current study added a new dimension to nursing education's transition from the face-to-face paradigm to a learner-centered virtual simulation-based education. Although conducted before COVID-19 when in-person learning and face-to-face was the predominant teaching paradigm, this study was particularly important in the context of limited in-person contact during the COVID-19 pandemic and post-pandemic [38]. This study also provided suggestions on how virtual simulation strategies might be improved and what might be developed in future studies through focus interviews. Further studies could also create more virtual reality scenarios since only one scenario was employed in the current study. Only non-immersive virtual simulation was applied in the current; further studies may also develop semi-immersive or immersive virtual simulation and investigate the effect of this virtual simulation on healthcare education. This study only included nursing students taking clinical training in ICU; studies may also explore the effect of the integrated virtual simulation and face-to-face simulation program among nursing students receiving training in other clinical settings.

Limitations

Several limitations of this study need attention. First, the study period ranged from 2015 to 2017 though we controlled other variables, i.e., using the same teaching material, same instructor, and same settings of the simulation. We used convenience sampling instead of a randomized controlled design, and all students participated in the simulation, which may cover the effect of virtual simulation (vSim for Nursing). In addition, this study was conducted in one center. The clinical judgment ability was self-reported other than acquired by objective assessment, which may impact the findings, and the improvement does not guarantee improvement of patients’ outcomes. The improved clinical judgment ability was measured immediately after the simulation; the improvement may fade after long periods. We did not compare the two arms in terms of the scores on acute pulmonary embolism and the amount of time spent in the face-to-face simulation. A comparison of the score and the amount of time spent in the simulation may provide more information regarding the effect of the integrated virtual simulation and face-to-face simulation program. Further multicenter randomized controlled studies may recruit a larger sample to validate the current findings. Other studies may also consider adding an objective assessment of clinical judgment ability as well as including patients’ outcomes if possible.

Conclusions

This mixed-methods study comparatively verified the effect of a combination of virtual simulation and face-to-face simulation for enhancing nursing students’ self-reported clinical judgment ability. Quantitative data analysis supported the clinical judgment improvement, and qualitative data confirmed the quantitative findings. The integrated virtual simulation and face-to-face simulation program provides a forum to practice clinical judgment skills such as critical thinking, communication, and decision-making. Findings from this study may also benefit new graduate nurses in ICU since the scenario was developed for nursing internships in ICU.

Availability of data and materials

Deidentified data will be available upon reasonable request. Requests to access these datasets should be directed to jc22db@fsu.edu.

References

  1. Román-Cereto M, García-Mayor S, Kaknani-Uttumchandani S, García-Gámez M, León-Campos A, Fernández-Ordóñez E, et al. Cultural adaptation and validation of the Lasater Clinical Judgment Rubric in nursing students in Spain. Nurse Educ Today. 2018;64:71–8.

    Article  Google Scholar 

  2. Sommers CL. Measurement of critical thinking, clinical reasoning, and clinical judgment in culturally diverse nursing students – A literature review. Nurse Educ Pract. 2018;30:91–100.

    Article  Google Scholar 

  3. Nieto Fernández-Pacheco A, Castro Delgado R, Arcos González P, Navarro Fernández JL, Cerón Madrigal JJ, Juguera Rodriguez L, et al. Analysis of performance and stress caused by a simulation of a mass casualty incident. Nurse Educ Today. 2018;62:52–7.

    Article  Google Scholar 

  4. González-Gil MT, González-Blázquez C, Parro-Moreno AI, Pedraz-Marcos A, Palmar-Santos A, Otero-García L, et al. Nurses’ perceptions and demands regarding COVID-19 care delivery in critical care units and hospital emergency services. Intensive Crit Care Nurs. 2021;62: 102966.

    Article  Google Scholar 

  5. Stewart C. Understanding new nurses’ learning experiences in intensive care. Intensive Crit care Nurs. 2021;67: 103094.

    Article  Google Scholar 

  6. Yuan H Bin, Williams BA, Fang JB, Ye QH. A systematic review of selected evidence on improving knowledge and skills through high-fidelity simulation. Nurse Educ Today. 2012;32:294–8.

  7. Duff E, Miller L, Bruce J. Online Virtual Simulation and Diagnostic Reasoning: A Scoping Review. Clin Simul Nurs. 2016;12:377–84.

    Article  Google Scholar 

  8. Lavoie P, Clarke SP. Simulation in nursing education. Nursing (Lond). 2017;47:18–20.

    Article  Google Scholar 

  9. Chen J, Yang J, Hu F, Yu S-H, Yang B-X, Liu Q, et al. Standardised simulation-based emergency and intensive care nursing curriculum to improve nursing students’ performance during simulated resuscitation: A quasi-experimental study. Intensive Crit Care Nurs. 2018;46:51–6.

    Article  Google Scholar 

  10. Hu F, Yang J, Yang BX, Zhang F-J, Yu S-H, Liu Q, et al. The impact of simulation-based triage education on nursing students’ self-reported clinical reasoning ability: A quasi-experimental study. Nurse Educ Pract. 2021;50: 102949.

    Article  Google Scholar 

  11. Labrague LJ, McEnroe – Petitte DM, Fronda DC, Obeidat AA. Interprofessional simulation in undergraduate nursing program: An integrative review. Nurse Educ Today. 2018;67:46–55.

  12. Karlsen MMW, Gabrielsen AK, Falch AL, Stubberud DG. Intensive care nursing students’ perceptions of simulation for learning confirming communication skills: A descriptive qualitative study. Intensive Crit Care Nurs. 2017;42:97–104.

    Article  Google Scholar 

  13. Badowski D, Rossler KL, Reiland N. Exploring student perceptions of virtual simulation versus traditional clinical and manikin-based simulation. J Prof Nurs. 2021;37:683–9.

    Article  Google Scholar 

  14. Kalisch BJ, Aebersold M, McLaughlin M, Tschannen D, Lane S. An Intervention to Improve Nursing Teamwork Using Virtual Simulation. West J Nurs Res. 2015;37:164–79.

    Article  Google Scholar 

  15. Verkuyl M, Lapum JL, St-Amant O, Hughes M, Romaniuk D, McCulloch T. Exploring Debriefing Combinations After a Virtual Simulation. Clin Simul Nurs. 2020;40:36–42.

    Article  Google Scholar 

  16. Lasater K. Clinical judgment development : Using simulation to create an assessment rubric. J Nurs Educ. 2007;46:496–503.

    Article  Google Scholar 

  17. Sittner BJ, Aebersold ML, Paige JB, Graham LLM, Schram AP, Decker SI, et al. INACSL Standards of Best Practice for Simulation: Past, Present, and Future. Nurs Educ Perspect. 2015;36:294–8.

    Article  Google Scholar 

  18. Creswell JW, Plano Clark VI. Designing and Conducting Mixed Methods Research. Thousand Oaks: Third Edit. Sage Publications Sage CA; 2018.

    Google Scholar 

  19. O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. 2008;13:92–8.

    Article  Google Scholar 

  20. Jeffries PR. A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nurs Educ Perspect. 2005;26:96–103.

    Google Scholar 

  21. Jeffries PR. Simulation in nursing education: From conceptualization to evaluation. New York: National League for Nursing; 2007.

    Google Scholar 

  22. Faul F, Erdfelder E, Lang A, Buchner A. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39:175–91.

    Article  Google Scholar 

  23. Faul F, Erdfelder E, Buchner A, Lang A-G. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav Res Methods. 2009;41:1149–60.

  24. Cohen J. A power primer. Psychol Bull. 1992;112:155–9.

    Article  Google Scholar 

  25. Caracelli VJ, Greene JC. Data analysis strategies for mixed-method evaluation designs. Educ Eval Policy Anal. 1993;15:195–207.

    Article  Google Scholar 

  26. Wang AL, Fitzpatrick JJ, Petrini MA. Comparison of two simulation methods on chinese bsn students’ learning. Clin Simul Nurs. 2013;9:e207–12.

    Article  Google Scholar 

  27. Wang AL, Fitzpatrick JJ, Petrini MA. Use of Simulation among Chinese Nursing Students. Clin Simul Nurs. 2013;9:e311–7.

    Article  Google Scholar 

  28. Mohan D, Farris C, Fischhoff B, Rosengart MR, Angus DC, Yealy DM, et al. Efficacy of educational video game versus traditional educational apps at improving physician decision making in trauma triage: randomized controlled trial. BMJ. 2017;359: j5416.

    Article  Google Scholar 

  29. Cummings CL, Connelly LK. Can nursing students’ confidence levels increase with repeated simulation activities? Nurse Educ Today. 2016;36:419–21.

    Article  Google Scholar 

  30. Al Gharibi KA, Schmidt N, Arulappan J. Effect of Repeated simulation experience on perceived self-efficacy among undergraduate nursing students. Nurse Educ Today. 2021;106: 105057.

    Article  Google Scholar 

  31. Foronda CL, Shubeck K, Swoboda SM, Hudson KW, Budhathoki C, Sullivan N, et al. Impact of Virtual Simulation to Teach Concepts of Disaster Triage. Clin Simul Nurs. 2016;12:137–44.

    Article  Google Scholar 

  32. Tjoflåt I, Brandeggen TK, Strandberg ES, Dyrstad DN, Husebø SE. Norwegian nursing students’ evaluation of vSim® for Nursing. Adv Simul. 2018;3:10.

    Article  Google Scholar 

  33. Wright RR, Tinnon EA, Newton RH. Evaluation of vSim for Nursing in an Adult Health Nursing Course: A Multisite Pilot Study. Comput Informatics Nurs. 2018;36:84–9.

    Article  Google Scholar 

  34. Noh GO, Park MJ. Effectiveness of Incorporating Situation-Background-Assessment-Recommendation (SBAR) methods into simulation-based education for nursing students: A quasi-experimental study. Nurse Educ Today. 2022;109: 105252.

    Article  Google Scholar 

  35. Keys E, Luctkar-Flude M, Tyerman J, Sears K, Woo K. The Integration of Virtual Simulation Gaming Into Undergraduate Nursing Resuscitation Education: A Pilot Randomised Controlled Trial. Clin Simul Nurs. 2021;54:54–61.

    Article  Google Scholar 

  36. Mendez KJW, Piasecki RJ, Hudson K, Renda S, Mollenkopf N, Nettles BS, et al. Virtual and augmented reality: Implications for the future of nursing education. Nurse Educ Today. 2020;93: 104531.

    Article  Google Scholar 

  37. Verkuyl M, Hughes M, Tsui J, Betts L, St-Amant O, Lapum JL. Virtual Gaming Simulation in Nursing Education: A Focus Group Study. J Nurs Educ. 2017;56:274–80.

    Article  Google Scholar 

  38. Hu F, Ma J, Ding X-B, Li J, Zeng J, Xu D, et al. Nurses’ experiences of providing care to patients with COVID-19 in the ICU in Wuhan: a descriptive phenomenological research. BMJ Open. 2021;11: e045454.

    Article  Google Scholar 

Download references

Acknowledgements

We thank all the students who participated in the study. We also would like to thank the simulation center at Wuhan University School of Nursing which provided technical support of this study.

Funding

No.

Author information

Author notes

  1. Jian Yang and Wen Jie Zhou contributed equally and share the first authorship.

Authors and Affiliations

  1. Department of Thoracic Surgery, Zhongnan Hospital of Wuhan University, No. 169 Donghu Road, Wuchang District, Wuhan, 430071, Hubei, China

    Jian Yang & Xue Yu He

  2. Department of Cardiology, Renmin Hospital of Wuhan University, No. 238 Jiefang Road, Wuchang District, Wuhan, 430060, Hubei, China

    Wen Jie Zhou

  3. School of Nursing, Wuhan University, No. 115 Donghu Road, Wuchang District, Wuhan, 430071, Hubei, China

    Si Chen Zhou, Dan Luo, Qian Liu, Ai-Ling Wang, Si-Hong Yu & Bing Xiang Yang

  4. Hospital Quality and Safety Management Office, Zhongnan Hospital of Wuhan University, No. 169 Donghu Road, Wuchang District, Wuhan, 430071, Hubei, China

    Xiao-Ping Zhu

  5. Department of Critical Care Medicine, Zhongnan Hospital of Wuhan University, No. 169 Donghu Road, Wuchang District, Wuhan, 430071, Hubei, China

    Fen Hu

  6. Clinical Research Center of Hubei Critical Care Medicine, No. 169 Donghu Road, Wuchang District, Wuhan, 430071, China

    Fen Hu

  7. Center for Critical Care and Anesthesia Nursing Research, Wuhan University School of Nursing, No. 115 Donghu Road, Wuchang District, Wuhan, 430071, Hubei, China

    Fen Hu

  8. Florida State University College of Nursing, 98 Varsity Way, Tallahassee, FL, 32306, USA

    Jie Chen

Authors
  1. Jian Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen Jie Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Si Chen Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qian Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ai-Ling Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Si-Hong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xiao-Ping Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Xue Yu He
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Fen Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Bing Xiang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Jie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Study design: JY, WJZ, QL, DL, ALW, SHY, XPZ, XYH, FH, BXY, JCDate collection: JY, WJZ, DL, ALW, XPZ, FH, BXY, JCData analysis: JY, WJZ, SCZ, BXY, JCDraft of the manuscript: JY, WJZ, SCZ, XYH, FH, BXY, JCCritical review and final approval of the manuscript: All.

Corresponding authors

Correspondence to Xue Yu He, Fen Hu, Bing Xiang Yang or Jie Chen.

Ethics declarations

Ethics approval and consent to participate

The study protocol had been approved by the IRB in Wuhan University School of Medicine (NO.20140011). Informed consent was acquired from all the participants and their legal guardians. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Supplementary file 1. Good Reporting of A Mixed Methods Study (GRAMMS) checklist. Supplementary file 2. Focus group interview protocol.

Additional file 2:

Supplementary Table 1. Distribution of students in two arms (n=122).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, J., Zhou, W.J., Zhou, S.C. et al. Integrated virtual simulation and face-to-face simulation for clinical judgment training among undergraduate nursing students: a mixed-methods study. BMC Med Educ 24, 32 (2024). https://doi.org/10.1186/s12909-023-04988-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12909-023-04988-6

Keywords

BMC Medical Education

ISSN: 1472-6920

Contact us