A study on the impact of open source metaverse immersive teaching method on emergency skills training for medical undergraduate students

Abstract

Background

In recent years, the traditional simulation-based medical teaching approach has faced challenges in meeting the requirements of practical emergency medicine education. This study utilized open-source tools and software to develop immersive panoramic videos using virtual reality technology for emergency medical teaching. It aims to investigate the efficacy of this novel teaching methodology. This transformation shifted the focus from physical simulation to virtual simulation in medical education, establishing a metaverse for emergency medical teaching.

Methods

In accordance with the curriculum guidelines, the instructors produced panoramic videos demonstrating procedures such as spinal injury management, humeral fracture with abdominal wall intestinal tube prolapse, head and chest composite injuries, cardiopulmonary resuscitation, and tracheal intubation. Using Unity software, a virtual training application for bronchoscopy was developed and integrated into the PICO4 VR all-in-one device to create a metaverse teaching environment. Fourth-year medical undergraduate students were allocated into either an experimental group (n = 26) or a control group (n = 30) based on student IDs. The experimental group received instruction through the metaverse immersive teaching method, while the control group followed the traditional simulation-based medical teaching approach. Both groups participated in theoretical and practical lessons as usual. Subsequently, all students underwent a four-station Objective Structured Clinical Examination (OSCE) to assess the effectiveness of the teaching methods based on their performance. Additionally, students in the experimental group provided subjective evaluations to assess their acceptance of the new teaching approach.

Results

Before the training commenced, there were no significant statistical differences in the first aid test scores between the experimental and control groups. Following the training, the experimental group outperformed the control group in the four-station OSCE examination, with all P-values being less than 0.05. The satisfaction rate among the experimental group regarding the new teaching method reached 88.46%, reflecting levels of satisfaction and extreme satisfaction.

Conclusion

The open-source metaverse immersive teaching method has demonstrated a positive impact on enhancing the emergency skills of medical undergraduate students, with a high level of acceptance among students. In comparison to traditional simulated medical teaching methods, this approach requires less time and space, incurring lower costs, and is deemed worthy of wider adoption.

Introduction

Medicine is inherently a practical field, necessitating exposure to numerous cases, procedures, and surgeries to cultivate proficient medical students. Recent years have witnessed an increase in Chinese patients’ awareness of safety and rights, leading to a reluctance on being attended to by medical students. This shift has constrained traditional apprenticeship-style teaching methods [1, 2]. Even after becoming resident physicians, medical students acknowledge the importance of reinforcing clinical operation training [3]. The unique nature of emergency medicine poses challenges in providing students with high-quality learning opportunities during rotations. Firstly, emergency medicine demands immediate attention; hence, teaching staff may lack the time or resources to instruct students on rescue and treatment procedures when emergency cases arise. Secondly, the unpredictable timing of emergencies and the characteristics of the shift system during rotations means students may not encounter cases requiring urgent intervention [4,5,6]. The growing emphasis on quality healthcare and skilled medical professionals underscores the importance of identifying more effective teaching methodologies to enhance the emergency skills of medical undergraduates [7]. Moreover, the escalating frequency of natural disasters and traffic accidents underscores the societal need for a proficient emergency response team [8].

While teaching methods such as smart classrooms and online training offer flexibility in terms of time and space, they have been deemed inadequate due to limited student engagement and shortcomings in practical skill development. Consequently, these methods have gradually been phased out from skill development initiatives [9,10,11]. Currently, simulated medical teaching is widely regarded as the most effective approach to address these challenges. By allowing students to learn and practice on simulated human models, this method compensates for the constraints of clinical learning in terms of time and space, enabling students to gain practical insights not readily available in clinical settings [12]. Previous studies have demonstrated that simulated medical teaching enhances the clinical skills of medical students [13,14,15]. However, the high cost associated with acquiring advanced simulation equipment poses a barrier to widespread adoption of this method, leading to resource shortages.

In terms of clinical training hardware for high simulation scenarios, high-end comprehensive simulation personnel are required. Currently, the commonly used Simman-3G in the market has exceeded $416,000, and the LLEAP system is also around $138,000. In addition to other supporting equipment such as trauma modules and case library data, a complete training room hardware equipment can approach or even exceed $700,000 [15, 16]. Simulation training requires at least three teachers, one SP, one operating software, and one explaining; Teachers must not only have rich clinical knowledge, but also possess considerable computer knowledge; Programs written by different teachers cannot be interchangeable with each other; A workshop teaching requires at least 3 experienced teachers to cooperate, and the number of participating students cannot exceed 10 [17, 18]. In recent years, the advancement of virtual reality technology has shown a promising trend in addressing the aforementioned challenges in simulated medical education [19]. Within a virtual reality setting, participants can fully immerse themselves in various activities. Immersive stereoscopic display technology, commonly referred to as extended reality (XR), encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), each offering distinct immersive experiences and varying levels of integration between real and virtual environments [20]. The emergence of the “metaverse” concept in 2021 has ushered in a new era for virtual medical education. The metaverse represents a virtual realm accessed through technological means, providing authentic sensory encounters. The adoption of artificial panoramic imaging, panoramic games, VR headsets, and glasses to facilitate immersive learning for students has gained traction since 2021 and is poised to become the prevailing approach in simulated medical instruction [21].

Employing professional panoramic filming equipment to produce medical videos, which are then viewed through VR glasses or helmets, serves to eliminate environmental distractions typically present when viewing conventional videos. The high-definition, three-dimensional attributes of these videos engender an immersive learning environment, enhancing the educational experience. Since the inception of the metaverse concept in 2021, the integration of virtual reality technology in medical education has garnered significant interest within the industry. Particularly during the COVID-19 pandemic, where physical distancing and infection control measures were imperative, traditional in-person teaching methods became unfeasible. Numerous medical institutions globally turned to virtual reality technology to train students, hastening its integration into medical education. Previous research indicates positive outcomes from such applications. Scholars have utilized this technology for dental implant training, with the system usability scale scoring 82.00 ± 10.79, indicating a high level of student acceptance [22]. In a separate study on oral medicine training, students in the VR learning group outperformed those in the traditional learning group in examinations [23]. However, existing research in virtual medical education primarily relies on software developed by technology firms, necessitating specialized equipment and facilities for teaching. This results in relatively high teaching costs and limits the scalability of these methods across various research centers.

Previous studies have predominantly concentrated on oral science, nursing, and anatomy, with limited exploration of emergency techniques. To address this gap, we propose a methodology involving the creation of emergency medical panoramic videos and virtual reality teaching software using cost-effective devices and open-source software, culminating in the establishment of a metaverse. This study seeks to evaluate the efficacy of this approach in teaching emergency medicine to undergraduate students and assess students’ experiences with VR all-in-one devices, thereby exploring innovative applications of virtual reality technology in medical education.

Methods and methodology

Following the guidelines outlined in the five-year undergraduate clinical medicine emergency medicine teaching curriculum in China, specific scenarios such as spinal injury management, composite injury treatment (e.g., humeral fracture combined with abdominal wall intestinal tube prolapse, head and chest composite injury), cardiopulmonary resuscitation, tracheal intubation, and bronchoscopic lung segment recognition were selected as training and assessment components. Scenario simulation cases were designed based on the Collection of Clinical Medical Scenario Simulation Teaching Plans [24]. A student assessment scoring table was developed using criteria from the Chinese Medical Student Clinical Skills Competition and the assessment standards for Chinese practicing physicians. Professional photographers utilized the Insta360 × 3 panoramic camera to record and capture panoramic videos. These videos were then exported to a computer using Insta360 Studio software and subsequently imported from the computer to the PICO4 VR all-in-one machine via a data cable (see Fig. 1). Collaboratively, teachers and engineers developed a virtual bronchoscopy operation APK using Unity 2021.03.9, which was then transferred to the PICO4 VR all-in-one machine through the 87VR assistant. In the experimental group, students were instructed by teachers to wear the PICO4 VR all-in-one machine and operate it using a VR controller to access the metaverse panoramic scene for standard viewing procedures. The instructor utilized the PICO app to establish video connectivity between the VR all-in-one machine and their phone or laptop, providing guidance to students on essential operational steps and precautions (see Fig. 2). Within the metaverse field, students could adjust their position, perspective, distance, and more using the controller, enabling training from all angles without any blind spots (see Fig. 3). The control group underwent standard computer (or mobile phone) video viewing and rubber simulated human operation training as part of their instructional sessions. The training duration for both groups was set at 8 h.

Fig. 1

Imaging technology used in this study

Fig. 2

Teachers instructing students in a real-world metaverse setting

Fig. 3

The educational environments viewed by students within the metaverse, facilitated by joystick manipulation and visual transformations

Research design

Selecting of fifth-year undergraduate clinical medicine students from the Third Clinical College of Guangzhou Medical University as the research participants was done on a voluntary basis, with prior informed consent obtained and signed consent forms. The students were divided into two groups: an experimental group of 26 students and a control group of 30 students, selected randomly by sorting their student numbers using a computer algorithm. Both groups had previously completed relevant medical theory courses and engaged in hospital internships. The average age of the students in the experimental group was 22.96 ± 0.53 years, with a pre-training emergency test score percentage of 71.19 ± 7.67 points. In comparison, the average age of the control group was 22. ± 0.50 years, with a pre-training emergency test score percentage of 74.28 ± 6.75 points. The P-value of the pre-training test scores between the two groups was calculated to be 0.134, which is greater than 0.05.

Assessment and evaluation

In order to assess students’ learning effectiveness accurately and objectively, this study employed the internationally recognized and highly objective Objective Structured Clinical Examination (OSCE) method [25]. The OSCE designed for this study comprises four stations, all employing a team collaboration scenario simulation approach, with an 8-minute time limit allocated for each station. The first station focuses on managing spinal injuries, the second station involves two cases of composite injury treatment, the third station covers cardiopulmonary resuscitation and tracheal intubation procedures, and the fourth station requires the identification of ten lung segments during bronchoscopy. Following the training, two groups of three students each will undergo an OSCE assessment in small groups. Each student will be evaluated based on the same operational criteria using simulated teaching aids. Through a random selection process, each student will perform the primary operation, initial aid, and secondary aid procedures, with separate scoring for each task. Each operation carries a maximum score of 100 points. It is imperative that instructors do not serve as examiners simultaneously. Both groups of students have received training on the OSCE exam before the official exam and have a thorough understanding of the exam methods. Moreover, the experimental group of students will also complete an internationally recognized virtual teaching method experience evaluation form [26], assessing the metaverse immersive teaching approach across nine dimensions: VR operation difficulty, stable panoramic video playback, video realism, deepening understanding of knowledge points, better control of learning time and location, learning interest, achieving teaching objectives, comfortable usage time, and overall satisfaction. Each dimension will be rated on a scale of 1 to 5, with 1 denoting very poor, 2 indicating unsatisfactory, 3 representing uncertain, 4 signifying good, and 5 reflecting very good. Additionally, to evaluate the impact of the new teaching method on team collaboration, an team collaboration scoring Table (24) was provided to the experimental group, assessing performance across eight dimensions: task division, understanding of one’s own limitations, constructive intervention, knowledge sharing, analysis and summary, closed-loop communication, clear information, and mutual respect. Each dimension will be rated on a scale of 1 to 5, with 1 indicating very poor, 2 denoting poor, 3 representing uncertain, 4 signifying good, and 5 reflecting very good.

Statistical analysis

The data was organized and analyzed using SPSS 22.0 statistical software, with quantitative data presented as (‾X ± s) for both student groups. The scores from the OSCE examination were subjected to an independent sample t-test, with a significance level of P < 0.05 indicating a statistically significant difference. The subjective evaluation scores of the experimental group students were analyzed through descriptive analysis.

Results

Comparison of Four Stop OSCE Skill Assessment Results between Two Student Groups. A total of 26 students in the experimental group and 30 students in the control group completed their studies and engaged in the OSCE assessment. The scores of the experimental group students at all four stations surpassed those of the control group students, with a statistically significant difference, as illustrated in Table 1.

Table 1 Results of skill assessment of two groups of students

We utilized the established and reputable “Student Satisfaction Table for Virtual Reality Technology Courses” [26] and “Team Collaboration Table” [24] for direct comparison. All 26 students in the experimental group completed the two forms. The overall satisfaction level for the virtual reality technology course was rated as good by 11 students, very good by 12 students, resulting in an overall satisfaction rate of 88.46%. Furthermore, the evaluation of the enhancement of team collaboration skills in this course exceeded 80%. Detailed results are presented in Tables 2 and 3.

Table 2 Student satisfaction for virtual reality technology courses

Table 3 Teamwork rating

Discussions

The shortage of professional teaching resources has long impeded the progress of medical education, but the emergence of virtual reality technology offers a novel solution to this issue. Previous studies have demonstrated that virtual teaching systems for genetic testing can swiftly elevate students’ awareness of biosafety, foster interest in experimental learning and skills, cultivate clinical experimental thinking, and enhance comprehensive abilities [27]. In the realm of neurosurgery education, findings indicate that 81% of trainees who underwent virtual reality technology training exhibited heightened interest in neurosurgery, with 47% acknowledging the potential influence of videos on their future professional choices [28]. In pharmacology instruction, the acceptance rate of virtual reality teaching has been as high as 90% [29]. Research into teaching single injuries in emergency medicine, such as cardiopulmonary resuscitation, has revealed that virtual reality technology outperforms traditional teaching methods in enhancing student performance [30]. Scholars have even developed virtual reality software for cricoid puncture using the Unity platform for teaching purposes. Results indicate that virtual reality technology can improve students’ injury speed (81%) and program steps (92%), although no significant difference was observed in the time required for circumcision surgery between the research group and the control group (p > 0.05) [31]. Our study aims to investigate the efficacy of the virtual reality metaverse in educating medical undergraduate students. During the instructional design phase, lesson plans were crafted, and virtual teaching software was developed based on the teaching outline to meet the educational needs of medical undergraduates. To comprehensively achieve the evaluation objectives, the assessment focused on multiple skills rather than individual competencies. Besides evaluating students’ skill mastery, it also assessed their teamwork capabilities. To ensure robust results, an internationally recognized objective structured site-based assessment method was employed. The outcomes indicated that the scores of the experimental group students surpassed those of the control group across four sites: spinal injury management, composite injury treatment, cardiopulmonary resuscitation and tracheal intubation, and bronchoscopic lung segment identification. These findings align with prior research outcomes, affirming the beneficial educational impact of virtual reality technology, particularly within the metaverse, in medical education.

To understand the level of acceptance of virtual reality technology among students, a satisfaction survey was designed. The assessment of machine usage indicators revealed a relatively high acceptance rate for VR operation difficulty and 3D video playback stability, standing at 80.77%. The satisfaction rate regarding video realism was noted at 76.92%, while comfort during usage scored relatively low at 69.23%, potentially linked to machine quality. Various subjective evaluations concerning teaching effectiveness indicated that 92.31% of students felt that the new method enhanced their comprehension of key concepts and met learning objectives. Additionally, 96.15% expressed increased interest in learning, and all students (100%) were content with the improved control over learning time and location. Overall, student satisfaction reached 88.46%, showcasing their positive reception of this innovative learning approach. The team collaboration ratings demonstrated that the new teaching method positively impacted student teamwork. The satisfaction rate for task division, the lowest-rated aspect, was still at 84.62%, while mutual respect, the highest-rated item, scored 96.15%. These findings suggest that our novel teaching methods enhance students’ teamwork skills and are well-received by them.

Educators must weigh the technical complexity and implementation costs when considering the efficacy of introducing a new technology. Simulated medical education has long been touted as an effective solution to the scarcity of teaching resources for medical students, a notion supported by various prior studies [15, 18, 32]. However, with the comprehensive Simman-3G simulator used for simulated teaching exceeding $400,000 and the LLEAP system also exceeding $100,000, the high cost of simulated medical education should also be taken into consideration. With the addition of other supporting equipment such as trauma modules and case library materials, a complete training room hardware equipment can reach or even exceed 500,000 US dollars. Although there are few reports on the cost of virtual reality teaching for medical students, the cost-effectiveness of virtual reality simulators and phacoemulsification simulation training in wet laboratories on operating room performance in similar patient training was estimated to increase the cost by 50% and to increase per capita cost by nearly $10,000 [33]. While carrying out this study, we learned that the price of commercially available virtual teaching videos and software range from $8,000 to $10,000 per piece, whereas the price of a complete set of head display controller hardware, software, and display system could add up to $150,000. In this study, we delve into the potential of employing an efficient, cost-effective, high-quality metaverse immersive teaching approach in emergency medical education. All video and software production for this research was undertaken by the researchers using freely available open-source software. The panoramic camera utilized in this study captured crisp videos at a superior high-definition level of 5.6 K, outshining the prevalent 4 K standard in most panoramic videos. Post-capture, the panoramic video could be edited, enhanced with color, and adorned with filter effects through the integrated free software during the export-import process to the all-in-one machine. Furthermore, the software boasts an AI automatic painting feature, significantly reducing the IT skill demands for medical personnel. Upon completion of the final panoramic video, students could fully immerse themselves in the metaverse environment, achieving a level of immersion unattainable by 3D videos. The stability of the all-in-one machine surpassed that of VR glasses, with no reports of dizziness or discomfort from any students. Equipped with a head display and operating handle, the all-in-one machine obviated the necessity for computer streaming. Within the metaverse, students could adjust their position and perspective through controller manipulation, mitigating the issue of obscured teacher operations faced by numerous students in traditional training setups. Virtual teaching transcended temporal and spatial constraints, enabling students to access training at their convenience. Throughout the training sessions, teacher-student interaction was facilitated via screen mirroring, facilitating explanations and error corrections. All hardware utilized in the study was purchased from the market, with a price of only $350 per all-in-one machine and $500 per camera, rendering it suitable for widespread adoption and implementation.

Due to financial constraints and other limitations, only three VR all-in-one machines were utilized in this study, allowing for training sessions with a maximum of three students at the same time. Consequently, the study’s sample size is modest. The ensuing evaluation involved a trio of students assessing the team’s treatment, precluding broader extrapolation to collaborative treatment scenarios involving more individuals. The VR all-in-one machine employed in this investigation belongs to a relatively economical brand, offering a comfort level of merely 69.23% for students during usage, with 11.54% (3/26) of students expressing reservations about the experience. Owing to the absence of somatosensors and mechanical algorithms in the open-source video and teaching software, students are restricted to visual perceptions within the metaverse, precluding tactile instructional outcomes. While these limitations are acknowledged within the study, they do not compromise the research findings and conclusions, representing necessary compromises to balance cost considerations with efficacy. Moving forward, our educational endeavors will concentrate on refining and promoting the open-source metaverse immersive teaching approach to enhance its efficacy in emergency medical education. In short, the open-source metaverse immersive teaching methodology emerges as a potent, cost-effective, and easily replicable pedagogical tool, bearing significant value for the training of medical students.