Virtual reality simulation training in laparoscopic surgery – does it really matter, what simulator to use? Results of a cross-sectional study

Abstract

Background

Virtual reality simulation training plays a crucial role in modern surgical training, as it facilitates trainees to carry out surgical procedures or parts of it without the need for training “on the patient”. However, there are no data comparing different commercially available high-end virtual reality simulators.

Methods

Trainees of an international gastrointestinal surgery workshop practiced in different sequences on LaparoS® (VirtaMed), LapSim® (Surgical Science) and LapMentor III® (Simbionix) eight comparable exercises, training the same basic laparoscopic skills. Simulator based metrics were compared between an entrance and exit examination.

Results

All trainees significantly improved their basic laparoscopic skills performance, regardless of the sequence in which they used the three simulators. Median path length was initially 830 cm and 463 cm on the exit examination (p < 0.001), median time taken improved from 305 to 167 s (p < 0.001).

Conclusions

All Simulators trained efficiently the same basic surgery skills, regardless of the sequence or simulator used. Virtual reality simulation training, regardless of the simulator used, should be incorporated in all surgical training programs. To enhance comparability across different types of simulators, standardized outcome metrics should be implemented.

Background

Minimally invasive surgery (MIS) is considered standard of care in many surgical procedures nowadays and laparoscopic techniques have revolutionized surgery in numerous fields [1]. The rise of robotic-assisted techniques is currently accelerating the growth of minimally invasive surgery. Acquiring the skills needed to perform laparoscopic procedures safely can be challenging. Limited working hours, increased patient safety requirements, as well as increasing time pressures in the operating room, make it difficult to learn and teach MIS “on the job,” as has been Halsted’s apprentice-tutor model for nearly a century.

The efficacy of virtual reality simulation training (VRST) has been shown in multiple studies among other things reducing error rates and improving working speed [2,3,4,5]. VRST is important for early technical skills acquisition in laparoscopic surgery [6]. Several randomized controlled trials highlighted their growing importance as a safe, ethical, and comparable way to train basic surgical skills [7,8,9].

Currently, several virtual reality simulators (VRS) are available from various companies. These offer several training options and curricula including basic tasks (e.g., camera guidance, bimanual working, eye-hand coordination), advanced skills (e.g., suturing, knot tying) and complete surgical procedures (e.g., laparoscopic cholecystectomy, appendectomy). However, there is no generally accepted standard for VRST, although they train similar basic and procedural skills. So far, however, there is no evidence of a different training effect of the various virtual reality simulators.

The aim of this study was to evaluate the training of similar basic surgical skills on three different VRS and to determine whether the participants' performance was influenced by the training sequence and the simulators used.

Methods

The dataset of the presented study was obtained during the 38th annual Davos Course in 2021, an international surgical training course in Davos, Switzerland (www.davoscourse.ch). The six-day course offers a blended learning experience with theoretical parts and a strong emphasis on hands-on training (open, laparoscopic & robotic).

Three different virtual reality simulators (VRS) were used: LapSim® (Surgical Science Sweden AB, Gothenburg, Sweden), LAP Mentor III® (Simbionix, Tel Aviv, Israel), and LaparoS® (VirtaMed AG, Zurich, Switzerland). The companies provided the simulators without any financial benefit and without sponsoring of the Davos course. Employees from the companies were present during the training and experienced surgeons acted as instructors. The study protocol was developed independently of the simulator companies but was approved by them before the start of the study.

We obtained written informed consent from all study participants. Data collection was completely anonymized. Swiss Ethics Committee "Swissethics.ch" grants a general waiver for the use of purely anonymized data and an application with approval is not requested (Swiss Federal Act on “Research involving Human Beings 810.30”). All methods were carried out in accordance with relevant guidelines and regulations.

Study participants were randomly assigned into three groups. All participants trained in time slots of two hours each on three consecutive days with one of the available three different VR simulators. Each participant started their timeslot with an entry exam of three exercises, followed by two hours of free training. Participants then moved to the next simulator, where they repeated that sequence (entry exam, two hours of training). After having trained on each of the three simulators, participants then returned to all three simulators in the same sequence for one hour of training and repeated the initial assessment as an exit exam (Fig. 1). The exercises performed on each simulator are listed in Appendix 1 and aimed to train the following skills: bimanual coordination, eye-hand-coordination, safe application of technical devices (e.g. clips) and tissue dissection. This manuscript was written according to the STROCCS 2021 guideline [10].

Fig. 1

Schedule of all participants during their virtual reality simulation training

For each participant, the following simulator metrics were collected: the total path length of all instruments used, and time taken to complete the exercises. Continuous data are given as median (IQR) or mean (SD) as appropriate.

Statistical analyses

Participants’ baseline data by group were compared using the chi-square test or Kruskal–Wallis Rank Sum test, as appropriate. Entry and exit examinations were compared overall and groupwise using Welch’s t-test for total path length as well as total time used [11]. The association of total time used and total instruments’ path length for all entry and all exit examinations, grouped by simulators, was evaluated using scatterplots with locally estimated scatterplot smoothers with a confidence interval of 95% (LOESS smoothers) [12]. A two-sided p-Value of < 0.05 was considered statistically significant. All statistical analyses were performed using R studio, version 2022.12.0 (www.r-project.org, Vienna, Austria).

Results

Eighteen participants took part in the study. There were no significant differences in surgical and training experience among the three groups (Table 1).

Table 1 Baseline data of participants by groups

In total, participants performed 135 entry examinations and 127 exit examinations on the three different simulators. The median total path length on entry examination was 830, 25 cm (IQR 322.60–1823.57) and 462.96 cm (IQR 224.96–1063.33) on exit examination (p < 0.001). Median time taken was 305.40 s (IQR 168.70–481.42) and 166.80 s (IQR 98.92–291.56), respectively (p < 0.001). The improvements are visualized as Scatterplots in Fig. 2. Forty-four (32.6%) of all entry examinations were performed on the LAP Mentor III (SIM) simulator, 45 (33.3%) on the LapSim (SS) simulator and 46 (34.1%) on the LaparoS (VM) simulator. Forty-four (34.6%) exit examinations were performed on SIM, 40 (31.5%) on SS, and 43 (33.9%) on VM. In all groups, there was a significant reduction in time taken to complete the exit exercises compared to the entry examinations. Path lengths were significantly reduced in participants using LAP Mentor III and LapSim, but not in participants starting on the LaparoS. Table 2 shows the different exercises. The exercises are described in detail in Appendix 1 and in several publications [7, 13, 14]. Table 3 shows time and path lengths of the three different groups. Table 4 shows total time and total path length at entry and exit examinations by the different exercises conducted. There was a statistically significant improvement of time needed in all exercises except PVT1, a knot-tying exercise. Path lengths were significantly lower in the following exercises: Gallbladder resection, Lap Chole Task 2, Lap Chole Task 3, Task 9, and PVT2 (see Appendix 1, Table 4).

Fig. 2

Scatterplots of total path length and total time on Entry and Exit Exams with LOESS smoothers, grouped by Simulators, SIM = Simbionix LapMentor III®, SS = Surgical Science LapSim®, VM = VirtaMed LaparoS®

Table 2 Number of exercises performed on the three simulators

Table 3 Differences of the mean were analyzed by Welch’s t-test

Table 4 Total path lengths and total time used at entry and exit examination by exercises

Discussion

In this study, we assessed the effect of basic laparoscopic virtual reality simulation training (VRST) on three different commercially available virtual reality simulators. Participants significantly reduced the time needed to complete tasks, and reduced path length of basic skills exercises and simulated cholecystectomy. The overall improvement of path lengths did not reach statistical significance on the LaparoS®, possibly due to the specific complexity of the exercises given. When assessing the single exercises (Table 4), there is a clear improvement of path lengths in the “Gallbladder Resection” exercise and a clear trend in the “Grasping Bimanual” exercise. However, in the “Vascular Injury” exercise, the improvement is not statistically significant. Nevertheless, all participants improved their skills to some extent, regardless the sequence of VRST or simulator used. Simulation in surgical training is considered a well-established technique of honing the skills necessary to perform surgical procedures. To date, multiple studies have shown that simulation training and, more specifically, virtual reality training is superior compared to the traditional surgical apprenticeship-model [6, 7, 9, 15]. Several meta-analyses summarize up to 31 randomized controlled trials evaluating the efficacy of VRST [5, 6, 9, 15]. As one would expect, in most publications, VRST is superior to traditional surgical training and conventional box trainers. There are different modalities in which VRST is held, proficiency based progression (PBP) being considered to be the most effective [7,8,9]. This particular training method requires a surgical task to be characterized into its individual parts and a benchmarking must be done to clearly define optimal, suboptimal and erroneous performance [16]. Moreover, PBP requires maintenance of a close supervision and repeated formative feedback [17]. All this leads to a relevant amount of infrastructure and personnel in order to meet the requirements of PBP. In this study, we did not evaluate PBP due to limited time, but could demonstrate that similar basic laparoscopic skills can be trained on different VRS available. The shortage of specialists already affects the whole of medicine and in particular the surgical subjects, more so in rural areas than in the cities. This shortage affects the US and Europe alike and different surgical specialties are involved [18,19,20,21]. Maintaining surgical training will be a challenge, especially in smaller hospitals that are affected by these bottlenecks. To date, modern technology (e.g. VRS) still has not become an integral part of surgical training. In this study, we show that free training can indeed lead to relevant improvements in basic surgical tasks and in more complex exercises, regardless of the simulator used. The strength in this study is that similar skills were trained on the different simulators, such as bimanual tissue or object handling, safe application of clips and tissue dissection. One major advantage of VRST is that even in with little external guidance and formal structure, VRST provides the unique opportunity to improve one’s individual surgical skills by enabling deliberate practice from the very beginning of a surgical career.

There are several limitations to this study. First, although the simulators are structurally similar and contain similar exercises, the specific metrics cannot be compared. Therefore, we chose to use a “before-after-comparison”, in order to show the effects of VRST itself rather than trying to evaluate the efficacy of specific exercises. This has been done extensively in several other studies and publications. Secondly, we only used total time and total path length of all instruments as surrogate parameters for proficiency/accuracy. It is well known that a task can be performed quickly, but badly executed and meticulously, but in a longer period. The same is true for path length as a surrogate for efficacy of movements. However, both metrics are widely used and enable universal comparability of different simulators and exercises without having to rely on proprietary evaluation algorithms from the manufacturers themselves. Furthermore, implementing and maintaining a mandatory VRST, including structured feedback and expert guidance, involves significant costs that may be prohibitive, especially for smaller, non-academic institutions that train few trainees. A centralized structure with training centers open to surgical trainees from different institutions could be one way of addressing this issue. Additionally, a combined approach to simulation training by the use of VRS such as the ones used in this study and conventional simulators such as box-trainers (e.g. FLS Box) could be a promising way to enhance surgical training.

Conclusion

This study showed that VRST leads to significant improvement already in short periods of time and with less-than-ideal training modalities, regardless of the sequence in which simulators were used. All VRS trained efficiently the same basic surgery skills, regardless of the sequence or simulator used. This should encourage surgical educators and trainees alike to adopt VRST as an integral part of basic surgical skills training. VRST, regardless of the simulator used, should be incorporated in all surgical training programs. However, standardized and validated outcome metrics should be implemented to reliably measure proficiency and performance of trainees.