Man or Machine? Comparison of the Effect of Tutor-Guided versus Simulator-Guided Short-Time Bronchoscopy Training on Navigational Skills and Knowledge of the Tracheobronchial Anatomy

Background: Simulation based medical education is ecient for the acquisition of exible bronchoscopy navigational skills and the knowledge of the tracheobronchial anatomy. However, bronchoscopy simulator training is not routinely integrated into pneumologic fellowship programs or undergraduate medical education for time and/or cost reasons. Our study compares the effect of self-guided bronchoscopy simulator training versus tutor guided training on the acquisition of navigational skills and knowledge of the bronchial anatomy. Methods: 3rd -year undergraduate medical students were randomized to either a tutor- or simulator guided bronchoscopy simulator training focusing on the acquisition of navigational skills and the knowledge of the tracheobronchial anatomy. Every student performed a baseline bronchoscopy followed by a structured bronchoscopy simulator training and nally an assessment bronchoscopy at the end of the training program. Groups were compared by means of a repeated measurement ANOVA and effect sizes calculated as Cohens’ d.


Background
Flexible bronchoscopy (FB) is a safe procedure with low mortality and complication rates (1,2), but the historical "in patient bronchoscopy education" poses a risk for both the learner and the patient. It can cause procedural anxiety, inappropriate diagnostic or therapeutic results, postprocedural complications and reduced learning opportunities in practices with low case loads (3).
Simulation based medical education is well established for the acquisition of procedural skills and superior to the traditional Halstedian "see one -do one -teach one" approach (4). Simulation based bronchoscopy training improves patient outcomes (4) and procedure time (5). With respect to educational outcomes, Gopal et al. found a signi cant improvement of anatomical knowledge and bronchoscopy navigation skills in medical students after a self-directed bronchoscopy simulator training (BST) (3). Numerous studies showed, that novice bronchoscopists bene t from simulator training, but exact duration of training to achieve pro ciency is unclear (3). Ost et al. further found, that the skills acquired during BST transfer into real-patient bronchoscopy-performance (6). Consequently, the CHEST Expert Panel Report on Adult Bronchoscopy Training suggests, that simulation should be part of a structured bronchoscopy teaching curriculum (7).
However, BST is not yet widely implemented in medical education. One of the barriers to its use is surely the cost of the currently available high-delity simulators, ranging between 20000 to > 100000 US$.
Furthermore, because physician workload is generally high, it is di cult to provide time slots to integrate simulator training into existing curricula.
Physician supervisors however are commonly assumed to be required to provide simulation training, mostly because they provide expert feedback and because feedback is known to have a strong impact on learning (8).
However, the source of feedback seems to be less important than its presence (3). Technically, feedback could come from different sources, e.g. from a supervisor, colleague or even the simulator itself (9).
Several trials have examined the effect of unsupervised self-guided BST (3,5,10), compared to no training (11). For more complex surgical skills, additional mentoring seems to improve the learning outcome of robotic surgery simulation training, compared to self-guided training (12). For FB to the best of our knowledge, no study has yet compared the effect of the presence of a supervisor to non-supervised simulator training on prede ned learning outcomes. Our study thus aims to investigate whether simulator-guided bronchoscopy training can achieve similar acquisition of FB navigational skills and knowledge of the bronchial anatomy like tutor-guided training. If so, this could be a possibility to facilitate implementation of BST more widely and independent of supervisor availability.

Study type and objectives
The primary objective of this prospective randomized controlled study was to compare acquisition of FB navigational skills and knowledge of the tracheobronchial anatomy down to the level of bronchial segments in novice bronchoscopists after a simulator-guided training to novice bronchoscopists who completed the same BST program, but were additionally guided by an experienced bronchoscopist as a tutor.

Setting
The study took place between November 25th and December 10th 2019 in the "Center of Bronchoscopy" of the Department of Pulmonary Medicine, University Hospital, Inselspital Bern, Switzerland on a portable high-delity bronchoscopy simulator ("3d Systems/Simbionix BronchExpress®", 3D Systems -Simbionix, Airport City 70151, Israel).

Participants/Recruitment
3rd year undergraduate medical students of the University of Bern participating in a mandatory seminar on respiratory medicine were invited by E-Mail to participate in this study.

Ethics and Consent to Participate
Participation in the study was voluntary. Swiss research legislation (Human research ordinance HRO, § 2) deems studies such as ours, which do not collect health related individual data, exempt from full ethical review. Consequently, this study has not been submitted for review to the Ethics Committee for Research of the Cantone of Bern). Nevertheless, at the beginning of each training session, participants were instructed about the objectives and purpose of the study and provided written informed consent.

Study organization a) Randomization and Brie ng
Upon arrival on the study site, students were randomized to either the tutor-guided group or the simulatorguided group. Both groups received an introduction to the simulator and its operation by watching two short videos, followed by another 5-minutes in-person introduction of the simulator and the course of the simulator training. As the simulators' program uses English abbreviations for the anatomical structures, a ipchart with explanations of the abbreviations was placed visibly in the training room, to reduce the cognitive load for our German-speaking students.

b) Pre-Training Assessment Flexible Bronchoscopy
Before beginning of the training, each student performed a 5-minute bronchoscopy. Navigational skills (operationalized as percent of time in mid lumen; percent of time with scope-wall contact) and knowledge of the tracheobronchial anatomy (bronchial segments inspected/skipped; bronchial segments correctly identi ed) were recorded by the simulator.

c) Bronchoscopy Simulator Training
This assessment was followed by the training session. The simulator-guided group trained on the simulator by completing the curriculum given by the simulator. The tutor-guided group underwent the same curriculum but was supported by a tutor (AS), with a particular focus on navigational skills and orientation within the endobronchial system.
The training session consisted, rst, of an exercise for basic scope manipulation in a non-anatomic environment. This was followed by a training within an anatomic environment, consisting of 5 consecutive 3-minutes-sessions, during which all anatomical structures were labeled by the simulator.
Feedback was provided by the simulator in two ways. First, during the exercise for basic scope manipulation, students had to follow a ball with the scope in a metal tube system. The simulator played a metallic noise whenever the scope touched the wall. Second, by at the end of every session the simulator provided a table, listing the above mentioned outcome parameters (percent of time in mid lumen; percent of time with scope-wall contact, bronchial segments inspected/skipped, and bronchial segments correctly identi ed).

d) Post-Training Assessment
The training session was followed by a post-training-assessment, that was identical to the pre-trainingassessment.

e) Questionnaire
Additionally, we developed a custom questionnaire assessing baseline characteristics, preexisting bronchoscopy experience, anatomical knowledge and feedback to the study session. A translated version of the original questionnaire is available as supplemental information.

Data management and analysis
The simulator recorded all relevant performance data from pre-, and post-assessment as well as during training. Data were exported into an excel le and merged with student questionnaire data.
Student characteristics are reported as numbers, frequencies, percentages and mean or median and standard deviation or IQR, as appropriate.
Groups were compared by means of a repeated measurement ANOVA with type of training as the between subject factor and time of measurement (i.e. pre-or post-assessment) as within subject factor. A p-level of < 0.05 was set as signi cant. Effect sizes were calculated as Cohens' d.

Student Characteristics
Out of 140 eligible students, 54 students participated. Their baseline characteristics are shown in Table 1.

Navigational Skills
There were no differences in navigational skills between groups before start of the training (table 2). After training, both groups had signi cantly less time in mid-lumen in post-assessment compared to preassessment (p = 0.04) and signi cantly more scope wall contacts (p = < 0.001); effects were of medium (Cohens' d = 0.52) and large size, respectively (d = 0.99). We did not observe a signi cant effect of type of training on acquisition of navigational skills (table 2).  1 bold for comparison between pre-and post-assessment, normal for interaction between groups and time of measurement.
2 bold for comparison between pre-and post-assessment, normal for difference in gain/loss between groups.

Knowledge of Tracheobronchial Anatomy
There were no differences in knowledge of tracheobronchial anatomy between groups before start of the training (table 2 and Fig. 1). After training, both groups correctly identi ed considerably more and skipped considerably less bronchial segments in post-assessment compared to pre-assessment (see table 2

Questionnaire
All 28 participants in the tutor-guided group valued the instruction by the tutor as helpful (n = 17 (60.7%) or very helpful (11 (39.3)) on a Likert scale.

Discussion
Our study found signi cant and large improvements in knowledge of the tracheobronchial anatomy after a short bronchoscopic simulator training. We did not observe any signi cant differences in this effect between modes of instructions, i.e. between simulator guided and tutor guided groups. Gains in anatomical performance were accompanied by signi cant and relevant drops in handling of the bronchoscope. Trainees spend less time in the mid lumen and had substantially more wall contacts. Again, this effect was independent of the mode of instruction.
There are at least two potential reasons why we did not nd signi cant differences between modes of instruction. There either simply may be no relevant difference between the two groups, or our sample size may be too small to detect a difference of relevance. Indeed, the tutor guided groups shows larger gains between pre-and post-assessment than the simulator guided group, with medium effect sizes ranging between 0.46 and 0.63. The effects by the training per se however are at least 3.5fold larger than those of the mode of instruction. This, together with the fact that in clinical practice providing a tutor requires much more resources, may justify the use of simulator-guided instruction only.
The slightly larger gain in anatomical knowledge in the tutor-guided group comes at the expense of slightly worse navigational skills. Students in the tutor-guided group had slightly more scope wall contacts in post-assessment than those in the simulator-guided group (d = 0.37, a medium effect size). However, again, the effect of training per se is larger than the effect of the mode of instruction. In contrast to other studies (5,10), that showed an improvement of navigational skills after a bronchoscopy simulator self-training, our results show the opposite. One possible reason is the short training time in our trial. Another aspect might be, that the students concentrated more on the identi cation of the bronchial segments to the disadvantage of the scope manipulation.
Training time in published bronchoscopy simulation studies range from a single 1-hour-session to 10 repetitive training sessions (13). For example, participants of an 8-hour simulator training session improved their navigational skills and missed fewer segments in the post-training assessment (10). Because physician and student workload is generally high, it is di cult to provide time slots for regular simulator training within existing curricula. Our study could achieve improvement of anatomical knowledge and accuracy of tracheobronchial inspection after only a 1-hour-simulator training, admittedly at the expense of navigational precision. An implication of our nding might be, that shorter training sessions should focus on only one or few objectives, e.g. either scope manipulation or identi cation of the anatomical structures.
Another point to discuss is the fact, that body posture, body language and movements, and the correct technique of advancing the scope without distortion cannot be observed by a simulator without a supervisor. Nevertheless these are important aspects of FB, that are frequently not intuitive for novice bronchoscopists and an accurate scope handling technique is mandatory for the performance of a high quality FB. Furthermore, 84.6% of our participants of the simulator-guided group would have appreciated the presence of an instructor for provision of guidance in handling of the scope. There may be a solution for that problem, as Collela et al. investigated a motion analysis system, that is able to provide automated feedback on correct movements during self-directed training on simulators. The authors commented that this approach opens the opportunity for trainees in bronchoscopy to receive automated feedback on their scope handling during training, without the need of an instructor being present (14).
Either way, feedback is frequently rated as one of the cornerstones of medical education (15), whereas the source of feedback seems to be less important (3). In our study, the SIM group received feedback solely from the simulator, the TUT group from the simulator and the supervisor. Although not signi cant, but the TUT group had a slightly better performance in identifying the endobronchial segments. That is in line with Lee et al, who demonstrated that a trainee group with additional personal mentoring in robotic surgery training achieved signi cant better learning outcomes compared to self-directed, mentor-free learning (12). This result is re ected in our nding, that 22 (84.6%) of the participants in the simulator guided group would have preferred an additional person-to-person mentoring.
However, this approach is again con icting with limited availability of supervisors due to their workload and thus shortage of training time (12). Being aware that additional supervision can enhance the learning experience and outcome, several studies, including ours, have shown a certain e ciency of self-directed simulator training compared to no training (16). Thus, one can argue, that, in place of not offering simulator training due to lack of a tutor, self-directed training can be a feasible compromise.
Earlier trials have frequently used the Bronchoscopy Skills and Tasks Assessment Tool (BSTAT) to evaluate pro ciency in FB (3,17). We intentionally decided to use the built-in assessment tool of the simulator, to again avoid the dependency of the presence of a human supervisor/assessor.
Our study has several limitations. First, the duration of the training session was short, with approximately 1 hour. However, this conforms to reality where time for simulator training is limited. Second, our study consisted of only one single training session with immediate assessment afterwards and did not assess long-term retention. However, we primarily aimed to assess the presence and extent of a difference in learning between the tutor-guided and the simulator-guided group. Additionally, simulator training cannot completely replace clinical practice in real patients. When basic pro ciency in FB is achieved through simulator training, parallel performance of "real bronchoscopy" will take place. This would interfere with any results of "long-term-skills-retainment-assessment of simulator training between the two groups. Furthermore it is known, that the learning curve is most effective in novices and then a certain plateau will be reached (5,18,19). Thus, it can be assumed that in case of a difference between simulator and tutor guided simulation training, it would most likely be apparent in the early training stages.
Third, in real life, all bronchial segments have to be investigated and procedural performance is more likely to be re ected by the time needed for this procedure than by the number of segments skipped. However, we intentionally limited the time available for the examination to 5 minutes per assessment, so that the outcomes of interest could be measured on a single scale (i.e. number of segments identi ed or skipped) rather than having to adjust for two different scales (i.e. number of segments and time for the procedure). Earlier trials indeed found a reduction in procedure time after BST (5). A shorter procedure time may result in better patient comfort (20), and less complications (5,21). However, despite the advantages of a short procedure time, we argue that the patient bene ts even more from a cautious and atraumatic scope manipulation and a precise and complete inspection of the bronchial system.
Despite these limitations and the necessity of further research, in times of high physician workload, scarcity of time, money and availability of educators, our study is an important step on the way to determine the ideal teaching approach for basic FB.

Conclusion
Short time BST is effective in improving the performance of FB tracheobronchial inspection and can thereby enhance the knowledge of the tracheobronchial anatomy. This effect is largely independent of the presence of a human supervisor. Instead, feedback, the most powerful tool for enhancement of learning, is provided by the simulator. Our ndings justify the implementation of short-time (1 hour) selfguided BST into medical education, whereas shorter training sessions should focus on only one or few teaching objectives. Sparing a supervisor and thereby human resources may facilitate the implementation of BST into existing curricula. Further studies are necessary to determine the optimal teaching modalities for the acquisition of basic FB skills.  Figure 1 Effect of training on number of bronchial segments correctly identi ed by group. All signi cant differences marked *.