- Research article
- Open Access
- Open Peer Review
High-fidelity simulation self-training enables novice bronchoscopists to acquire basic bronchoscopy skills comparable to their moderately and highly experienced counterparts
BMC Medical Education volume 18, Article number: 191 (2018)
We sought to determine whether a self-training program on a high-fidelity flexible bronchoscopy (FB) simulator would allow residents who were novices in bronchoscopy to acquire competencies similar to those of experienced bronchoscopists as concerns the visualization of the bronchial tree and the identification of its anatomical elements.
We performed a prospective cohort study, categorizing bronchoscopists into three groups according to their experience level: novice (Group A, no FBs performed, n = 8), moderate (Group B, 30 ≤ FBs performed ≤200, n = 17) or high (Group C, > 200 FBs performed, n = 9). All were initially evaluated on their ability to perform on a high-fidelity FB simulator a complete visualization/identification of the bronchial tree in the least amount of time possible. The residents in Group A then completed a simulation-based self-training program and underwent a final evaluation thereafter.
The median total procedure time for Group A fell from 561 s (IQR = 134) in the initial evaluation to 216 s (IQR = 257) in the final evaluation (P = 0.002). The visualization and identification scores for Group A also improved significantly in the final evaluation. Resultantly, the overall performance score for Group A climbed from 5.9% (IQR = 5.1) before self-training to 25.5% (IQR = 26.3) after (P = 0.002), thus becoming comparable to the overall performance scores of Group B (25.3%, IQR = 13.8) and Group C (22.2%, IQR = 5.5).
Novice bronchoscopists who self-train on a high-fidelity simulator acquire basic competencies similar to those of moderately or even highly experienced bronchoscopists. High-fidelity simulation should be rapidly integrated within the learning curriculum and replace traditional, in-patient learning methods.
Flexible bronchoscopy (FB) is a safe procedure, with rates of 0.05 and 0.5% for attributable mortality and major complications respectively [1,2,3]. Although FB has a low rate of minor complications , the tolerability of the procedure for the patient is mediocre , especially for prolonged procedures . Considering procedure tolerability, many patients report anxiety for FB, but may be reassured by the bronchoscopist’s experience . In contrast, the risk of complications may be significantly higher when FB is performed by an inexperienced bronchoscopist .
Simulation-based FB training enables learners to hone their skills in a safe training environment, and become at ease with the procedure before moving on to patients. Thus, simulation training should be able to contribute improving the tolerability and safety of FB procedures performed by new practitioners. Several methods of bronchoscopy simulation have been developed and studied. These include animal models [7,8,9], low-fidelity trainers [10,11,12,13,14,15], and, since the early 2000s, high-fidelity virtual-reality simulators [16,17,18,19,20]. In a meta-analysis published in 2013, bronchoscopy simulation provided clear benefits as concerned practitioner skills and behaviors and moderate benefits as concerned procedure length . However, many of the studies included in that meta-analysis were performed in the settings of otolaryngology or anesthesiology. Bronchoscopy in those fields is relatively simple; it does not involve detailed visualization of the bronchial tree as is the case in pulmonology.
In 2015, a panel of American experts published propositions for the development of lung bronchoscopy training programs . They suggested that high-fidelity simulation should be available to all learners. Despite those recommendations, FB training continues to be largely based on the supervised “see one, do one, teach one” model . Today however, it seems difficult to justify the subjection of patients to examinations performed by learners who are still in the cognitive or integrative phase of their training when there are simulators developed specifically for FB .
For the work presented here, we sought to determine whether a high-fidelity FB simulation self-training program would allow novice bronchoscopists to acquire competencies similar to those of trained bronchoscopists as concerns the visualization of the bronchial tree and the identification of its anatomical elements.
Study type and objectives
The primary objective of this prospective cohort study, carried out between October 2015 and September 2016, was to assess FB skill acquisition in novice bronchoscopists who self-trained on a high-fidelity bronchoscopy simulator (Simbionix Bronch Mentor), and compare their skills to those of moderately and highly experienced bronchoscopists (Additional file 1: Table S1). The capacity of this simulator to discriminate the competency (dexterity) and anatomical knowledge of trainees as a function of their past experience was demonstrated in a 2014 validation study .
The secondary objectives were to determine the ability of the simulator to discriminate the skill levels of bronchoscopists as a function of their past experience and to evaluate student/user satisfaction with the simulator.
All residents and pulmonologists included in the study practiced in the Auvergne Rhône-Alpes region of France. Participants were categorized according to their experience in bronchoscopy as commonly classified in the literature:
Group A (novices, no experience): This group comprised first semester pulmonology residents. Eligibility criteria for this cohort were no former experience with FB (in patients or in simulation) and commitment to completing the entire training program and undergoing the initial and final evaluations.
Group B (moderate experience): This group comprised residents and recently licensed pulmonologists who had themselves performed at least 30 FBs but less than 200 FBs in patients. Indeed, Wahidi et al. showed that pulmonary fellows learning of FB displayed a step upward curve in the 30 first bronchoscopies . None had never received high-fidelity simulator training.
Group C (extensive experience): This group comprised pulmonologists who had performed more than 200 FBs during their career - a commonly admitted threshold [16, 26]. Again, none had never received high-fidelity simulator training.
Groups A and B were composed largely of residents seeking certification in pulmonology and following the Harmonising Education in Respiratory Medicine for European Specialists (HERMES) program of the European Respiratory Society (ERS). HERMES comprises a specific module for bronchoscopy .
All three groups received an initial briefing to present the simulator and its operation. Group A benefited additionally from a short presentation on endobronchial anatomy and a special play-time dedicated to their first handle of an endoscope (10 min). Thereafter, all groups were given a 5 min “play time” to familiarize themselves with the simulator (Fig. 1).
Then, all of the groups were subjected to an initial evaluation (ev-A, ev-B and ev-C). Groups B and C underwent no further evaluations. Subsequently, the members of Group A were invited to use the simulator autonomously for as long as they wished, over one or several sessions (self-training). When they felt ready, the members of Group A requested a new evaluation (ev-A-bis) with the evaluator. Ev-A-bis was strictly identical to ev-A.
The ability of each participant to perform a complete visualization of the bronchial tree, up to segmental bronchi, in as little time as possible was evaluated using the “Essential Bronchoscopy” module programmed within the simulator. The participants were required to visualize all anatomical structures and identifying them orally. They could use common anatomical names or Boyden classifications  according to their personal preferences. The exercise included the passage through the vocal cords. The evaluator noted participant performance on a dedicated form:
The anatomical structures visualized during the procedure were marked each with a 1 and summed to establish a visualization (V) score from 0 to 28 points.
The structures correctly identified were marked each with 1 and summed to establish an identification (I) score from 0 to 28 points.
The sum of the two preceding scores provided the total (T) score (T = V + I), which ranged from 0 to 56.
At the end of the exercise, the evaluator noted the other performance variables provided by the simulator:
Total procedure time, which was set at a maximum of 15 min;
Percentage of time with scope-wall contact and the percentage of time with the scope at mid-lumen, both used to evaluate endoscope dexterity.
An overall performance (P) score, similar to that used in the seminal study by Ost et al. , was defined as the percentage of the number of segments correctly visualized and identified divided by the procedure time.
The evaluations were conducted by two chest physicians who mutually developed a process to ensure objective and reproducible assessments.
After their evaluations, the participants were asked to respond to a questionnaire assessing their satisfaction with the simulator. The eight assessed items were rated from 0 (ineffective) to 6 (excellent) and summed to determine a global satisfaction score ranging from 0 to 48.
The study was approved by the local ethic committee of Lyon University hospital on 22/09/2015. Signed consent was obtained from all participants before the first evaluation in line with French law (non-interventional study). Data collected were strictly anonymous.
Categorical variables were expressed as percentages and continuous variables as medians and interquartile ranges (IQR). The Kruskal-Wallis test was used to compare distributions of continuous variables across more than two groups and the Mann-Whitney U test for comparisons between two groups. Statistical significance was set at p < 0.05. Statistical analyses were performed using SPSS version 19.0 (IBM Corp. Armonk, NY, USA).
Thirty-four participants were enrolled during the study period, eight in Group A (novice) 17 in Group B (moderately experienced) and nine in Group C (highly experienced). Their characteristics are summarized in Table 1. We initially invited 22 novice bronchoscopists (Group A) to participate the initial evaluation; but only eight completed the full self-training sessions and the final evaluation. Group A was thus limited to those eight participants. We found no significant difference in the overall performance score between participants who dropped-out the program and those who complete it (7.99 [IQR1.9] vs. 5.93 [IQR 5.1]; p = 0.290). During the self-training period, seven of the Group A participants performed 20 or more FB simulations whereas one performed only 5.
No significant differences were observed between the initial evaluations of Group B (ev-B) and Group C (ev-C) (Table 2). The median overall performance scores for ev-B and ev-C were respectively 25.3% (IQR = 13.8) and 22.2% (IQR = 5.5) (p = 0.5). There were no significant differences among the other ev-B and ev-C variables, (visualization score, identification score, scope-wall contact time, scope at mid-lumen time).
The initial evaluation of Group A (ev-A) did show several significantly lower aptitudes among the novices compared to the moderately and highly experienced bronchoscopists. The identification score was significantly lower in ev-A compared to ev-B and ev-C (p = 0.001), as was the median overall performance score: 5.9% (IQR = 5.1) in ev-A compared to 25.3% (IQR = 13.8) in ev-B and 22.2%, IQR = 5.6) in ev-C (p = 0.0001). Total procedure time was also significantly longer for the novices, i.e. 561 s in ev-A vs 198 s in ev-B and 210 s in ev-C (p = 0.0001). In contrast, the visualization score, scope-wall contact time and scope at mid-lumen time did not differ significantly between the three groups for the initial evaluations (Table 2).
Group a performance progression
The final evaluation of Group A (ev-A-bis), performed after the period of simulator self-training, showed significant improvements for most variables in comparison to the results of ev-A (Table 3 and Additional file 2: Figures S1-S3). The median total procedure time fell from 561 s (IQR = 134) in ev-A to 216 s (IQR = 257) in ev-A-bis (p = 0.002) (Fig. 2). The visualization and identification scores also improved significantly from ev-A to ev-A-bis. Resultantly, the overall performance score climbed from 5.9% (IQR = 5.1) in ev-A to 25.5% (IQR = 26.3) in ev-A-bis (p = 0.002), i.e. a progression of 435%.
Comparison of group A-bis (post simulator self-training) to groups B and C
Group A, after their simulator self-training (Group A-bis), correctly visualized significantly more anatomical structures (median 28 [IQR = 0] in ev-A-bis) than did Group B (median 27 [IQR = 1] in ev-B) (p = 0.017) and correctly identified significantly more structures (median 27 [IQR = 4] in ev-A-bis) than did Group C (median 20 [IQR = 9] in ev-C) (p = 0.02). There were no other statistically significant differences between Group A-bis and Groups B and C (Table 4). After the simulator self-training, the median overall performance scores also became comparable, i.e. 25.5% (IQR = 26.3) in ev-A-bis vs 25.3% (IQR = 13.8) in ev-B (p = 0.954) and vs 22.2% (IQR = 5.52) in ev-C (p = 0.564).
The median global satisfaction scores were 44 (IQR = 4) for Group A-bis, 38 (IQR = 8) for Group B, and 35 (IQR = 9) for Group C (p = 0.02) (maximum possible score = 48; Additional file 2: Figure S4). The results for the individual items in the satisfaction survey are provided in Additional file 2: Figure S5. The simulator’s contribution to future practice received a median score of 6 from Group A and 5 from Groups B and C (p = 0.009). Group A gave a median score of 5 for the simulator’s potential to reduce anxiety during future FB procedures; Groups B and C gave median scores of 4 and 2 for this item respectively (p = 0.009).
In the present study, we showed the interest of a FB self-training program employing a high-fidelity simulator: in one to three half-day training sessions, residents who were novices in bronchoscopy significantly improved their knowledge of endobronchial anatomy and their basic capacities for FB (visualization and identification of bronchial elements, procedure time). In our study, the basic capacities of the novices who followed the program became comparable to those of moderately (30–200 FBs) and even highly (> 200 FBs) experienced bronchoscopists.
The results of our work support those of Colt et al. , who tested a high-fidelity simulator FB training program with self-training. They too found significant post-training improvements in FB skills, including a reduction in the number of non-visualized segments (4.4 before training vs 0.8 after; p = 0.029). As also reported by Moorthy et al  and Colt et al , the performance scores of our novices after their training program (Group A-bis) became statistically comparable to those of our highly experienced bronchoscopists (Group C). .
In the initial evaluation, we could not discriminate skill levels using the visualization score. This is partially explained by the standardized examination conditions which do not entirely reflect the difficulties encountered in real FBs. We do however underline the statistically significant progression for the visualization score that we observed in Group A after the self-training. Thus, this aspect of performance constitutes a reachable learning objective for novice bronchoscopists.
The reduction in total procedure time observed after the simulation self-training is remarkable (median procedure time reduced by more than 50%). Such a reduction could result in more patient comfort   and less complications  when newly trained bronchoscopists begin performing procedures in real patients.
The participants in all three groups expressed a high level of satisfaction with the simulation-based training process (Kirkpatrick level 1 evaluation ), suggesting good compliance for possible training programs to come. Similar observations were reported in the validation study for the Simbionix Bronch Mentor simulator .
One main limitation of all high-fidelity simulators is the cost of the equipment, maintenance and housing. Several ways can however limit this issue such as use low-fidelity simulators , or mutualize a high-fidelity simulator in a large area. In Lyon area, we chose to buy one simulator for 4 universities.
Out of the 22 initially invited novice bronchoscopists, only eight completed the self-training sessions and the final evaluation. Group A was thus limited to those eight participants. Several factors play a role in this loss of Group A participants but were unfortunately not fully assessed here. First, many of these latter were first-semester residents who could not find sufficient time to self-train on the simulator due to their hospital activities. Also, a certain number of those participants lived and/or practiced far away from the simulation center (> 2 h by car for some). In addition some participants experienced FBs on “true” patients after baseline evaluation and were thus not interested in following the full simulation-based training program; although local supervisors were fully implicated in this study and were aware to avoid proposing in-vivo FBs to these trainees.
Thus, continuing to subject patients to FBs performed by still-learning students seems undesirable in light of the development of simulation-based bronchoscopy training tools , the efficacy of which has been demonstrated in the literature [21, 30] and further reinforced with our study. For these ethical reasons, we chose to not include a demonstration of the transfer of simulation-acquired competencies to real practice, as this would require a control group of interventions by learners on real patients. However, Ost et al. did do that in their 2001 study .
The absence of a statistically significant difference for median performance between Groups B and C in our study does not necessarily indicate that our moderately experienced bronchoscopists have FB skills comparable to our highly experienced bronchoscopists. Indeed our study evaluated only endobronchial anatomical visualization and identification, crucial, but nonetheless relatively basic FB steps. We did not evaluate competencies for more complex procedures, such as biopsies or the extraction of foreign objects. We note particularly that the standardized nature of the examination conditions (normal bronchial tree, no coughing or bleeding) did not allow us to test participant competencies in emergency or stressful situations. However, an earlier work showed that the technical skills of experienced surgeons remain stable under stress whereas those of less experienced surgeons diminish . It is thus probable that more experienced practitioners do better than their less-experienced counterparts for the managerial (team, procedure and task management) aspects when faced with urgent and/or stressful FB situations (hemoptysis, acute respiratory distress, etc.). These competencies, which were not evaluated in our work, are obviously essential in critical situations  and, when mastered, contribute greatly to the expertise of the practitioner.
In conclusion, we showed here that freely accessible self-training on a high-fidelity bronchoscopy simulator enables the acquisition of basic flexible bronchoscopy skills needed for complete endobronchial exploration. In this study, novice bronchoscopists who trained themselves on a simulator acquired performance levels for rapidity, dexterity and precision similar to those of moderately and or even highly experienced bronchoscopists. Thus, the results of our study, combined with those of other studies, lead us to suggest that classical bronchoscopy training be replaced rapidly and obligatory by high-fidelity simulation-based training programs upstream of any interventions on real patients. The availability and accessibility of such programs would be greatly facilitated by mobile simulators.
European Respiratory Society
Harmonising Education in Respiratory Medicine for European Specialists
High-Fidelity Bronchoscopy Simulator
Pue CA, Pacht ER. Complications of fiberoptic bronchoscopy at a university hospital. Chest. 1995;107(2):430–2.
Facciolongo N, Patelli M, Gasparini S, et al. Incidence of complications in bronchoscopy. Multicentre prospective study of 20,986 bronchoscopies. Monaldi Arch Chest Dis. 2009;71(1):8–14.
Simpson FG, Arnold AG, Purvis A, Belfield PW, Muers MF, Cooke NJ. Postal survey of bronchoscopic practice by physicians in the United Kingdom. Thorax. 1986;41:311–7.
Barlesi F, Dissard-Barriol E, Gimenez C, Doddoli C, Greillier L, Kleisbauer J-P. Évaluation de la tolérance de la fibroscopie bronchique par auto-questionnaire : la parole aux patients. Rev Mal Respir. 2003;20(3):335–40.
Adjoh KS, Fiogbé AA, Adambounou AS, et al. Tolérance et vécu de la fibroscopie bronchique chez des patients du CHU Sylvanus Olympio de Lomé - TOGO. J Fran Viet Pneu. 2015;17(6):1–53.
Ouellette DR. The safety of bronchoscopy in a pulmonary fellowship program. Chest. 2006;130(4):1185–90.
Ram B, Oluwole M, Blair RL, Mountain R, Dunkley P, White PS. Surgical simulation: an animal tissue model for training in therapeutic and diagnostic bronchoscopy. J Laryngol Otol. 1999;113(2):149–51.
Hilmi OJ, White PS, McGurty DW, Oluwole M. Bronchoscopy training: is simulated surgery effective? Clin Otolaryngol Allied Sci. 2002;27(4):267–9.
Deutsch ES, Christenson T, Curry J, Hossain J, Zur K. Multimodality education for airway endoscopy skill development. Ann Otol Rhinol Laryngol. 2009;118(2):81–6.
King EG, Sproule BJ, Yamamoto I. A teaching model for bronchoscopy. Chest. 1976;70:72–3.
Cunanan OS, Mazza JE. Integrated electronic display model for teaching bronchoscopy. Chest. 1977;72(3):364–5.
Naik VN, Matsumoto ED, Houston PL, Hamstra SJ, Yeung RY, Mallon JS, et al. Fiberoptic orotracheal intubation on anesthetized patients: do manipulation skills learned on a simple model transfer into the operating room? Anesthesiology. 2001;95(2):343–8.
Martin KM, Larsen PD, Segal R, Marsland CP. Effective nonanatomical endoscopy training produces clinical airway endoscopy proficiency. Anesth Analg. 2004;99(3):938–44.
Di Domenico S, Simonassi C, Chessa L. Inexpensive anatomical trainer for bronchoscopy. Interact Cardiovasc Thorac Surg. 2007;6(4):567–9.
Parotto M, Jiansen JQ, AboTaiban A, Ioukhova S, Agzamov A, Cooper R, et al. Evaluation of a low-cost, 3D-printed model for bronchoscopy training. Anaesthesiol Intensive Ther. 2017;49(3):189–97.
Colt HG, Crawford SW, Galbraith O. Virtual reality bronchoscopy simulation: a revolution in procedural training. Chest. 2001;120(4):1333–9.
Ost D, DeRosiers A, Britt EJ, Fein AM, Lesser ML, Mehta AC. Assessment of a bronchoscopy simulator. Am J Respir Crit Care Med. 2001;164(12):2248–55.
Wahidi MM, Silvestri GA, Coakley RD, Ferguson JS, Shepherd RW, Moses L, et al. A prospective multicenter study of competency metrics and educational interventions in the learning of bronchoscopy among new pulmonary fellows. Chest. 2010;137(5):1040–9.
Davoudi M, Osann K, Colt HG. Validation of two instruments to assess technical bronchoscopic skill using virtual reality simulation. Respir Int Rev Thorac Dis. 2008;76(1):92–101.
Colt HG, Davoudi M, Murgu S, Zamanian RN. Measuring learning gain during a one-day introductory bronchoscopy course. Surg Endosc. 2011;25(1):207–16.
Kennedy CC, Maldonado F, Cook DA. Simulation-based bronchoscopy training: systematic review and meta-analysis. Chest. 2013;144(1):183–92.
Ernst A, Wahidi MM, Read CA, Buckley JD, Addrizzo-Harris DJ, Shah PL, et al. Adult bronchoscopy training: current state and suggestions for the future: CHEST expert panel report. Chest. 2015;148(2):321–32.
Hureaux J, Urban T. Simulation training in pulmonary medicine: rationale, review of the literature and perspectives. Rev Mal Respir. 2015;32(10):969–84.
Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med J Assoc Am Med Coll. 2003;78(8):783–8.
Pastis NJ, Vanderbilt AA, Tanner NT, Silvestri GA, Huggins JT, Svigals Z, et al. Construct Validity of the Simbionix Bronch Mentor Simulator for Essential Bronchoscopic Skills. J Bronchol Interv Pulmonol. 2014;21(4):314–21.
Moorthy K, Smith S, Brown T, Bann S, Darzi A. Evaluation of virtual reality bronchoscopy as a learning and assessment tool. Respir Int Rev Thorac Dis. 2003;70(2):195–9.
Loddenkemper R, Séverin T, Eiselé J-L, Chuchalin A, Donner CF, Di Maria G, et al. HERMES: a European Core syllabus in respiratory medicine. Breathe. 2006;3(1):59–69.
Boyden EA. Segmental anatomy of the lungs. A Study of the Patterns of the Segmental Bronchi and Related Pulmonary Vessels. New York: McGraw-Hill Book Company, Inc; 1955.
Kirkpatrick DL. Evaluating Training Programs. The Four Levels. Berrett-Koehler Organizational Performance Series. San Francisco: Berrett-Koehler Publishers, Inc; 1994. Available on : http://eric.ed.gov/?id=ED382790
Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, et al. Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA. 2011;306(9):978–88.
Hsu KE, Man F-Y, Gizicki RA, Feldman LS, Fried GM. Experienced surgeons can do more than one thing at a time: effect of distraction on performance of a simple laparoscopic and cognitive task by experienced and novice surgeons. Surg Endosc. 2008;22(1):196–201.
Gaba DM, Fish KJ, Howard SK. Crisis management in anesthesiology. New York: Churchill Livingstone; 1994.
Authors thank M. Kevin Erwin for expert English rewording.
This project was funded through the PIA IDEFI grant from French National Research Agency to the SAMSEI project (Grant N°ANR-11-IDFI-0034).
English rewording was supported by through a grant from Bibliothèque Scientifique de l’Internat de Lyon and Hospices Civils de Lyon.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The study was approved by the local ethic committee of Lyon University hospital on 22/09/2015. Written consent to participate in this study was obtained from each participant.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Table S1. Sequential steps during a normal flexible bronchoscopy procedure and correspondent skills to acquire. (DOCX 17 kb)
Figure S1. Distribution of identification scores for Groups A, B and C. Figure S2. Distribution of visualization scores for Groups A, B and C. Figure S3. Distribution of overall performance scores (%) for Groups A, B and C. Figure S4. Distribution of global satisfaction scores for Groups A, B and C. Figure S5. Distributions for individual satisfaction survey items (n = 8, scored 0 (ineffective) to 6 (excellent)) for Groups A, B and C. Figure S6. Satisfaction survey. (DOCX 237 kb)