Effect of an audience on trainee stress and performance during simulated neonatal intubation: a randomized crossover trial
BMC Medical Education volume 18, Article number: 230 (2018)
Neonatal intubation is a stressful procedure taught to trainees. This procedure can attract additional observers. The impact of observers on neonatal intubation performance by trainees has not been studied. Our objective was to evaluate if additional observers present during neonatal mannequin endotracheal intubation (NMEI) by junior trainees, affects their performance and their stress levels.
A randomized cross over trial was conducted. First year residents with no experience in neonatal intubation were assigned to NMEI condition A or B randomly on day 1. Subjects were crossed over to the other condition on day 2.
Condition A: Only one audience member was present Condition B: Presence of an audience of 5 health care providers.
Differences in the time to successful NMEI was recorded and compared between conditions. A portable heart rate monitor was used to measure peak heart rate above baseline during NMEI under both conditions.
Forty nine residents were recruited. 72% were female with a median age of 25 years (IQR: 24–27). Time to successful intubation was comparable under both conditions with a mean difference of − 3.94 s (95% CI: -8.2,0.4). Peak heart rate was significantly lower under condition A (mean difference − 11.9 beats/min, 95% CI -15.98 to − 7.78).
Although the time required to NMEI did not increase, our results suggest that presence of observers significantly increases trainee stress. The addition of extraneous observers during simulation training may better equip residents to deal with such stressors.
Date of registration: March 2016, NCT 02726724.
Endotracheal intubation (ETI) is a common procedure in neonatal intensive care units (NICU). This procedure is associated with adverse effects such as laryngospasm and subsequent risk of intracranial hemorrhage. Such effects can cause significant neonatal morbidity . Traditionally, residents are often taught neonatal ETI during their first NICU rotation, and any poor delivery room experience can create a lack of confidence . Due to increased trainee numbers, reduced duty hours, increased use of non-invasive ventilation, as well of the presence of other health care professionals competing for a limited number of procedures, opportunities to teach ETI in a clinical setting are dwindling. In order to prepare trainees adequately for their first clinical encounter, simulation is increasingly used. ETI in the NICU must often be performed in the presence of multiple health care workers who may be observing the trainee attempting their initial ETI in a live neonate.
The presence of observers is known to increase stress levels. The Trier Social Stress Test (TSST) is a widely used protocol to induce stress in laboratory settings. In the original TSST, where subjects have to deliver a speech and perform mental arithmetic in front of an audience, there was a significant increase in stress hormones and individual heart rates . Most of the studies using the TSST show heart rate increases of 24 to 34% [4, 5]. It has also been suggested that stress can interfere with the performance of technical skills in critical situations . Currently, simulated ETI or other pediatric procedures are taught to trainees without accounting for the presence of multiple observers. When a trainee is faced with an unfamiliar situation, he will experience it as stressful and this could lead to adverse performance . It is unknown if additional observers could adversely affect performance of trainees. Therefore, we conducted a randomized crossover study to obtain objective evidence that junior trainees are adversely affected by the presence of a large audience during neonatal mannequin endotracheal intubation (NMEI).
We hypothesized that the time required to successful NMEI would be negatively impacted by the presence of observers. We also hypothesized that stress levels would be higher in the presence of an audience as measured by the subject heart rate.
We conducted a randomized cross over study at Maisonneuve Rosemont Hospital, a Level III NICU in Montreal, Canada. The institutional scientific and ethics review board approved the study. Every participant gave written informed consent before enrolment. Participants were informed of their right to discontinue participation at any time.
Residents were recruited during their pediatrics or neonatology rotation in our hospital. All first-year trainees were approached sequentially by the main investigator to participate during the recruitment period. Trainees with prior experience in live newborn intubation were excluded in order to select novice learners. Students with a history of beta blocker use within the past year, and students with a history of antidepressant medication were excluded. All trainees had completed the neonatal resuscitation program (NRP) training course during which they had the chance to practice NMEI on a mannequin at the start of their rotation block. For our study, participants were told they would perform NMEI and that their performance during two NMEI scenarios would be evaluated in the context of scientific investigation. The true goal of the study was not revealed until the end of the experience. Participants were asked to keep their experience confidential.
A full body neonatal mannequin approximating a term newborn (Newborn Anne, Laerdal Medical Corporation, Wappinger Falls, NY) was setup in a hospital delivery room suite on a radiant warmer. Subjects were asked to wear a cardiac monitor, and 5 min later after a period of rest, they were then called to the delivery room. Heart rate was assessed continuously, starting 5 min before NMEI and stopping 5 min after cessation of the procedure, using a wireless heart rate monitor (Polar M400). Five minutes after their arrival to the delivery room, they were given a stylet and a 3.5 ET tube and asked to intubate orally a neonatal mannequin. No practice attempts were allowed. After 30 s, the residents were reminded verbally that they had 15 remaining seconds. Each individual attempt was recorded as the time in seconds from insertion to the removal of the laryngoscope, using a chronometer operated by the main investigator. Success or failure of the attempt was assessed by the investigator after removal of the laryngoscope. If the resident failed the intubation, he was given the laryngoscope and the tube for a subsequent attempt and the chronometer was restarted from the time it was stopped. The duration of all attempts was limited to a total of 45 s. A participant who never succeeded, was assigned an intubation time of 45 s. A maximum time limit of 45 s was chosen, to best approximate the physiological tolerance of a live infant. The NRP recommends limiting each attempt to 30 s. However, previous studies suggest a mean intubation time of 38 s in trainees .
In this randomized crossover study, each volunteer participated in two NMEI scenarios under two different conditions, A and B, and served as his/her own control. The sequence, A then B, or B then A, was assigned randomly.
Only one staff neonatologist was present with the participant during NMEI.
An audience of 5 people were present to watch the participant during NMEI, with at least 2 clearly identified neonatologists, one being the main investigator. The other 3 observers were health care professionals (nurses or other trainees) from the perinatal service. Audience members stood within 2 m of the subject, but remained silent, neutral and did not interact with the trainee.
The residents who performed NMEI in condition A on day one were crossed-over to NMEI the next day under condition B, and vice versa. The time and the location of the 2 procedures were the same.
We chose time to successful intubation as the primary outcome. We did not use success or failure of intubation as the primary outcome, since we expected all trainees to eventually be able to intubate a static mannequin with no vital signs given enough time. The mean difference in total intubation time between condition A and B was compared.
The increase in heart rate from baseline (5 min before NMEI) to peak (during laryngoscopy) as a percentage value was calculated for each subject, and the mean difference between condition A and B was compared.
The rate of success on first attempt as well as the rate of success across all attempts was reported under each condition. Successful intubation was defined as tube placement below the vocal cords within 45 s.
To estimate the required sample size, we used the study by O’Donnel et al. who gave an approximate intubation time for residents of 38 s in live infants, with a standard deviation of 20 s . Using a two tailed alpha threshold of 0.05 and power of 80%, a sample size of 50 subjects was estimated to detect an 8 s difference time between the two different groups. Eight seconds represents a 25% change in time to intubate and is clinically significant for a newborn.
A simple randomization scheme was used, and the sequence of envelopes was generated by a random coin flipper (www.random.org). Allocation was revealed by opening a pre-prepared sealed envelope before participants were called to the delivery room. Neither the participants nor the investigator was blinded to allocation. The audience present under condition B did not know if the resident had already performed under condition A.
The difference in NMEI time and heart rate increase was calculated between conditions A and B for each candidate, and the mean difference within subjects was compared to the null value of 0 using a two tailed paired t-test. The rate of success of intubation between both conditions was compared with the McNemar chi-squared test. An alpha level of 0.05 was considered significant. All calculations were done with SAS 9.4 (Toronto, ON).
Subjects were recruited from November 2015 to November 2016. Participant flow is shown in Fig. 1. Of the 60 eligible first year residents in the period, 10 declined or did not respond to our request to participate. One participant completed condition A but never presented for condition B, and no outcome data is available for this subject. Median age for all participants was 25 years (IQR 24–27) and 72% were female.
Results for all 49 analyzed participants are presented in Table 1.
There was no significant difference in the time to intubation under condition A and B. However, under condition B, the increase in heart rate over baseline was significantly higher than condition A. The rate of successful intubation on first attempt, or all attempts was comparable.
Due to the crossover design, additional analyses were performed to control for sequence. To test for carryover effects, the effect of sequence was compared (AB vs BA) with no significant difference either in time to intubation (p = 0.09) or heart rate (p = 0.79).
This is the first study in the medical area which evaluated the impact of an audience on performance during simulated intubation, in junior trainees. The time and the rate of successful NMEI was identical under both conditions, whether with one or multiple observers. However, we documented that a larger audience alone, neutral and without other distractors was associated with increased heart rate, a possible indicator of increased stress. There are many definitions of stress. The World Health Organization defined work-related stress as the response people may have when presented with work demands and pressures that are matched to their knowledge and abilities and which challenge their ability to cope. The relationship between stress and performance is not linear. Stress can have different effect on individuals. Studies have demonstrated that the effects of stress intensity on behavior are characterized by an inverted-U-shaped function while low or high levels of stress lead to performance impairment on tests of vigilance and working memory, a moderate level of stress leads to performance improvement . It was also reported that baseline stress in human participants measurements were predictive of individual resilience to stress, including the impact stress had on physiological reactivity and performance . The response to acute stress is highly dependent on the individual’s perception of demands and resources . Research has found that psychological and social resources may be most protective at relatively low or moderate levels of stress and less so at high levels of stress .
Increased stress level does not always affect performance in the medical domain as was reported by many investigators. In a neonatal simulation procedure when both subjective and objective measures of stress increased over the duration of the simulated experience, there was no association between performance and either cortisol or subjective stress . However, in a simulated trauma scenario, some aspects of performance and immediate recall appear to be impaired in complex clinical scenarios in which they exhibit elevated subjective and physiologic stress responses . The impact of stress on performance may vary with the trainee’s perception, the task at hand and the ability to cope.
The presence of an audience has been described as increasing stress in the literature. In a psycho-social study with 183 participants going through the TSST who experienced either an unsupportive audience, a supportive audience, or no audience, both audience conditions produced significantly higher cortisol, heart rate, and blood pressure responses to the stressful tasks, relative to the no-audience control . Belletier and al provided evidence that simply being watched by evaluative others leads individuals to perform poorly and to choke on a classic measure of executive control .
Two different processes leading to poor performance in high-pressure situations were described in the literature. The first process, choking, occurs because worries distract executive attention. In the second, the pressure shifts too much executive attention toward the task at hand, which may cause poor performance in routine (non-attention-demanding) tasks relying on skill processes and procedures that normally run best outside of conscious awareness . Choking under pressure is in situations where one is being watched by others especially when one’s performance is being evaluated [16, 17].
A study conducted in thirty physicians who performed two simulated resuscitation scenarios in random order, one scenario without additional distractors and one scenario with additional distractors (noise, scripted family member) shows that the performance scores were lower under experimental conditions than under control conditions. External distractors markedly reduced the quality of cardiopulmonary resuscitation and there were no overall significant differences in median performance scores between groups with different levels of training (1st/2nd year residents, 4th/5th year residents: consultant anaesthetists, p = 0.519) . Hunziker and al investigated the influence of a short task focusing strategy on perceived stress levels and performance of rescuers in a simulated CPR scenario . In this randomized-controlled trial, a total of 124 volunteer medical students were randomized to receive a 10-min instruction to cope with stress by loudly posing two task-focusing questions (“what is the patient’s condition?”, “what immediate action is needed?”) when feeling overwhelmed by stress (intervention group) or a control group. A brief stress-coping strategy moderately decreased perceived stress without significantly affecting performance in a simulated CPR . The presence of family members, which is a particular kind of audience, during procedures, has been studied in recent literature. In a retrospective study, the presence of family members did not have an impact on the success of pediatric intubation . However, junior residents are not in favor of the presence of family because they are worried about being judged by the parents [21, 22].
Residents are faced with stressful procedures in the NICU and they are afraid of being judged by multiple observers, whose presence is often unnecessary, and by the attending staff responsible for their evaluation. The present study suggests another mechanism of stress in the junior resident. By understanding mechanisms of stress, medical educators might improve training by providing learners with resources in stress management. The repetition of stressful events can lead to depression, burnout and anxiety .
This study has several limitations. The ETI is done on a neonatal mannequin and does not necessarily represent real life. There were no distractors around the procedure and the results may have been different if there was a combination of an audience and distractors, as would be the case in a real NICU setting. In particular, there were no vital signs given to subjects to simulate patient bradycardia and desaturation during attempts. Our study was powered to detect a change in time to intubation of 8 s, based on the mean time to intubation of 38 s in O’Donnel et al. study done in real infants . The mean duration in our study was 30 s under condition A, which is 20% shorter than expected. This would reduce the minimal effect size we could detect. Our study was not powered to detect a difference in intubation success rates, though our success rate of 65 and 79% in condition A and B was much higher than rates 20% reported in live infant . This suggests a lower difficulty level for NMEI versus live ETI, particularly since attempt durations were not limited by physiologic tolerance. Heart rate measurements were a secondary outcome of our study, since our sample size was calculated based on estimated time to intubation. Furthermore, heart rate is one of several measures of stress, and we did not look at others measures, such as blood pressure or salivary cortisol.
The presence of an external audience did not negatively impact the duration of simulated intubation by novice trainees, but the presence of non-mandatory personnel as observers is stressful to trainees. Our study has important implications for simulation training in neonatal ETI, and possibly in training for other stressful procedures. To adequately prepare trainees to manage stress, it is imperative that simulation scenarios be representative of real life clinical situations where multiple observers are often present.
Neonatal endotracheal mannequin intubation
Neonatal intensive care unit
The Trier Social Stress Test
Marshall TA, Deeder R, Pai S, Berkowitz GP, Austin TL. Physiologic changes associated with endotracheal intubation in preterm infants. Crit Care Med. 1984;12(6):501–3.
Lee HC, Chitkara R, Halamek LP, Hintz SR. A national survey of pediatric residents and delivery room training experience. J Pediatr. 2010;157(1):158–61 e3.
Kirschbaum C, Pirke KM, Hellhammer DH. The 'Trier social stress Test'--a tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology. 1993;28(1–2):76–81.
Schommer NC, Hellhammer DH, Kirschbaum C. Dissociation between reactivity of the hypothalamus-pituitary-adrenal axis and the sympathetic-adrenal-medullary system to repeated psychosocial stress. Psychosom Med. 2003;65(3):450–60.
Kotlyar M, Donahue C, Thuras P, Kushner MG, O'Gorman N, Smith EA, et al. Physiological response to a speech stressor presented in a virtual reality environment. Psychophysiology. 2008;45(6):1034–7.
Bjorshol CA, Myklebust H, Nilsen KL, Hoff T, Bjorkli C, Illguth E, et al. Effect of socioemotional stress on the quality of cardiopulmonary resuscitation during advanced life support in a randomized manikin study. Crit Care Med. 2011;39(2):300–4.
Saunders T, Driskell JE, Johnston JH, Salas E. The effect of stress inoculation training on anxiety and performance. J Occup Health Psychol. 1996;1(2):170–86.
O'Donnell CP, Kamlin CO, Davis PG, Morley CJ. Endotracheal intubation attempts during neonatal resuscitation: success rates, duration, and adverse effects. Pediatrics. 2006;117(1):e16–21.
Lupien SJ, Maheu F, Tu M, Fiocco A, Schramek TE. The effects of stress and stress hormones on human cognition: implications for the field of brain and cognition. Brain Cogn. 2007;65(3):209–37.
Winslow BD, Carroll MB, Martin JW, Surpris G, Chadderdon GL. Identification of resilient individuals and those at risk for performance deficits under stress. Front Neurosci. 2015;9:328.
Lazarus RS. The psychology of stress and coping. Issues Ment Health Nurs. 1985;7(1–4):399–418.
Whisman MA. Mediators and moderators of change in cognitive therapy of depression. Psychol Bull. 1993;114(2):248–65.
Finan E, Bismilla Z, Whyte HE, Leblanc V, McNamara PJ. High-fidelity simulator technology may not be superior to traditional low-fidelity equipment for neonatal resuscitation training. J Perinatol. 2012;32(4):287–92.
Harvey A, Bandiera G, Nathens AB, LeBlanc VR. Impact of stress on resident performance in simulated trauma scenarios. J Trauma Acute Care Surg. 2012;72(2):497–503.
Taylor SE, Seeman TE, Eisenberger NI, Kozanian TA, Moore AN, Moons WG. Effects of a supportive or an unsupportive audience on biological and psychological responses to stress. J Pers Soc Psychol. 2010;98(1):47–56.
Belletier C, Davranche K, Tellier IS, Dumas F, Vidal F, Hasbroucq T, et al. Choking under monitoring pressure: being watched by the experimenter reduces executive attention. Psychon Bull Rev. 2015;22(5):1410–6.
DeCaro MS, Thomas RD, Albert NB, Beilock SL. Choking under pressure: multiple routes to skill failure. J Exp Psychol Gen. 2011;140(3):390–406.
Krage R, Tjon Soei Len L, Schober P, Kolenbrander M, van Groeningen D, Loer SA, et al. Does individual experience affect performance during cardiopulmonary resuscitation with additional external distractors? Anaesthesia. 2014;69(9):983–9.
Hunziker S, Pagani S, Fasler K, Tschan F, Semmer NK, Marsch S. Impact of a stress coping strategy on perceived stress levels and performance during a simulated cardiopulmonary resuscitation: a randomized controlled trial. BMC Emerg Med. 2013;13:8.
Sanders RC Jr, Nett ST, Davis KF, Parker MM, Bysani GK, Adu-Darko M, et al. Family presence during pediatric tracheal intubations. JAMA Pediatr. 2016;170(3):e154627.
Boudreaux ED, Francis JL, Loyacano T. Family presence during invasive procedures and resuscitations in the emergency department: a critical review and suggestions for future research. Ann Emerg Med. 2002;40(2):193–205.
Beesley SJ, Hopkins RO, Francis L, Chapman D, Johnson J, Johnson N, et al. Let them in: family presence during intensive care unit procedures. Ann Am Thorac Soc. 2016;13(7):1155–9.
Chrousos GP. Stress and disorders of the stress system. Nat Rev Endocrinol. 2009;5(7):374–81.
Haubner LY, Barry JS, Johnston LC, Soghier L, Tatum PM, Kessler D, et al. Neonatal intubation performance: room for improvement in tertiary neonatal intensive care units. Resuscitation. 2013;84(10):1359–64.
No external funding for this manuscript. All the authors have indicated there is no financial relationships relevant to this article to disclose.
Availability of data and materials
Ethics approval and consent to participate
The institutional scientific and ethics review board of Maisonneuve Rosemont Hospital (CIUSSS de l’Est de Montréal) approved the study (Number: CER 15012). Every participant gave written informed consent before enrollment.
Consent for publication
None of the authors have any conflicts of interest to disclose.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bensouda, B., Mandel, R., Mejri, A. et al. Effect of an audience on trainee stress and performance during simulated neonatal intubation: a randomized crossover trial. BMC Med Educ 18, 230 (2018). https://doi.org/10.1186/s12909-018-1338-4