- Open Access
Is it valid to assess an individual’s performance in team training simulation when the supporting team are confederates? A controlled and randomized clinical trial
BMC Medical Education volume 22, Article number: 685 (2022)
During simulation training, the confederate is a member of the pedagogical team. Its role is to facilitate the interaction between participants and the environment, and is thought to increase realism and immersion. Its influence on participants' performance in full-scale simulation remains however unknown. The purpose of this study was to observe the effect of the presence of confederates on the participants’ performance during full-scale simulation of crisis medical situations.
This was a prospective, randomized study comparing 2 parallel groups. Participants were emergency medicine residents engaging in a simulation session, with or without confederates. Participants were then evaluated on their Crisis Resource Management performance (CRM). The overall performance score on the Ottawa Global Rating Scale was assessed as primary outcome and the 5 non-technical CRM skills as secondary outcomes.
A total of 63 simulation sessions, including 63 residents, were included for statistical analysis (n = 32 for Control group and 31 for Confederate group). The mean Overall Performance score was 3.9 ± 0.8 in the Control group and 4.0 ± 1.1 in the Confederate group, 95% confidence interval of the difference [-0.6; 0.4], p = 0.60. No significant differences between the two groups were observed on each CRM items (leadership, situational awareness, communication, problem solving, resource utilization)
In this randomized and controlled study, the presence of confederates during full-scale simulated practice of crisis medical situations does not seem to influence the CRM skills performance of Emergency medicine residents.
This study does not need to be registered on Clintrial as it does not report a health care intervention on human participants.
A fundamental assumption of simulation is that learning is enhanced when the environment seems realistic. Full-scale simulation attempts to recreate every elements of a real-life situation in order to persuade the participants to accept the “fiction contract” and to become fully engaged in the simulation experience [1,2,3]. Perceived realism is linked to the interactions between the participant and the environment. The use of a confederate in a full-scale simulation is of common practice to facilitate these interactions and enhance the immersion of the participant in the environment. The confederate is not a learner, but a member of the pedagogical team with a scripted role during the simulated practice phase . He aims to provide realism, challenges or information when they cannot be transcribed by the simulator [5, 6]. The confederate is used to improve the realism by limiting the biases induced by the simulated environment.
Experts suggest using full-scale simulation to enhance interprofessional teamwork and to deepen Crisis Resource Management (CRM) skills, as leadership, communication, problem solving, resources management and situational awareness . The lack of CRM skills in the management of critical situations has been proven to be associated with adverse events and malpractice cases [8, 9]. Learning CRM skills leads to better safety in patient outcomes, including a decrease in mortality .
The main limitation to the implementation of full-scale simulation in medical education programs is human resources . Appropriate and justified supervision is therefore essential [12,13,14]. The influence and role of each member of the pedagogical team should be argued, considering the performance of the participants as the main outcome. To our knowledge, the effects of confederate presence on the performance of participants in a full-scale simulation has not been addressed in proper quality studies.
The purpose of this study was to observe the effect of the presence of confederates on the participants’ performance during full-scale simulation of crisis medical situations.
This randomized controlled study took place in a French university medical simulation center from December 2018 to February 2020. According to the French ethic and regulatory law, article R1121-1 of the public health code, ethical approval for the study was waived by the national ethical committee (CPP, Comité de Protection des Personnes). It was registered at the register of epidemiologic studies of Toulouse University Hospital (RnIPH 2019–53) and has also been declared to National Commission of Informatics and Liberty (CNIL number: 2206723 v0). The University Hospital signed a commitment of compliance to the reference methodology MR-004. All participants provided written informed consent to participate.
Participants were residents within the 2nd year, 3rd year or 4th year post-graduate of Emergency Medicine at Toulouse University. In the curriculum of a resident in Emergency Medicine in Toulouse University, a 2nd year participates to 2 simulation of crisis medical situation within the year, 4 sessions for a 3rd year and a 4th year. The criterion for non-inclusion was the resident's refusal to participate in the study. Each resident participated in a single simulation session as part of the study.
The planning of the simulation sessions was fully integrated into the curriculum of the Emergency Medicine residents according to a simulation training program . Each session was developed in accordance with the French National Health Authority (Haute Autorité de Santé).
The session was led by one simulation instructor and one medical expert. The instructors belong to a group of 9 instructors (7 from a University Hospital and 2 from a General Hospital, including 2 nurses and 7 physicians) with a university degree in simulation training. The medical experts are emergency physicians (n = 15) with at least 5-year experience and specific knowledge in pediatrics, advanced life support, obstetrics, resuscitation, and airway management. The overall purpose of these sessions was to learn communication skills, knowledge of respective roles (leadership/fellowship), involvement in shared decision-making and team coordination.
A simulation session was intended for a complete medical team composed of an emergency physician (resident), a nurse and a medical support worker.
Each simulation session lasted 1 h and was divided into 3 parts: briefing (15 min), simulated practice (15 min) and debriefing (30 min). The briefing prepared the team for the simulated practice, created a positive learning environment, encouraged the emotional security and introduced the clinical situation. Debriefing was conducted by the assessors according to the RUST model and the good-judgement practice .
In the simulation, participants' performance was based on the assessment of CRM skills, a set of non-technical skills required to manage medical emergencies .
This assessment was carried out immediately after the debriefing by the simulation instructor and the expert physician who conducted the debriefing and were trained in the use of this assessment grid. The assessors (medical expert and instructor) were randomly assigned to each simulation session considering their availability. The participants were assessed individually by the assessors. Then results were shared among assessors in order to find a consensus on the final scoring.
Participant demographics were collected using a paper questionnaire to be completed before the simulation session. This data was then transferred to an anonymized Excel file.
In the Confederate group, two confederates playing the role of a nurse and a medical support worker were part of the session. The confederates are familiar with the simulated environment. They were asked to take no initiatives and to wait for the participant instructions without guidance. The role assigned to the confederates was to play their professional role (nurse emergency or medical support worker) and to help learners work in an unfamiliar simulation environment, guide learners to meeting learning objectives (for example, if the learner did not realize the patient was having a severe blood loss, the confederate could ask: “do you want me to do a hemoglobin test?”), protect simulators from damage, increase learners’ engagement in the scenario by selectively increasing participants’ cognitive load (by taking no initiative without a formal demand from the leaner (i.e. the physician) enrolled in the simulation), communicate with ‘‘control room’’ during scenarios (through an earpiece), offer insider experience during debriefing and/or evaluation, standardize manner in which information is conveyed to study participants (e.g., laboratory data, physical signs and symptoms, whether the patient has a known allergy, and so forth) to limit variability, minimizing risk for bias .
In the Control group, there was no confederate. An interprofessional group composed of a nurse and a medical support worker from the Emergency Department which were operating during the simulated practice but unfamiliar with the scenarios beforehand. They were asked to act as health professionals just as they would in a real-life situation. They could take initiatives and suggest therapeutics. The “real” team member wore earpiece, was briefed prior to scenario they will be asked to convey clues and then was clued by instructors at appropriate times/instructors phone into room with “results” at specified points.
The professionals of the Confederate group and the Control group both work at the pre-hospital medical service (SAMU) of the CHU of Toulouse. They have the same pre-requirement and clinical experience of at least 5 years of practice. Participants did not know the professionals in advance and could not identify whether they were confederates or not.
Scenarios used in the study
The scenarios were created by the different pedagogical managers of the simulation centers, in collaboration with the medical and paramedical trainers of each discipline. Each scenario was validated according to a standardized grid by the research team, composed of the university program director and simulation teachers (Additional file 1: Appendix A). They all have academic qualification in medical education.
It made it possible to evaluate the authenticity through the characteristics of a "complex problem" and the relevance of the professional situation described to promote learning to manage a critical situation in an interprofessional team.
Their design took into account the possibility of adapting the scenario on certain aspects related to environmental specificities linked to the learners' working conditions (emergency room, pre-hospital environment, ward…). Several professional situations were modelled in order to expose teams to various crisis situations such as cardiac arrest, difficult airway management, traumatic shock (Additional file 2: Appendix B).
The scenarios were performed in the emergency room or pre-hospital training room simulated of Toulouse Institute of Health Simulation center. The simulation laboratory was configured as an emergency room or prehospital setting (living room or bedroom) with a full-body manikin patient simulator (SimMan 3G; Laerdal Medical, Stavanger, Norway). The essential equipment were provided exactly as in real life, with a patient monitoring, a defibrillator, a resuscitation trolley or bag. Participants were briefed about the equipment before the beginning of the simulation session.
The choice of scenario was made randomly by the assessors at the beginning of the session according to their year of residency of each student.
Confederates were health professionals of the Emergency Department of the University Hospital. They have all benefited from a training-course in medical education. This is a five-day training course in order to master active teaching techniques, to integrate the specificities of the professional context into the practice of emergency care and to harmonize evaluation techniques. Within this framework, they were also trained in the use of full-scale simulation and specifically in the role of confederate during a three-day training course. They participated to debriefing as their own professional role. They did not take part in the assessment of the participants. Thirteen confederates participated in the study out of 24 trainers available at the Emergency department (Table 1). They were selected on a voluntary basis and according to their availability.
The assessors (medical expert and instructor) had at least 3 years' experience in CRM course. They had undergone training to use Ottawa GRS and use it frequently. The participants were assessed individually by each assessor. Then results were shared among assessors in order to find a consensus on the final scoring. Instructor and medical expert were not aware of the research hypothesis and outcomes of the study.
At the end of the simulation session, the resident was assessed on CRM performance according to the Ottawa Global Rating Scale (OGRS), composed of five CRM-related domains (leadership, communication, problem solving, resource utilization, situational awareness) and an overall performance rating. Each domain was scored on a seven-point Likert-like response format (7 being the highest), with descriptive anchors to aid to scoring (Additional file 3: Appendix C). The overall performance rating is an independent criteria. The inter-rater measurement difference in this evaluation grid is very small (Intraclass Correlation Coefficient > 0.6 interpreted as good according to Cicchetti) . This scale is a good instrument in simulated crisis situation assessment with a good Inter CC score (α = 0.96), and a good user-friendly, which is important in a complex assessment . It has shown strong interrater reliability and discriminative ability among postgraduate physician trainees .
The primary outcome for assessing the performance of the participants was the average scores obtained by the residents in each group (Control and Confederate) on the OGRS Overall Performance during the simulated practice.
The secondary outcomes for assessing each CRM performance of the residents were the average scores obtained by participants in each group (Control and Confederate) on 5 skills: leadership, communication, problem solving, resource utilization and situational awareness.
A planning of the simulation sessions on the study inclusion period had been prepared. The randomization of the sessions was realized by the Clinical Research Unit at Toulouse University Hospital in a 1:1 ratio, into 2 groups: Control group and Confederate group. The confederates were attributed to each session according to the randomization. No sessions were excluded prior to randomization (Fig. 1).
The number of simulation sessions was determined on the primary assumption that participants in the Confederate group would have a 0.4 point variability on the OGRS Overall Performance, compared to participants in the Control group . Based on Cohen's definition of statistical analyses in behavior science, a standard deviation of 0.5 was used . Given the initial hypothesis, a two-tailed test with a power of 80% was used. With these parameters, the number of simulation sessions required is a minimum of 50 sessions, i.e. two groups of 25 sessions.
Statistical analysis of the anonymized data was performed using BiostaTGV® software. All study variables were analyzed individually, checking for missing data and outliers. No imputation method was used for missing values.
The distribution of the quantitative variables was represented by the mean followed by the standard deviation after checking for normality. The comparative analyses were performed using the Student's test after checking for application criteria (distribution of values according to a Gaussian Normal Law). The threshold of statistical significance allowing to reject the H0 hypothesis, according to which the means are equal in the two groups, was considered reached when the risk of error was less than 5% (p-value < 0.05).
General characteristics of the participants
Sixty-three simulation sessions, including 63 residents, were included for analysis: 32 for Control group and 31 for Confederate group (Table 2). No student refused to participate to the study. The residents’ demographic characteristics were homogeneous between the two groups.
Primary outcome (Table 3).
The mean Overall Performance score was 3.9 in the Control group and 4.0 in the Confederate group, 95% confidence interval of the difference [-0.6; 0.4], p = 0.60.
Secondary outcomes (Table 3).
No significant differences between the two groups were observed on each CRM items (leadership, situational awareness, communication, problem solving, resource utilization).
In this randomized and controlled study, the presence of confederates during full-scale simulated practice of crisis medical situations does not seem to improve the CRM skills performance of Emergency medicine residents.
In the literature, it has been shown that performance and learning depend on the immersion quality into the simulated environment . To ensure participant’s immersion, standards of best practice for simulation design expressly require to use confederates who have formal training in simulation-based education. The facilitative approach must also be predetermined during the design phase of the scenario and the level of confederates’ involvement must be inversely proportional to the participant’s knowledge and experience . Our randomized controlled study suggests that using confederates with formal training has no added value on the participant’s performance. Indeed, situational awareness, problem solving and resource utilization scores showed no significant difference between the two groups. The confederates do not seem to have more effect on residents’ CRM performance than nurse and/or medical support worker without formal training in simulation-based education or predetermined facilitative approach. Our results do not show that the confederates improve resident’s immersion and interaction with the simulated environment enough to influence their performance.
In this study, within the Control group, nurses and medical support workers were not informed beforehand of the outcome of the scenario enabling a spontaneous attitude as a real life. The learning objective were team management in crisis situations. We could think that the interactions between team members is essential for the realism and impact the CRM performance. The nurses and medical support workers have been probably more focus on their role as healthcare professionals and allowed more realistic interactions with the resident.
Research suggests that in a full-scale simulation, participant’s behavior is closely linked to his confederate’s one so that he comes to imitate their actions . In the case of confederates with very precise scripts often far from reality, deliberately less "helpful" to increase the difficulty of a simulation session, it can be assumed that the simulation session would be locked into a sterile situation without participation or mutual support . In our study, in the Confederate group, confederates were given instructions to act less helpful by taking no initiatives even though they probably would in a real-life situation. By being part of the pedagogical team and knowing the scenarios outcomes beforehand, it could be hypothesized that their involvement within the simulation was less intense especially with less emotional participation such as doubt or stress and decreased the interaction between the participants and the confederates. However, the communication skill scores showed no significant differences between the two groups. Suggesting that the presence of trained confederates has no negative impact on participants’ CRM performance. Nevertheless, the standardization of confederates’ behavior in full-scale simulation could prevent some errors of the team during the trained session and prevent them from being addressed during the debriefing. According to our results, confederates do not seem to limit the biases related to the simulated environment that could influence the performance of the participants . The development of training team-based simulation is challenged by the availability of pedagogical staff and by the limited number of learners in each session .
As the two groups were only compared on their CRM performance in a simulated scenario which has poor significance in a real life setting, it would be interesting to assess for a further work the effects if any, of confederate/non-confederate on learning and subsequent transfer to the clinical environment.
One of the main limitations of this study is the lack of blinded and compared assessment. Assessment was performed by a group of instructors who undergone training to use the OGRS and they use it frequently. The use of the OGRS has been shown to have a good reproducibility between observers but the study design did not allow to have a blinded nor a compared assessment . Furthermore, assessor distribution was not random but according to their availability. It would have strengthen the design if assessors assessed similar numbers of scenarios across similar year groups of participants.
Another limitation of the study concerns the differences in the participants’ semester of residency between the two groups. Despite randomization, more 7th semester residents were present in the Control group and more 3rd semester residents in Confederate group. This could lead to a difference in performance in favor of control group. Nevertheless, this difference is relatively modest and probably insufficient to impact the CRM performance of either group. Moreover, the experience for simulation tool use was similar between groups. This have been shown to be an import impact on performance .
The presence of confederates during the simulated practice of a full-scale simulation seems to have no significant impact on the participants’ CRM performance.
This study questions the interest of confederates in simulation-based education. Not using confederates could reduce the pedagogical staff mobilized in full-scale simulation and improve the implementation of interprofessional simulation but still preserve participants’ performances. The impact of confederates’ presence during the debriefing phase has not been studied by the present study and should eventually be explored in further studies.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
All authors confirm that all methods were carried out in accordance with relevant guidelines and regulations.
Crisis Resource Management
Ottawa Global Rating Scale
Bearman M, Greenhill J, Nestel D. The power of simulation: a large-scale narrative analysis of learners’ experiences. Med Educ. 2019;53(4):369–79.
Dieckmann P, Manser T, Wehner T, Rall M. Reality and Fiction Cues in Medical Patient Simulation: An Interview Study with Anesthesiologists. J Cogn Eng Decis Mak. 2007;1(2):148–68.
Rudolph JW, Simon R, Raemer DB. Which reality matters? Questions on the path to high engagement in healthcare simulation. Simul Healthc. 2007;2(3):161–3.
Lopreiato J O. Healthcare Simulation Dictionary. Rockville, MD: Agency for Healthcare Research and Quality. AHRQ Publication. 2016;16(17)-0043.
Nestel D, Mobley BL, Hunt EA, Eppich WJ. Confederates in health care simulations: not as simple as it seems. Clin Simul Nurs. 2014;10(12):611–6.
Sanko JS, Shekhter I, Kyle RR, Di Benedetto S, Birnbach DJ. Establishing a convention for acting in healthcare simulation: merging art and science. Simul Healthc. 2013;8(4):215–20.
L’Her E, Geeraerts T, Desclefs JP, Benhamou D, Blanie A, Cerf C, Delmas V, Jourdain M, Lecomte F, Ouanes I, Garnier M, Mossadegh C. Intérêts de l’apprentissage par simulation en soins critiques, Recommandations communes SRLF-SFAR-SFMU-SOFRASIMS – 2018–1019.
Michel P, Minodier C, Lathelize M, Moty-Monnereau C, Domecq S, Chaleix M, Izotte-kret M, Bru-Sonnet R, Quenon J.-L, Olier L, 2010, « Les événements indésirables graves associés aux soins observés dans les établissements de santé », Dossiers Solidarité et santé, DREES, n° 17.
Risser DT, Rice MM, Salisbury ML, et al. The potential for improved teamwork to reduce medical errors in the emergency department. The Med Teams Research Consortium Ann Emerg Med. 1999;34(3):373–83.
Boet S, Bould MD, Fung L, Qosa H, Perrier L, Tavares W, et al. Transfer of learning and patient outcome in simulated crisis resource management: a systematic review. Can J Anaesth. 2014;61(6):571–82.
Zendejas B, Wang AT, Brydges R, Hamstra SJ, Cook DA. Cost: the missing outcome in simulation-based medical education research: a systematic review. Surgery. 2013;153(2):160–76.
Boet S, Bould MD, Sharma B, Revees S, Naik VN, Triby E, et al. Within-team debriefing versus instructor-led debriefing for simulation-based education: a randomized controlled trial. Ann Surg. 2013;258(1):53–8.
Boet S, Pigford A-A, Fitzsimmons A, Reeves S, Triby E, Bould MD. Interprofessional team debriefings with or without an instructor after a simulated crisis scenario: An exploratory case study. J Interprof Care. 2016;30(6):717–25.
Isaranuwatchai W, Alam F, Hoch J, Boet S. A cost-effectiveness analysis of self-debriefing versus instructor debriefing for simulated crises in perioperative medicine in Canada. J Educ Eval Health Prof. 2017;13:44.
Houzé-Cerfon CH, Lauque D, Wiel E, Bounes V, Charpentier S. Conception d’un programme d’enseignement par simulation dans le DES de médecine d’urgence selon la méthode du modèle logique. Ann Fr Med Urgence. 2020;10:14–30.
Karlsen R. Stable Program. Adaptation of the RUS model. Cambridge: Original work from the Center for Medical Simulation (D.R.); 2013.
Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb YA. Simulation-Based Training in Anesthesia Crisis Resource Management (ACRM): A Decade of Experience. Simul Gaming. 2001;32(2):175–93.
Truog RD, Meyer EC. Deception and death in medical simulation. Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare. 2013;8(1):1–3.
Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as “CRM simulator study IB”). Simul Healthc. 2009;4(1):6–16.
Jirativanont T, Raksamani K, Aroonpruksakul N, Apidechakul P, Suraseranivongse S. Validity evidence of non-technical skills assessment instruments in simulated anaesthesia crisis management. Anaesth Intensive Care. 2017;45(4):469–75.
Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994;6(4):284–90.
Cohen J. Statistical Power Analysis for the Behavioural Sciences. Hillsdale: Lawrence Erlbaum Associates, Publishers; 1988.
Gutiérrez F, Pierce J, Vergara VM, Coulter R, Saland L, Caudell TP, Goldsmith TE, Alverson DC. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education. Stud Health Technol Inform. 2007;125:155–60.
INACSL. INACSL standards of best practice: Simulation simulation design. Clinical Simulation in Nursing. 2016;12:S5–12.
Scharoun S, Mintz A, Glazebrook CM, Roy E, Gonzalez D. How Working With a Helpful Versus Less Helpful Confederate Influences Joint-Action in a Pegboard Task. J Mot Behav. 2017;49(6):619–28.
Adler MD, Overly FL, Nadkarni VM, Davidson J, Gottesman R, Bank I, et al. An Approach to Confederate Training Within the Context of Simulation-Based Research. Simul Healthc. 2016;11(5):357–62.
Houzé-Cerfon CH, Boet S, Marhar F, Saint-Jean M, Geeraerts T. Simulation-based interprofessional education for critical care teams: Concept, implementation and assessment. Presse Med. 2019;48:780–7.
Clarke S, Horeczko T, Carlisle M, Barton JD, Ng V, Al-Somali S, et al. Emergency medicine resident crisis resource management ability: a simulation-based longitudinal study. Med Educ Online. 2014;19(9):25771.
We would like to thank Vanessa Houzé-Cerfon who actively participated in the copy editing of the manuscript.
Authors declare no source of funding for the research.
Ethics approval and consent to participate
According to the French ethic and regulatory law, article R1121-1 of the public health code, ethical approval for the study was waived by the national ethical committee (CPP, Comité de Protection des Personnes). It was registered at the register of epidemiologic studies of Toulouse University Hospital (RnIPH 2019-53) and has also been declared to National Commission of Informatics and Liberty (CNIL number: 2206723 v0). The University Hospital signed a commitment of compliance to the reference methodology MR-004. All participants provided written informed consent to participate. No physical, emotional, psychological or social risks are induced by the study. Participation or refusal to participate will have no impact on any professional or academic evaluations.
Consent for publication
Every participant has agreed for the use of their data in the manuscript. They have given their consent for publication by signing the consent form.
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work is attributed to the Toulouse Institute of simulation healthcare and the University Hospital Toulouse, Toulouse, France.
Additional file 1: APPENDIX A.
SCENARIO ASSESSMENT GRID.
Additional file 2: APPENDIX B.
Additional file 3: APPENDIX C.
OTTAWA GLOBAL RATING SCALE.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Traoré, J., Balen, F., Geeraerts, T. et al. Is it valid to assess an individual’s performance in team training simulation when the supporting team are confederates? A controlled and randomized clinical trial. BMC Med Educ 22, 685 (2022). https://doi.org/10.1186/s12909-022-03747-3
- Health education
- Simulation training
- Interprofessional education
- Crisis resource management
- Emergency medicine