- Open Access
GENESISS 1—Generating Standards for In-Situ Simulation project: a scoping review and conceptual model
BMC Medical Education volume 22, Article number: 479 (2022)
In-Situ Simulation (ISS) enables teams to rehearse and review practice in the clinical environment to facilitate knowledge transition, reflection and safe learning. There is increasing use of ISS in healthcare organisations for which patient safety and quality improvement are key drivers. However, the effectiveness of ISS interventions has not yet been fully demonstrated and requires further study to maximise impact. Cohesive programmatic implementation is lacking and efforts to standardise ISS terms and concepts, strengthen the evidence base and develop an integrated model of learning is required. The aim of this study was to explore the current evidence, theories and concepts associated with ISS across all areas of healthcare and develop a conceptual model to inform future ISS research and best practice guidance.
A scoping review was undertaken with stakeholder feedback to develop a conceptual model for ISS. Medline, OpenGrey and Web of Science were searched in September 2018 and updated in December 2020. Data from the included scoping review studies were analysed descriptively and organised into categories based on the different motivations, concepts and theoretical approaches for ISS. Categories and concepts were further refined through accessing stakeholder feedback.
Thirty-eight papers were included in the scoping review. Papers reported the development and evaluation of ISS interventions. Stakeholder groups highlighted situations where ISS could be suitable to improve care and outcomes and identified contextual and practical factors for implementation. A conceptual model of ISS was developed which was organised into four themes: 1. To understand and explore why systematic events occur in complex settings; 2.To design and test new clinical spaces, equipment, information technologies and procedures; 3. To practice and develop capability in individual and team performance; 4. To assess competency in complex clinical settings.
ISS presents a promising approach to improve individual and team capabilities and system performance and address the ‘practice-theory gap’. However, there are limitations associated with ISS such as the impact on the clinical setting and service provision, the reliance of having an open learning culture and availability of relevant expertise. ISS should be introduced with due consideration of the specific objectives and learning needs it is proposed to address. Effectiveness of ISS has not yet been established and further research is required to evaluate and disseminate the findings of ISS interventions.
In healthcare, scenario-based simulation increasingly involves multi-professional teams with the aim of enhancing the application of knowledge, embedding evidence-based practice and improving the performance of the team as a whole [1, 2]. In-situ simulation (ISS) enables teams to practice in the clinical environment where genuine care takes place, and has principally focused on low probability, high consequence events . ISS may offer additional benefits to traditional ‘education-centre’ based simulation, enabling participants to problem solve within their own dynamic setting and facilitating the contextualised implementation of learning into practice [4, 5]. ISS can be a mechanism to explore the interplay between and within micro systems (individual, team and task factors) and macro level phenomena (hospital departments, facilities and systems; ), enabling latent safety threat (LST) detection leading to improvements in safety and performance [1, 7]. ISS models are relevant to various healthcare settings as they are flexible to different contexts . There may also be economic benefits associated with ISS when compared to simulation training undertaken in dedicated simulation suites [4, 9]. During the COVID-19 pandemic, ISS interventions have been used to test the ability of healthcare teams to effectively implement use of personal protective equipment (PPE), test infection control guidelines and operational readiness of intensive care units and operating rooms [10,11,12,13,14].
Haji et al.  developed a theory-based, iterative, programmatic framework for simulation interventions adapted from the Medical Research Council (MRC) framework for complex interventions . Multiple theories were identified which are applicable to stages of simulation development: learning or instructional design theories for intervention development and modelling; cognitive and behavioural science where participant behaviour change or transfer of knowledge and skills is required; socio-cognitive theories for team based training; and implementation and complexity science to explore integration and contextual factors . Although there is a paucity of conceptual, planning and evaluation frameworks which are solely focused on ISS, components of existing conceptual frameworks for general simulation training and existing literature reporting and evaluating ISS interventions may provide a useful starting point.
The National Simulation Development Project Report (The Higher Education Academy, Association for Simulated Practice in Healthcare (ASPiH) & Health Education England, 2014) identified that the use of ISS is increasing within the United Kingdom (UK) but requires more comprehensive and cohesive strategic and operational support to achieve the potential benefits offered. Efforts to standardise ISS terms and concepts and develop an integrated model of learning is required [2, 4, 17]. Previous reviews have focused on ISS for education and training of healthcare professionals , ISS in operating rooms , effect on patient outcomes  or have included ISS within a broader simulation approach within acute care settings , for caesarean section training  and obstetric emergency teams . Reviews have highlighted a limited but promising evidence base for ISS and reported high variability of ISS approaches to design, delivery, and evaluation [18,19,20,21,22,23].
The aim of this study was to explore the current evidence, theories and concepts associated with ISS across all areas of healthcare and develop a conceptual model to inform future ISS research and best practice guidance.
This study involved three components:
A scoping review of the current literature relating to ISS
Consultation with stakeholders from clinical and health education organisations
Development of conceptual and logic models of ISS interventions
Component one: scoping review
We selected a scoping review as they are designed to explore the extent, range, and nature of the emerging evidence [24,25,26]. A scoping review would enable us to categorise the concepts and theoretical approaches for ISS in healthcare and develop theories about how distinctive mechanisms of ISS (‘natural teams in natural settings’) have the potential for addressing specific learning and clinical needs for the individual, team and various organisation levels. We conducted a scoping review of the published ISS literature following the framework by Levac et al. .
The scoping review aimed to address the following questions:
What types of ISS healthcare interventions have been evaluated and reported?
What were the reported objectives, design and outcomes reported in ISS studies?
Studies were included in they: 1) included any type of healthcare professionals and/or healthcare support workers as participants; 2) reported ISS interventions as part or whole of an intervention; 3) were conducted in any healthcare setting including primary or secondary care. The scoping search was limited to OECD countries. Studies which conducted simulation in laboratory, off site or training facilities, were solely focused on pre-registration or undergraduate participants were excluded.
A two-step search strategy was used, the initial search was conducted in September 2018 and updated on 10th December 2020. The search included papers published in the English language from inception to December 2020. Due to time restrictions, we limited the search to three database: Medline, OpenGrey (now archived in the DANS EASY data archive) and Web of Science.
Broad search terms were developed and refined by the study team  which included ‘simulation; drills; simulation training; patient simulation’ AND ‘in situ; clinical care; practice; real world; point of care; workplace’ (Additional file 1). Reference lists from all identified studies were reviewed for additional citations to enhance rigour of the approach. Searches and screening were completed by one reviewer (KE). Full text review was completed by KE and members of the study team. Agreement was reached through group discussion. The process of charting the data was conducted by two reviewers (KE, LB) from included papers using a standardised data extraction form to record characteristics of the included studies and the key information relevant to the review question .
Data were analysed using a descriptive approach, summarising the data and study characteristics. Data were then organised into categories based around the different motivations, concepts and theoretical approaches for ISS as identified in previous reviews [27, 29]. This was an iterative process completed through discussion with the project team and further refined through the stakeholder engagement.
Guidelines: The scoping review was conducted following the methods described by Levac et al. . The protocol development and scoping review reporting was guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist .
Component two stakeholder engagement
Levac et al.  recommend incorporating consultation with stakeholders as a required knowledge translation component of scoping study methodology. We sought expert feedback on the preliminary scoping review findings to build on the evidence and offer a higher level of meaning, content expertise, and perspective to inform the conceptual model [27, 30]. An ISS workshop was held at the Association of Simulated Practice in Healthcare (ASPIH) national conference (2018). As part of the workshop, attendees were presented with the preliminary study findings. A ‘World Café’ method [31, 32] with four × 10 min small group discussions which focused on the ISS categories identified from the preliminary findings. Participants were presented with a series of questions What are the benefits (if any) of ISS over other methods?
What type of enquiry is more suited to ISS and why?
What type of ISS design should be used for different types of enquiry?
How can ISS support psychological safety of participants?
Data from flipcharts, posters and facilitators field notes, were transcribed and summarised.
Attendees from higher education and healthcare provider institutions including multi-professional clinical and managerial staff groups attended the workshop.
Component three: developing a conceptual model of ISS ion healthcare settings.
Data from the scoping search findings, stakeholder discussion and debate were synthesised and developed into a conceptual model of the concepts and theories associated with ISS in healthcare settings to address specific contextual needs. Logic models were then developed to identify short, medium and long-term outcomes that are linked to the key activities of ISS mechanisms .
The results of the three components will now be presented.
Component one: scoping review
The search identified 4237 papers which were assessed for eligibility via title and abstract. Eighty papers were retrieved for full text assessment, following discussion and agreement with the review team, 3 reviews and 35 studies were selected for inclusion (Fig. 1, Additional file 2). Papers were from Australia, Canada, Denmark, France, Hong Kong, Israel, Japan, Netherlands, Norway, Sweden, Switzerland, UK and the US and publication dates ranged from 2008 to 2018.
Systematic reviews of ISS interventions
An umbrella review of simulation-based training for nursing education and practice  included one paper focused on ISS . ISS was found to foster improved competencies related to patient safety and collaborative practice such as interdisciplinary communication and teamwork, as well as identifying and correcting actual clinical safety issues. Rosen et al.  conducted a systematic review of ISS interventions that included 29 papers which were focused mainly on surgical or maternity care. Most studies were rated as low quality. Approaches to design, delivery, and evaluation of ISS were highly variable across studies. Formal needs analysis was rarely used to develop simulations, there was little evidence of formal training or performance management for facilitators and few programmes reported meaningful evaluations of programme effectiveness. However, a positive impact of ISS on learning and organisational performance was demonstrated in a small number of studies. Owei et al.  conducted a systematic review of operating room ISS which included 19 papers, describing its application for a variety of purposes and in a variety of settings, all premised on the potential to offer unique advantages over other types of simulation. One randomized controlled trial (RCT) comparing ISS to off-site simulation found few significant differences. One large-scale outcome study showed improved perinatal outcomes in obstetric care. The authors concluded that although ISS theoretically offers certain advantages over other types of simulation, especially in addressing system-wide or environmental threats, its efficacy has yet to be reliably demonstrated.
Characteristics of studies reporting ISS interventions
The remaining papers reported ISS interventions conducted in maternity care settings, paediatrics, neonatal, trauma and emergency departments, resuscitation response teams, nursing, mental health and primary care settings. Papers reported various methods for evaluating ISS interventions including: RCTs, surveys, focus groups, cohort studies, observation (pre and post intervention studies) and clinical audit. Papers reported ISS interventions lasting from 15 min to 12 h, describing exercises that were announced and unannounced, and providing single or repetitive ISS sessions. Most papers reported that ISS had been developed by clinical educators and senior clinical staff. Various frameworks for designing, conducting and evaluating ISS were utilised, including systems science models, such as the Systems Engineering Initiative for Patient Safety (SEIPS) model , and improvement science models such as the ‘plan, do, study, act’ (PDSA) cycle .
Studies aimed to evaluate ISS on various outcomes (Table 1), including:
Component two: stakeholder workshops
The stakeholder group indicated formative rather than summative ISS assessments have the potential to improve learning and encourage Healthcare Professionals (HCPs) to identity their own training needs. Formal teaching and traditional simulation assessment programmes often fail to prepare staff for the ‘real world’ and ISS may help apply learning and reduce the theory-to-practice gap. However there are no robust strategies or non-technical skills standards to assess behaviours, attitudes and communication in changing complex and dynamic settings. It would be very difficult if not impossible to present all participants with equal opportunities to demonstrate their skills during ISS. It was suggested that ISS assessment interventions should:
aim for realism rather than hyper realism; attempts to exaggerate or exacerbate scenarios should be avoided.
have clear learning objectives; the object of assessment (i.e. individual, team and/or system) should be defined as each requires a different approach and assessment technique.
have objectives informed by training needs analyses which form part of an integrated curricula employing numerous learning and assessment approaches. The ‘gaps’ in learning which ISS can address should be clearly identified.
ensure information including learning resources is standardised and available for staff to access prior to ISS implementation.
ensure ISS is conducted within supportive learning cultures which have well defined supportive and training packages.
provide facilitators with specialist training including peer assessment.
ensure ISS interventions involve creative planning to avoid being continually cancelled in busy departments; consider alternative setting such as staff areas and social spaces.
Stakeholders suggested that ISS has potential to complement traditional investigation approaches but that not all clinical incidents were thought to be appropriate for ISS exploration. Comments provided during the workshop were formed into a series of questions to help HCPs and healthcare educators decide if ISS is an appropriate intervention to aid in the investigation of clinical incidents with a view to foster deeper learning of the factors involved and how these might be mitigated in future (Additional file 3).
Component three: developing a conceptual model of ISS in healthcare settings.
The scoping review findings and stakeholder feedback identified various principles, theories and approaches for ISS in healthcare settings (Fig. 2).
ISS to understand why errors have occurred
ISS to design and testing new equipment, spaces, pathways, systems and procedures
ISS to practice skills and develop competency
ISS to assess, evaluate and improve performance
In-situ simulation: understand why events have occurred
Learning from adverse incidents should move beyond attributing cause wholly to human failings and more towards investigating the role of the system in which humans operate [69,70,71]. Root cause analysis (RCA) investigations promote a systematic approach to investigating serious incidents, although RCA may impede organisational learning as it can restrict explanation to single causes and is often over reliant on individual testimonies or medical notes . Alternative ways of capturing less reductive incident ‘stories’ can encourage reflection and wider organisational learning, considering the interaction between a range of systemic factors which contribute to incidents. ISS may provide a useful mechanism for such inquiry, assisting clinicians to discover potential interacting components and identify additional LSTs [65, 73,74,75].
As part of a wider approach which aims to encourage healthcare organisations to progress from a ‘blame culture’ to a just, learning culture , embedding simulation activities underpinned by Human Factors principles can help to focus on the organisational, procedural and contextual influences on clinical reasoning and actions. Participants of ISS interventions should be encouraged to observe, reflect, ask questions, raise concerns, challenge practices and clarify actions while taking context and complexity into account.
Patient involvement in the design and delivery of simulation training enables the patients’ experience to be expressed and considered. Many ISS scenarios have been developed to reflect ‘real life’ risks drawn from patterns in care, sentinel events, or concern from advocacy groups. However it maybe be possible to work in collaboration families involved in a serious incident to help facilitate learning opportunities and disseminate the findings . This would involve careful attention to local sensitivity; where harm has occurred this of course may be distressing for staff, patients and families involved (particularly where organisations have a culture of appropriating blame and censure). The acceptability of co-designed ‘reconstruction’ via ISS scenarios from care histories should always assess the potential effect on patients and staff involved.
In-situ simulation: design and testing
Tools of safety and complexity science such as Discrete Event Simulation , Cognitive Task Analysis  and general system thinking models have been used to design ISS interventions to enhance patient flow, improve the design of clinical spaces, and identify LSTs within new emergency and obstetric departments. For example, Bender  scripted commonly encountered clinical scenarios to explore the functioning of a new perinatal facility. Scenarios challenged participants to test the new facilities under stressful conditions, focusing on identifying LSTs and making improvements which were more readily adopted into practice as they were driven the clinical teams. Medwid et al.  used ISS to test a new emergency department; numerous LSTs were identified and addressed prior to the department opening. ISS facilitated deliberate practice within the new space and helped orientate staff to the new facility. ISS has also been reported to assist the development and testing of new clinical services and procedures for cardiac services  and stroke thrombolysis pathways . The MHRA guidance for medical devices  highlights that usability testing with representative users in a simulated environment or the actual environment of use can help identify which device features people find easy to use and which cause problems, thus determining whether the device is susceptible to user errors that could cause harm.
Design-driven improvement is a core Human Factors principle which can benefit healthcare organisations [80, 81]. The performance of a process is the emergent property of the whole interacting system which is complex, dynamic and situation specific [82, 83]. The ability to experiment and see what occurs through interactions, attunement and disturbances enables participants to question how things might be done differently, try out various options and consider possible unintended outcomes . ISS can be designed to test the synergy or dissonance between micro and macro factors: task factors, organisational factors, internal environments and external environments . The logic model proposed in Fig. 4 has been developed with reference to the current literature and approaches to ISS design and testing interventions. The model highlights the mechanisms of ISS which would be potentially amenable to standardised approach.
In-situ simulation: practice, developing capability and resilience
Organisational resilience is focused on understanding how healthcare organisations can deliver standardised, replicable and predictable services while embracing inherent variations, disruptions and unexpected events . This involves designing, testing and improving the organisational systems that support rapid and adaptive responses to emergency situations. Deficiencies and LSTs in current systems can be identified and addressed through ISS activities as they enable a systematic examination to provide a realistic picture of work, contextualised in real time and place [7, 55, 62, 86]. ISS scenarios can also support HCPs to develop non-technical skills: task management; teamwork; situation awareness; problem-solving; and decision-making, while testing and probing real-world organisational systems [4, 43, 85, 87,88,89]. ISS to assist teams prepare, rehearse and practice for low frequency, high impact events was the most reported ISS activity included in the review and was often underpinned by a combination of established learning theories (Fig. 5).
Skills are developed through repetition. Learning and behaviour change occurs through feedback from the simulation activity, interaction between the task, environment, and the team. ISS provides opportunity for teams to identify solutions and take action to introduce and reinforce changes . Video playback can support behavioural learning alongside a discussion of strategies, alternative approaches, personal experiences and emotional aspects. Teams and individuals can be supported to identify gaps in knowledge, challenge current procedures and identify and address system weaknesses.
Preconceptions are explored, and new or unexpected events are presented via the simulation activity to challenge precognitions . Questioning and reflection help participants acquire deeper understanding, develop problem solving skills and new insights . ISS enables learning from making mistakes in a way that would be inconceivable with actual patients; ‘thinking aloud’ helps participants reflect on why they took a particular course of action and present an opportunity to correct their actions [84, 93, 94]. ISS activities should provide a concrete experience, debrief with reflexive observation and conceptualisation, ideally followed by a second scenario for active experimentation . Knowledge transfer is optimal when the learning environment matches the environment in which it will be applied . ISS can help develop and maintain situational awareness , for example, awareness of vital signs, medication requirements, actions of other team members and equipment function. Comprehension of a current situation based on these elements, understanding their significance within the environment, and forming a holistic picture helps anticipate future actions and consequences .
Learning through observing others and developing a shared understanding of roles and processes within a team which result in the desired outcomes. ISS enables interdisciplinary team practice while maintaining interaction with the environmental and system factors present during actual patient care events . Individuals and teams explore the social processes involved in building shared expectations, establishing patterns of collective working and building trust between multidisciplinary teams . An open and safe context encourages professionals from different backgrounds to acknowledge their strengths and address their weaknesses in a respectful and trustful manner [8, 85].
In-situ simulation: assess, evaluate and improve performance
Assessing competency through simulation is well established in the military, nuclear and aviation industries and is used in healthcare to assess clinical competency via objective structured clinical examinations (OSCE). Brunette and Thibodeau-Jarry  suggest that simulation, through the application of mastery learning theory can be used to formatively assess competency in clinical environments. Learners are required to achieve a level of mastery in particular essential skills and knowledge before progressing onto new or more advanced practice. Learners progress at their own pace and are provided with opportunities for deliberate practice via simulation. Again, ISS enables participants to operate and be assessed in a familiar work environment .
Miller’s pyramid for assessing clinical competence distinguishes between four different levels of competence: knows; knows how; shows how; and does. The level ‘does’ is described as the most accurate way to assess competence in actual clinical practice . However, assessment in actual clinical practice has the potential to distress patients and can be problematic in terms of controlling variables such as task difficulty . Sørensen et al.  stress the difference between simulation-based training and simulation-based assessment (SBA). In SBA, participants need to be well informed about the proposed activity and know what will be expected of them . Careful attention needs to be paid to creating a safe learning environment. Validated metrics and standards need to be developed for individuals and teams. Strategies to assess non-technical skills during ISS have been developed for medical students, obstetrics and anaesthesiology and could be refined and adopted for other clinical specialties [100,101,102].
Griswold et al.  identify that for clinical procedures with clear chains of action and well-defined processes and standards, summative assessment via ISS is much simpler than in more “dynamic, multifactorial practices in which cognitive, procedural, and communication skills are simultaneously applied in a team environment” (Griswold et al. 2018, page 170). Measurement methods for assessing competencies involved in complex care processes are less well-defined, and further complicated when individual performance needs to be isolated from the wider team. Concepts such as ‘effective communication’ are subject to interpretation, and clinical outcomes may be attributed to concepts such as teamwork and coordination in addition to individual clinical skills and knowledge . Criterion standards and benchmarks of quality performance need to be defined to reliably and accurately capture the individual performance which is linked to relevant outcomes. Strategies to define and control for difference in skill mix, staffing and capacity pressures are also required.
Simulation for training has been reported as effective when instructional features, such as the event-based approach to training (EBAT) are embedded within the simulation . EBAT has been used in aviation and military environments and relies on the “a priori” embedding of multiple events into the scenario at different time intervals. These events are designed to enable participants to exhibit competencies and measure performance. Scenario development can be facilitated by performing a Cognitive Task Analysis to identify the cues expected to be used to perform complex tasks . Data collection should capture ‘moment-to moment’ actions and behaviours to identify how performance can be improved.
The scoping review sought to identify and categorise ISS approaches within a wide range of healthcare settings, however only three databases were searched. Selection and screening were completed by a single reviewer (KE) and the review protocol was not registered. The included studies were conducted across thirteen countries and the cultural context in which ISS was conducted was not explored within the review. Stakeholder feedback which were used to further inform the development of the conceptual model may not represent the whole picture of the concepts and mechanisms of ISS being conducted in healthcare and health education settings. However, we consider that the scoping review and conceptual model have highlighted key characteristics related to ISS and thus provided a useful starting point to develop more specific questions to addressed by a systematic review or primary research.
This paper provides an overview of the application of the use of ISS in healthcare settings. The recent literature highlights the heterogeneity in ISS objectives and the complex delivery landscape which has resulted in a lack of an integrated ISS approach across healthcare organisations. Most reported ISS interventions provide little evidence of formal development processes and lack validated outcome measures. ISS in healthcare is often underpinned by Human Factors (HF) principles which overlap and synergise with other approaches, methods and theories including non-technical skills development, quality improvement (QI) methods, and systems modelling. Many existing models and frameworks exist within a wide ISS curriculum as part of a general approach to simulation training. This can result in a lack of clear guidance to inform ISS designs. A conceptual model has been provided to inform discussion and debate about the objectives, feasibility and usefulness of ISS interventions to guide clinicians and educators. We have set out the learning mechanisms intrinsic within ISS and suggest the context in which these mechanisms can be actualised. We have highlighted the potential for ISS to improve the design of clinical spaces and equipment, develop team performance and healthcare systems resilience, and support clinical investigation and competency assessment. An ISS approach presents distinctive advantages to explore and improve clinical team and organisational functioning. In addition, the ability to address the practice-theory gap makes ISS an attractive approach for educators, managers and policymakers as part of quality and safety improvement strategies. However there are limitations associated with ISS, such as the impact on the clinical setting, the provision of an open learning culture and availability of relevant expertise. We strongly recommend that ISS is not introduced without due consideration of the specific objectives and learning needs it is proposed to address. Effectiveness of ISS has not yet been established and further research is required to assess the specific effect of particular ISS designs on clinical outcomes, learning outcomes, team performance, non-technical skill development, acceptability and perceived benefit. We encourage researchers, clinicians and educators to work collaboratively to rigorously design, develop, evaluate ISS interventions and disseminate the findings to further inform the evidence base. We recommend future systematic reviews are conducted to assess ISS intervention effectiveness to guide clinicians, researchers and educators to develop effective ISS interventions and provide useful guidance as they continue to address various clinical concerns by ISS interventions in dynamic settings.
Availability of data and materials
All data generated or analysed during this study are included in this published article [and its supplementary information files].
Guise J-M, Mladenovic J. In situ simulation: Identification of systems issues. Semin Perinatol. 2013;37(3):161–5.
Pucher P, Tamblyn R, Boorman D, Dixon-Woods M, Donaldson L, Draycott T, Forster A, Nadkarni V, Power C, Sevdalis N, et al. Simulation research to enhance patient safety and outcomes: recommendations of the simnovate patient safety domain group. BMJ Simul Technol Enhanc Learn. 2017;3:S3–7.
Kurup V, Matei V, Ray J. Role of in-situ simulation for training in healthcare. Current Opinion in Anesthesiology. 2017;30(6):755–60. https://doi.org/10.1097/ACO.0000000000000514.
Kelsey NC, Claus S. Embedded, in situ simulation improves ability to rescue. Clin Simul Nurs. 2016;12(11):522–7.
Lavelle M, Attoe C, Tritschler C, Cross S. Managing medical emergencies in mental health settings using an interprofessional in-situ simulation training programme: a mixed methods evaluation study. Nurse Educ Today. 2017;59:103–9.
Bender GJ. In situ simulation for systems testing in newly constructed perinatal facilities. Semin Perinatol. 2011;35(2):80–3.
Knight P, MacGloin H, Lane M, Lofton L, Desai A, Haxby E, Macrae D, Korb C, Mortimer P, Burmester M. Mitigating Latent Threats Identified through an Embedded In Situ Simulation Program and Their Comparison to Patient Safety Incidents: A Retrospective Review. Front Pediatr. 2018;5(281). https://doi.org/10.3389/fped.2017.00281.
Brandstorp H, Halvorsen PA, Sterud B, Haugland B, Kirkengen AL. Primary care emergency team training in situ means learning in real context. Scand J Prim Health Care. 2016;34:295–303.
Fan M, Petrosoniak A, Pinkney S, Hicks C, White K, Almeida APSS, Campbell D, McGowan M, Gray A, Trbovich P. Study protocol for a framework analysis using video review to identify latent safety threats: trauma resuscitation using in situ simulation team training (TRUST). BMJ Open. 2016;6(11):e013683.
Choi GYS, Wan WTP, Chan AKM, Tong SK, Poon ST, Joynt GM. Preparedness for COVID-19: in situ simulation to enhance infection control systems in the intensive care unit. Br J Anaesth. 2020;125(2):e236–9.
Fregene TE, Nadarajah P, Buckley JF, Bigham S, Nangalia V. Use of in situ simulation to evaluate the operational readiness of a high-consequence infectious disease intensive care unit. Anaesthesia. 2020;75(6):733–8.
Dharamsi A, Hayman K, Yi S, Chow R, Yee C, Gaylord E, Tawadrous D, Chartier LB, Landes M. Enhancing departmental preparedness for COVID-19 using rapid-cycle in-situ simulation. J Hosp Infect. 2020;105(4):604–7.
Lie SA, Wong LT, Chee M, Chong SY. Process-Oriented In Situ Simulation Is a Valuable Tool to Rapidly Ensure Operating Room Preparedness for COVID-19 Outbreak. Simul Healthc. 2020;15(4):225–33. https://doi.org/10.1097/SIH.0000000000000478.
Muret-Wagstaff SL, Collins JS, Mashman DL, Patel SG, Pettorini K, Rosen SA, Shaffer VO, Sumler ML, Sweeney JF, Sharma J. In Situ simulation enables operating room agility in the COVID-19 pandemic. Ann Surg. 2020;272(2):e148–50.
Haji FA, Da Silva C, Daigle DT, Dubrowski A. From bricks to buildings: adapting the Medical Research Council framework to develop programs of research in simulation education and training for the health professions. Simul Healthc. 2014;9(4):249–59.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008;337: a1655.
Posner GD, Clark ML, Grant VJ. Simulation in the clinical setting: towards a standard lexicon. Adv Simul. 2017;2(1):15.
Rosen MA, Hunt EA, Pronovost PJ, Federowicz MA, Weaver SJ. In situ simulation in continuing education for the health care professions: a systematic review. J Contin Educ Health Prof. 2012;32(4):243–54.
Owei L, Neylan CJ, Rao R, Caskey RC, Morris JB, Sensenig R, Brooks AD, Dempsey DT, Williams NN, Atkins JH, et al. In situ operating room-based simulation: a review. J Surg Educ. 2017;74(4):579–88.
Goldshtein D, Krensky C, Doshi S, Perelman VS. In situ simulation and its effects on patient outcomes: a systematic review. BMJ Simul Technol Enhanc Learn. 2020;6(1):3–9.
Armenia S, Thangamathesvaran L, Caine AD, King N, Kunac A, Merchant AM. The role of high-fidelity team-based simulation in acute care settings: a systematic review. Surg J (N Y). 2018;4(3):e136–51.
Zetner DB, Petersen I, Konge L, Thinggaard E. Training cesarean section: a scoping review. Simul Healthc. 2019;14(4):264–70.
Brogaard L, Glerup Lauridsen K, Løfgren B, Krogh K, Paltved C, Boie S, Hvidman L. The effects of obstetric emergency team training on patient outcome: a systematic review and meta-analysis. Acta Obstet Gynecol Scand. 2022;101(1):25–36.
Maggio LA, Larsen K, Thomas A, Costello JA, Artino AR Jr. Scoping reviews in medical education: a scoping review. Med Educ. 2021;55(6):689–700.
Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18(1):143.
Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, Levac D, Ng C, Sharpe JP, Wilson K, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16(1):15.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5(1):69.
Peters MD, Godfrey CM, Khalil H, McInerney P, Parker D, Soares CB. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6. https://doi.org/10.1097/XEB.0000000000000050.
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.
Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
Schiele H, Krummaker S, Hoffmann P, Kowalski R. The “research world café” as method of scientific enquiry: Combining rigor with relevance and speed. J Bus Res. 2022;140:280–96.
Broom M, Brady B, Kecskes Z, Kildea S. World Café Methodology engages stakeholders in designing a Neonatal Intensive Care Unit J Neonatal Nurs. 2013;19(5):253–58. ISSN 1355-1841. https://doi.org/10.1016/j.jnn.2012.12.002.
Using logic models in evaluation [https://www.strategyunitwm.nhs.uk/publications/using-logic-models-evaluation]
Cantrell MA, Franklin A, Leighton K, Carlson A. The evidence in simulation-based learning experiences in nursing education and practice: an umbrella review. Clin Simul Nurs. 2017;13(12):634–67.
Villemure C, Tanoubi I, Georgescu LM, Dubé JN, Houle J. An integrative review of in situ simulation training: Implications for critical care nurses. Can J Crit Care Nurs. 2016;27(1):22–31.
Barbeito A, Bonifacio A, Holtschneider M, Segall N, Schroeder R, Mark J. In situ simulated cardiac arrest exercises to detect system vulnerabilities. Simul Healthc. 2015;10(3):154–62.
Deming WE. Out of the crisis, 1986. Cambridge: Massachusetts Institute of Technology Center for Advanced Engineering Study xiii; 1991:507.
Ben-Ari M, Chayen G, Steiner IP, Schinasi DA, Feldman O, Shavit I. The effect of in situ simulation training on the performance of tasks related to patient safety during sedation. J Anesth. 2018;32(2):300–4.
Fialkow MF, Adams CR, Carranza L, Golden SJ, Benedetti TJ, Fernandez R. An in situ standardized patient-based simulation to train postpartum hemorrhage and team skills on a labor and delivery unit. Simul Healthc. 2014;9(1):65–71.
Herbers MD, Heaser JA. Implementing an in situ mock code quality improvement program. Am J Crit Care. 2016;25(5):393–9.
Jung D, Carman M, Aga R, Burnett A. Disaster preparedness in the emergency department using in situ simulation. Adv Emerg Nurs J. 2016;38(1):56–68.
Kurosawa H, Ikeyama T, Achuff P, Perkel M, Watson C, Monachino A, Remy D, Deutsch E, Buchanan N, Anderson J, et al. A randomized, controlled trial of in situ pediatric advanced life support recertification (“pediatric advanced life support reconstructed”) compared with standard pediatric advanced life support recertification for ICU frontline providers*. Crit Care Med. 2014;42(3):610–8.
Rubio-Gurung S, Putet G, Touzet S, Gauthier-Moulinier H, Jordan I, Beissel A, Labaune JM, Blanc S, Amamra N, Balandras C, et al. In situ simulation training for neonatal resuscitation: an RCT. Pediatrics. 2014;134(3):e790-797.
Siegel NA, Kobayashi L, Dunbar-Viveiros JA, Devine J, Al-Rasheed RS, Gardiner FG, Olsson K, Lai S, Jones MS, Dannecker M, et al. In situ medical simulation investigation of emergency department procedural sedation with randomized trial of experimental bedside clinical process guidance intervention. Simul Healthc. 2015;10(3):146–53.
Sørensen JL, Lottrup P, van der Vleuten C, Andersen KS, Simonsen M, Emmersen P, Rosthøj S, Ottesen B. Unannounced in situ simulation of obstetric emergencies: staff perceptions and organisational impact. Postgrad Med J. 2014;90(1069):622–9.
Marshall NE, Vanderhoeven J, Eden KB, Segel SY, Guise JM. Impact of simulation and team training on postpartum hemorrhage management in non-academic centers. J Matern Fetal Neonatal Med. 2015;28(5):495–9.
Yager P, Collins C, Blais C, O’Connor K, Donovan P, Martinez M, Cummings B, Hartnick C, Noviski N. Quality improvement utilizing in-situ simulation for a dual-hospital pediatric code response team. Int J Pediatr Otorhinolaryngol. 2016;88:42–6.
Amiel I, Simon D, Merin O, Ziv A. Mobile in situ simulation as a tool for evaluation and improvement of trauma treatment in the emergency department. J Surg Educ. 2016;73(1):121–8.
Härgestam M, Lindkvist M, Jacobsson M, Brulin C, Hultin M. Trauma teams and time to early management during in situ trauma team training. BMJ Open. 2016;6(1):e009911.
Patterson MD, Geis GL, Falcone RA, LeMaster T, Wears RL. In situ simulation: detection of safety threats and teamwork training in a high risk emergency department. BMJ Qual Saf. 2013;22(6):468–77.
Zimmermann K, Holzinger IB, Ganassi L, Esslinger P, Pilgrim S, Allen M, Burmester M, Stocker M. Inter-professional in-situ simulated team and resuscitation training for patient safety: description and impact of a programmatic approach. BMC Med Educ. 2015;15:189–189.
Geis GL, Pio B, Pendergrass TL, Moyer MR, Patterson MD. Simulation to assess the safety of new healthcare teams and new facilities. Simul Healthc. 2011;6(3):125–33.
Gibbs K, DeMaria S, McKinsey S, Fede A, Harrington A, Hutchison D, Torchen C, Levine A, Goldberg A. A novel in situ simulation intervention used to mitigate an outbreak of methicillin-resistant staphylococcus aureus in a neonatal intensive care unit. J Pediatr. 2018;194:22-27.e25.
Kobayashi L, Dunbar-Viveiros JA, Devine J, Jones MS, Overly FL, Gosbee JW, Jay GD. Pilot-phase findings from high-fidelity In Situ medical simulation investigation of emergency department procedural sedation. Simul Healthc. 2012;7(2):81–94.
O’Leary F, McGarvey K, Christoff A, Major J, Lockie F, Chayen G, Vassiliadis J, Wharton S. Identifying incidents of suboptimal care during paediatric emergencies-an observational study utilising in situ and simulation centre scenarios. Resuscitation. 2014;85(3):431–6.
Theilen U, Fraser L, Jones P, Leonard P, Simpson D. Regular in-situ simulation training of paediatric Medical Emergency Team leads to sustained improvements in hospital response to deteriorating patients, improved outcomes in intensive care and financial savings. Resuscitation. 2017;115:61–7.
Lutgendorf MA, Spalding C, Drake E, Spence D, Heaton JO, Morocco KV. Multidisciplinary in situ simulation-based training as a postpartum hemorrhage quality improvement project. Mil Med. 2017;182(3):e1762–6.
Knight LJ, Gabhart JM, Earnest KS, Leong KM, Anglemyer A, Franzon D. Improving code team performance and survival outcomes: implementation of pediatric resuscitation team training. Crit Care Med. 2014;42(2):243–51.
Gundrosen S, Solligård E, Aadahl P. Team competence among nurses in an intensive care unit: the feasibility of in situ simulation and assessing non-technical skills. Intensive Crit Care Nurs. 2014;30(6):312–7.
Miller D, Crandall C, Washington C 3rd, McLaughlin S. Improving teamwork and communication in trauma care through in situ simulations. Acad Emerg Med. 2012;19(5):608–12.
Sørensen JL, van der Vleuten C, Rosthøj S, Østergaard D, LeBlanc V, Johansen M, Ekelund K, Starkopf L, Lindschou J, Gluud C, et al. Simulation-based multiprofessional obstetric anaesthesia training conducted in situ versus off-site leads to similar individual and team outcomes: a randomised educational trial. BMJ Open. 2015;5(10):e008344.
Wheeler DS, Geis G, Mack EH, LeMaster T, Patterson MD. High-reliability emergency response teams in the hospital: improving quality and safety using in situ simulation training. BMJ Qual Saf. 2013;22(6):507–14.
Ventre KM, Barry JS, Davis D, Baiamonte VL, Wentworth AC, Pietras M, Coughlin L, Barley G. Using in situ simulation to evaluate operational readiness of a children’s hospital-based obstetrics unit. Simul Healthc. 2014;9(2):102–11.
Medwid K, Smith S, Gang M. Use of in-situ simulation to investigate latent safety threats prior to opening a new emergency department. Safety Sci. 2015;77:19–24.
Yajamanyam PK, Sohi D. In situ simulation as a quality improvement initiative. Arch Dis Child Educ Pract Ed. 2015;100(3):162.
Chen PP, Tsui NT, Fung AS, Chiu AH, Wong WC, Leong HT, Lee PS, Lau JY. In-situ medical simulation for pre-implementation testing of clinical service in a regional hospital in Hong Kong. Hong Kong Med J. 2017;23(4):404–10.
Kobayashi L, Dunbar-Viveiros JA, Sheahan BA, Rezendes MH, Devine J, Cooper MR, Martin PB, Jay GD. In situ simulation comparing in-hospital first responder sudden cardiac arrest resuscitation using semiautomated defibrillators and automated external defibrillators. Simul Healthc. 2010;5(2):82–90.
Kobayashi L, Parchuri R, Gardiner FG, Paolucci GA, Tomaselli NM, Al-Rasheed RS, Bertsch KS, Devine J, Boss RM, Gibbs FJ, et al. Use of in situ simulation and human factors engineering to assess and improve emergency department clinical systems for timely telemetry-based detection of life-threatening arrhythmias. BMJ Qual Saf. 2013;22(1):72–83.
Wallace B, Ross A. Beyond Human Error. Boca Raton, FL: CRC Press; 2006.
Leveson N, Samost A, Dekker S, Finkelstein S, Raman J. A systems approach to analyzing and preventing hospital adverse events. J Patient Saf. 2020;16(2):162–7.
Vosper H, Hignett S, Bowie P. Twelve tips for embedding human factors and ergonomics principles in healthcare education. Med Teach. 2018;40(4):357–63.
Waterson P, Catchpole K. Human factors in healthcare: welcome progress, but still scratching the surface. BMJ Qual Saf. 2016;25(7):480–4.
Commission on Education and Training for Patient Safety. Improving Safety Through Education and Training. London: Health Education England; 2016. https://www.hee.nhs.uk/sites/default/files/documents/Improving%20safety%20through%20education%20and%20training.pdf.
Simms ER, Slakey DP, Garstka ME, Tersigni SA, Korndorffer JR. Can simulation improve the traditional method of root cause analysis: a preliminary investigation. Surgery. 2012;152(3):489–97.
Slakey DP, Simms ER, Rennie KV, Garstka ME, Korndorffer JR Jr. Using simulation to improve root cause analysis of adverse surgical outcomes. Int J Qual Health Care. 2014;26(2):144–50.
Zhang X. Application of discrete event simulation in health care: a systematic review. BMC Health Serv Res. 2018;18(1):687.
Potworowski G, Green L. Cognitive Task Analysis: Methods to Improve Patient-Centered Medical Home Models by Understanding and Leveraging its Knowledge Work. Rockville: Agency for Healthcare Research and Quality; 2013.
Combes J. 0121 Sequence simulation ‘the hyper acute stroke thrombolysis pathway. BMJ Simulation and Technology Enhanced Learning. 2015;1.
Medicinies and Healthcare Products Regulatory Agency. Human Factors and Usability Engineering – Guidance for Medical Devices Including Drug-device Combination Products. London: Medicinies and Healthcare Products Regulatory Agency; 2017.
Dul J, Bruder R, Buckle P, Carayon P, Falzon P, Marras WS, Wilson JR, van der Doelen B. A strategy for human factors/ergonomics: developing the discipline and profession. Ergonomics. 2012;55(4):377–95.
Holden RJ, Carayon P, Gurses AP, Hoonakker P, Hundt AS, Ozok AA, Rivera-Rodriguez AJ. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics. 2013;56(11):1669–86.
Dekker SWA, Breakey H. ‘Just culture:’ Improving safety by achieving substantive, procedural and restorative justice. Saf Sci. 2016;85:187–93.
Wilson JR. Fundamentals of systems ergonomics/human factors. Appl Ergon. 2014;45(1):5–13.
Lefroy J, Yardley S. Embracing complexity theory can clarify best practice frameworks for simulation education. Med Educ. 2015;49(4):344–6.
Macrae C, Draycott T. Delivering high reliability in maternity care: In situ simulation as a source of organisational resilience. Safety Sci. 2016;117:490–500.
Zimmermann K, Holzinger IB, Ganassi L, Esslinger P, Pilgrim S, Allen M, Burmester M, Stocker M. Inter-professional in-situ simulated team and resuscitation training for patient safety: description and impact of a programmatic approach. BMC Med Educ. 2015;15(1):189.
Boet S, Bould MD, Fung L, Qosa H, Perrier L, Tavares W, Reeves S, Tricco AC. Transfer of learning and patient outcome in simulated crisis resource management: a systematic review. Can J Anaesth. 2014;61(6):571–82.
Gibbs K, DeMaria S, McKinsey S, Fede A, Harrington A, Hutchison D, Torchen C, Levine A, Goldberg A. A novel in situ simulation intervention used to mitigate an outbreak of methicillin-resistant <em>staphylococcus aureus</em> in a neonatal intensive care unit. J Pediatr. 2018;194:22-27.e25.
Murphy M, Curtis K, McCloughen A. What is the impact of multidisciplinary team simulation training on team performance and efficiency of patient care? An integrative review. Australas Emerg Nurs J. 2016;19(1):44–53.
Ross EM, Deaton T, Hurst N, Siefert J. Operational point-of-care ultrasound review: low-cost simulators and resources for advanced prehospital providers. J Spec Oper Med. 2015;15(1):71–8.
Fenwick T, Dahlgren MA. Towards socio-material approaches in simulation-based education: lessons from complexity theory. Med Educ. 2015;49(4):359–67.
Bradley P, Postlethwaite K. Simulation in clinical learning. Med Educ. 2003;37(s1):1–5.
Brunette V, Thibodeau-Jarry N. Simulation as a tool to ensure competency and quality of care in the cardiac critical care unit. Can J Cardiol. 2017;33(1):119–27.
Kneebone RL, Scott W, Darzi A, Horrocks M. Simulation and clinical practice: strengthening the relationship. Med Educ. 2004;38(10):1095–102.
Stocker M, Burmester M, Allen M. Optimisation of simulated team training through the application of learning theories: a debate for a conceptual framework. BMC Med Educ. 2014;14(1):69.
Endsley MR. Toward a theory of situation awareness in dynamic systems. Hum Factors. 1995;37(1):32–64.
Roussin CJ, Weinstock P: SimZones: An Organizational Innovation for Simulation Programs and Centers. Academic Medicine 2017, 92(8).
Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63-67.
Sørensen JL, Østergaard D, LeBlanc V, Ottesen B, Konge L, Dieckmann P, Van der Vleuten C. Design of simulation-based medical education and advantages and disadvantages of in situ simulation versus off-site simulation. BMC Med Educ. 2017;17(1):20.
Bracco F, Masini M, De Tonetti G, Brogioni F, Amidani A, Monichino S, Maltoni A, Dato A, Grattarola C, Cordone M, et al. Adaptation of non-technical skills behavioural markers for delivery room simulation. BMC Pregnancy Childbirth. 2017;17(1):89.
Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R. Anaesthetists’ Non-Technical Skills (ANTS): evaluation of a behavioural marker system. Br J Anaesth. 2003;90(5):580–8.
MacCrossan M, Hamilton A, Kerrins J, Tallentire V. 15 Non-technical skills: developing a behavioural marker system to assess medical student performance within high fidelity simulation. BMJ Simul Technol Enhanc Learn. 2016;2(Suppl 1):A26–7.
Griswold S, Fralliccardi A, Boulet J, Moadel T, Franzen D, Auerbach M, Hart D, Goswami V, Hui J, Gordon JA. Simulation-based education to ensure provider competency within the health care system. Acad Emerg Med. 2018;25(2):168–76.
Salas E, Burke CS. Simulation for training is effective when. Qual Saf Health Care. 2002;11(2):119–20.
The project was commissioned by Health Education England, Midlands and East.
Ethics approval and consent to participate
Consent for publication
Not applicable as all images within the manuscript were created by the research team.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Baxendale, B., Evans, K., Cowley, A. et al. GENESISS 1—Generating Standards for In-Situ Simulation project: a scoping review and conceptual model. BMC Med Educ 22, 479 (2022). https://doi.org/10.1186/s12909-022-03490-9
- In-situ simulation
- Simulation-based education
- Clinical training
- Simulated practice
- Health professions