To engage large numbers of students in a stressful clinical environment with time pressures of managing trauma patients, a mass casualty simulation was utilised.
Participants
Second and fourth year medical students (4 year graduate program) and final year nursing students (2 year graduate program and 3 year undergraduate program) from a large Australian university were invited to participate. Students were recruited via an email providing details about the study. Ethical approval was granted by the University Human Research Ethics Committee (2014/425 and 2014/697) and all students provided written, informed consent prior to commencing the study.
Instrument development
The 17-item SATTS questionnaire used a 7-point Likert scale ranging from poor (scored as 1), to excellent (scored as 7). Weller et al. [19, 20] provided descriptors for each item, used to assist scoring. See Additional file 1.
The SATTS questionnaire was adapted from the 23-item teamwork tool developed for critical care teams (doctors and nurses) (Weller et al. [19, 20]). The tool has 3 factors with constructs: leadership and team co-ordination, sharing situational information, and mutual performance monitoring. In their psychometric analysis of the teamwork tool, Weller et al. [19] found that 20 items were associated with the three constructs, with three items not loading against any factor. For the SATTS, 12 items were retained from Weller et al. tool, including modification of two leadership-specific items in the leadership and team co-ordination construct. For these two items, the word ‘leadership’ was removed, as example, ‘the leader’s plan for treatment was communicated to the team’, was modified to ‘a plan for treatment was communicated to the team’ as leadership was considered to be an advanced teamwork attribute [1]. The remaining seven items were deemed too clinically complex for student-led teamwork. Four new simpler items related to teamwork communication and informational sharing were included, as these have been shown to be critical to teamwork [13]. A final item that rated overall teamwork was also included to provide students with a general rating of teamwork, although this has not been included in the analyses. These new items were developed by teamwork experts who considered a range of items and assessed their suitability and relevance for students and the context of the planned simulation activity.
The simulations
Two full-scale simulation scenarios were constructed with support from expert simulation facilitators. On two separate days, two mixed cohorts (medical and nursing students) were immersed into mass casualty scenarios designed to provide novel and challenging situations that students had not encountered previously in clinical practice. The two simulations were designed to have similar levels of difficulty (patient casualties) and support services (simulation facilitators who were emergency nurses and physicians, and paramedics). Student teams were first responders to events where treatment of numerous casualties was required. The SATTS were administered to students immediately following the simulation activities and they then attended a debriefing session.
Simulation activity one
A disaster scene replicating building collapses from the 2011 Christchurch earthquake in New Zealand in which 185 people died was used. The teaching auditorium was constructed of eight disaster clusters each containing four patients over a total space of approximately 600 m2. To augment physical and psychological fidelity, video footage of the earthquake was played at the outset and loud sirens were broadcast to imitate emergency services. This had been successfully piloted the previous year with a voluntary cohort of 117 s year medical students [21].
All students were provided a 15-min briefing outlining the scenario, assembled into groups of 4–5 students, and entered the disaster zone. Standardised patients were played by medical students who had injuries marked on their bodies with large adhesive stickers.
Student teams were required to assess, provide first-line treatment and later, triage information to emergency services when they arrived. The scenario unfolded over 50 min and patient observations were added via a large screen as the scenario progressed. An expert simulation facilitator observed four student teams simultaneously and provided limited cues about the patient’s injuries and management required.
Simulation activity two
The second scenario consisted of a 21st birthday party where the roof of the hall collapsed falling onto guests below, injuring 28 people. Similar to simulation one, a large teaching hall was used to recreate the disaster scene with physical props including debris (bricks, plaster board), party wear, tables and chairs. Standardised patients who acted as casualties were played by faculty members and health science students with moulage applied to create mock injuries.
At the beginning of the simulation all students watched a pre-recorded short video handover (<90 s) from the scene commander of the incident which included information on the number of casualties and hazards at the scene. Students were allocated to groups of five consisting of four final year nursing students and one final year medical student. Each student team was required to attend to two injured patients. Simulation facilitators were available within the scenario to assist student teams with clinical information related to physical deteriorations and improvements in the casualties’ health status and to ensure safety. The student teams’ main task was to undertake assessment and early interventions to prevent further clinical deterioration of disaster patients. After 30 min, teams were approached by a paramedic and told that they are able to transfer patients to hospital and teams were required to determine which patient to transfer.
Debriefing sessions
After scenario one, students reassembled into their teams and participated in a 30-min detailed debriefing and feedback with their expert facilitators. The aim of the debriefing was to focus on the students’ experiences and behaviours on teamwork and to highlight components of team interactions. This was followed by a full-cohort debriefing that generalised the simulation experience to the major concepts associated with teamwork and human factors.
At the end of the second simulation activity, students attended a 10-min facilitated debrief which provided an opportunity for students to discuss emotional experiences during the simulation. A whole-group facilitator-led debrief followed in which a panel of emergency specialist experts (physicians and nurses) provided guidance on the teamwork management of the casualty presentations and real-world experience-based perspectives.
Statistical analysis
All data are presented as means and standard deviations (±SD). Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to determine questionnaire factor structure. Factor analysis has been used extensively in questionnaire development to reduce the number of items to composite variables, known as factors. The analysis determines the intrinsic dimensions which are found between the measured variables and latent constructs. Factor analysis provides evidence of construct validity [22, 23]. Exploratory factor analysis is used typically when researchers do not have predetermined expectations of the number of variables in each factor.
Principal Component Analysis, using Principal Axis Factoring with varimax rotation, was used to investigate common variance in the questionnaire. Items with an inter-item correlation less than 0.30 were removed. Kaiser-Meyer-Olkin measure of sampling adequacy was determined with scores greater than 0.5 considered sufficient. Only eigenvalues greater than 1.0 were retained. Any factor coefficient greater than 0.4 was kept for interpretation of the factor structure. Corrected-item total correlation (the degree to which each item correlates with the total score) was performed to identify items that are problematic and need to be revised or discarded. Items with cross factor loadings were removed to improve factor structure.
The recommended ratio of cases to variables is 5:1 if data are normally distributed and 10:1 if not [23]. This study met the criteria for >10:1 [24]. Cronbach alpha was used to determine internal consistency across items. Cronbach alpha of greater than 0.70 was deemed to be acceptable reliability coefficient for internal consistency of the tool. Exploratory factor analysis was performed using SPSS Version 20 (SPSS Inc., Chicago, IL, USA).
In contrast to EFA, CFA is used to test a model assumption of the number of factors and the degree of fit [25]. Confirmatory factor analysis reports several indices that determine the model acceptability. The chi-square goodness of fit test (χ2) and the goodness of fit index (GFI). The GFI is used to determine differences between observed and predicted covariance matrices and should approach one. The root-mean-square error of approximation (RMSEA) is considered suitable when in the range of 0 to 1. The other indices, the comparative fit index (CFI) and Tucker-Lewis index (TLI) are deemed acceptable when close to 0.95 when RMSEA values are near 0.06 [26]. Confirmatory factor analysis (AMOS Version 22) was undertaken to estimate the model fit of the factor structure as identified by the EFA.