To explore the possible benefit of introducing live mission debriefing as a proxy for the missed in-situ simulation and debriefing, we used a mixed-methods design [6,7,8]. Our triangulation included a prospective observational data collection to quantify and describe the number and types of intervention (i.e. simulation or live mission observation), a questionnaire to collect the facilitators’ and crews’ immediate experience of the training, and interviews for in-depth information about the experience. In our study, all data collection methods were given the same priority, although used partly in parallel and partly in a sequential manner.
Norwegian HEMS system
The Norwegian HEMS is a national service funded by the government. At the time of the project, two private companies were contracted to provide helicopters, pilots and HEMS technical crew members (HTCM). The standard crew configuration consists of a certified anaesthesiologist, an HTCM and a pilot. One base also includes an anaesthetic nurse in the crew at the time of the study. All HEMS bases respond to primary medical and trauma missions, day and night, and perform interhospital transports and search and rescue (SAR) missions. The organisation of the Norwegian HEMS has been described in detail in a previous article [9].
Study population
To achieve a representative sample of HEMS bases in Norway for our study, we recruited three bases with different mission profiles for the adapted simulation program. The base at Lørenskog covers a densely populated area and is the busiest HEMS base in Norway in terms of the number of missions. The base at Ål is located in the mountain region of South Norway with a low population density. The base at Ålesund is located in South West Norway’s coastal region with a mixed population density and is the only HEMS base in Norway with a 4- crew configuration. In the study period, the flight operation at Ålesund HEMS was run by a different company than at the other two bases. All bases operate 24 h a day, all week, all year around. The number of annual missions for the bases at Lørenskog (two helicopters), Ålesund and Ål are approximately 1900, 650 and 600, respectively [10].
Study design
The study was conducted in two stages. The first stage was a prospective multicentre study with observational data collected between May 19th, 2016, and May 18th, 2017. In-situ simulation training was offered as described in a previous study from an earlier stage of the project [3, 9]. The facilitators developed scenarios tailored to each base depending on each base mission profile. The in-situ simulation could be located indoor or outdoor and would involve all members of the crew. The training at each HEMS base was run by one or two experienced senior HEMS physicians selected by the lead physician at the HEMS base to be facilitators [9]. The facilitators were trained according to the EUSim simulation facilitator course [9, 11]. The facilitators were only responsible for the training at their own base and received remuneration for simulation training outside their ordinary hours of work at the HEMS base. The training was planned by the local facilitator on a convenience basis and took place during the daytime on weekdays. There were no requirements or expectations regarding the total number of sessions or their frequency during the study period. The facilitators emphasised to the crews that the training was an optional learning and training opportunity, rather than a compulsory task. In cases where the planned simulation was interrupted by live missions the facilitator could opt to join and observe the mission if appropriate. Crews were informed verbally and in writing, that the in-situ simulation could be changed to an observed live mission with debriefing if the in-situ simulation was interrupted. The debriefing after both in-situ simulation and live mission observation was performed using the PEARLS framework which is structured as; reaction, description, analysis, application/summary [12].
In the second stage of the project, structured individual interviews were conducted with selected crew members from the three bases that participated and the facilitator at each base to explore their experiences with HEMS mission observation as an alternative to the in-situ simulation. The second stage took place between June 1st and June 15th, 2017.
Data collection and analysis
Observational data
The facilitator collected data on the duration of the in-situ simulation and classified it as completed with debriefing or interrupted by a mission call-out with or without subsequent live mission observation. Data was instantly recorded on a premade paper form by the facilitator and later entered into an electronic database (Questback Essentials, Oslo, Norway). This paper form was designed by one author (PB) and approved by the other authors. It was designed for a practical purpose and collecting data about practicalities of the simulation including information about cancellations. The form was completed after the training and subsequently entered into the database. No data related to individual crew members was recorded. We summarised continuous data using median (quartiles) and categorical data as numbers (percentages) and compared non-paired observations with the Mann-Whitney U test. All statistics were calculated using SPSS (IBM Corp. SPSS Statistics for Windows, Version 25, IBM Corporation, Armonk, NY, USA).
Questionnaires
Following both in-situ simulation training and observed live mission debriefing, the participating crew and the facilitator rated their degree of satisfaction with the session on a visual analogue scale (VAS) from 0 to 100 mm, where 0 represented completely unsatisfactory and 100 represented maximum satisfaction [13]. The participating crew members were then asked to respond to one of two questionnaires with 14 questions, adapted to either training as planned with an in-situ simulation or observed live mission with a debriefing. All answers had to be indicated on a 7-point Likert scale where 1 corresponded to “I strongly agree “and 7 to “I strongly disagree” [14]. The questionnaires, to capture the participants evaluation of the simulation-based training, were similar to the questionnaire used at a previous stage of this project but adapted to the current context [3]. An initial draft was made by two of the authors (PB and SS). The questionnaire was then reviewed by a pilot, a HTCM and a physician. The modified questionnaires were piloted and reviewed by two separate full HEMS crews from the Lørenskog HEMS base to ensure that the questions were clear and comprehensible. An English version of the questionnaires, translated independently by two authors (PB and SS) is available as Additional file 1.
Interviews
Following the one-year study period, we invited a group of physicians, HTCMs and facilitators to take part in a structured individual interview. Interview candidates were chosen so that all participating bases were represented. We did not include the pilots in these interviews since they only had a minor role in the patient-centred part of the missions. An interview guide was used (Additional file 2), but interviewees were encouraged to elaborate and discuss other topics that would emerge during the interview if they wanted to. All interviews were done on a weekday during daytime and lasted approximately 20 min each. The interviews were undertaken via telephone for practical reasons and by the first author (PB) exclusively. The interviewer is experienced both as a consultant in prehospital critical care and as a simulation facilitator. He has conducted interviews previously and has conducted studies on in-situ simulation in HEMS services demonstrating a positive effect on training culture [3, 9]. When no further information emerged on a topic, the interview would continue according to the interview guide. At the end of the interview, the interviewees were encouraged to mention anything they felt had not been addressed. The interviews were recorded on two separate digital recorders for proper redundancy in case of technical difficulties or quality issues with the recordings. The digital interview files were stored on an encrypted server to which only the first author (PB) was granted access.
The recorded interviews were transcribed verbatim by a medical student who received an hourly payment for the job and was not part of the project. The primary author (PB) compared the transcriptions to the recorded interviews to ensure the transcription’s quality and accuracy. All digital recordings of the interviews were deleted after analysis.
The transcribed interviews were analysed according to Malterud’s “Systematic text condensation” by two of the authors (DØ and PB) [15, 16].
To obtain a total impression of the interviews, the main author (PB) read through the transcripts to get a sense of the interviews and identify themes. The themes were applied if appropriate by two authors. The interviews were then reread to define and identify “meaning units” covering the themes identified in the previous step. These meaning units were text fragments/quotations with information about the participant’s thoughts. The meaning units were then sorted into subthemes and then synthesised into text. After synthesising, the interviews were reread through to ensure that no information was lost.