Does participation in the European Trauma Course lead to new behaviours and organisational change? A Portuguese experience

Abstract

Background

Medical educational courses can be successful from an immediate feedback perspective but not lead to new behaviour or organisational changes in the workplace. The aim of this study was to assess the self-perceived impact of the European Trauma Course (ETC) on Reanima trainees’ behaviour and organisational change.

Methods

A 40-item questionnaire based on Holton's evaluation model was used to evaluate the candidate's perceptions. The results were analysed with descriptive and inferential statistical analysis using nonparametric tests with α = 0.05.

Results

Out of 295 participants, 126 responded to the survey. Of these, 94% affirmed that the ETC modified their approach to trauma patients, and 71.4% described a change in their behaviour. Postcourse responders changed their behaviour in their initial approach to trauma care in the nontechnical skills of communication, prioritisation and teamwork. Being an ETC instructor strongly influenced the acquisition of new material, and this group was able to implement changes in attitudes. Individuals with no previous trauma course experience identified lack of self-efficacy as a significant obstacle to introducing new work-based learning. In contrast, responders with ATLS training noted a lack of ETC colleagues as the main impediment for moving from conceptualisation to experimentation in the workplace.

Conclusions

Participation in the ETC led to behavioural changes in the workplace. However, the ability to influence others and bring about wider organisational changes was more difficult to achieve. Major factors were the status of the person, their experience and self-efficacy. National organisational impact was obtained, which went far beyond our aspirations in acknowledging change in individual daily practice.

Future research studies will include the effect of implementing the ETC methodology on the outcome of trauma patients.

Background

The European Trauma Course (ETC) is an evidence-based trauma management course for the early in-hospital phase of delivering trauma care. In addition to direct trauma resuscitation training, it also teaches team leadership, team membership and team management [1]. This is achieved by using a team-based practical approach that includes hybrid simulation that allows for local adaptations to meet regional European needs [1,2,3,4].

Reanima is a Portuguese nonprofit organisation that teaches critical care medicine (CCM), resuscitation and trauma management. It has participated in the ETC since its inception and has run twenty courses between 2015 and 2021.

This ETC study, organised by Reanima, was designed to blend qualitative and quantitative approaches in a framework of evaluative enquiry. Kirkpatrick hierarchy four-level evaluation model [5] is a widely used tool for evaluating medical training interventions but, there are accepted limitations. Reanima uses this model to evaluate first level results of training courses, obtaining important information to the training organisers. However, Kirkpatrick approach it is not so efficient in evaluating the knowledge transfer to the workplace because there is evidence that there are other variables involved in the knowledge transference. In our study we proposed the Holton's evaluation model with a more comprehensive framework for diagnosing and understanding the causal influences of Human Resource Development (HRD) intervention outcomes. Three outcome levels are assumed in this model: learning, individual performance and organisational performance. Outcomes can be influenced by primary intervening variables (e.g., ability, motivation to learn, reaction to learning, transfer conditions, expected utility, linkage to organisational objectives) and secondary intervening variables (e.g., job attitudes, personality characteristics) [6].

Assessment of the ETC, organised by Reanima, has been based on postcourse evaluations by candidates, national faculty and audits by international faculty. These have all been positive, and trainees have frequently indicated a high degree of satisfaction with changes in all three domains of learning (knowledge, skills and affect). Studies have also shown evidence of knowledge acquisitions via pre- and postevaluation, with similar findings in literature reviews [7,8,9]. However, there are accepted limitations to this form of evaluation. Subjective assessment represents the lowest level of Kirkpatrick’s hierarchy [5], and previous studies have shown that there is little or no relationship between a person’s experiential learning and the ability to transfer the new knowledge into their daily practice [10, 11]. Therefore it remains to be clarified whether trainees transfer their ETC experiential learning into the workplace.

The aim of the present study was to assess the self-perceived impact of the ETC on Reanima trainees’ behaviour and organisational change. To date, we are unaware of any other studies that have provided an analysis of ETC impact on trainees’ transfer of experiential learning and the mechanisms of implementation into their daily practice, taking into account the factors that hinder or facilitate knowledge transfer.

Methods

Study participants and setting

Reanima organised sixteen ETC courses between February 2015 and June 2019, with 295 participants from all regions of Portugal and different trauma realities. The number of participants per edition ranged from 12 to 24 candidates. The candidates gain access to ETC by paying the course themselves or were sponsored by their employer.

Intervention

ETC is an evidence-based trauma management course for the early in-hospital phase of delivering trauma care. It is a course with trauma scenarios simulations (30 trauma scenarios and 10 workshops) and 2 lectures covering all aspects of major trauma. One of the fundamental ETC points in trauma resuscitation is the importance of team horizontal approach to the primary survey. ETC reflects European practices, being flexible to reproduce local practices too. The TraumaTeam Members (TTM) include an Airway(A), Breathing(B), Circulation (C) person and a Trauma Team Leader (TTL).

The non-technical skills (NTS) are key areas developed in ETC and they are essential for team`s effectiveness, all of which are closely linked and integrated by communication. NTS are, in this context: team work (e.g., working together with a common goal, being accountable for your actions), task management (e.g., the Team Leader must ensure: A, B, C tasks according to team members’ competencies), situational awareness (e.g., knowing what is going on around you), decision making (e.g., prioritize, encourage team input) and communication (e.g., all key findings are clear to everyone, use of person´s names, closed loop communication).

Assessment on ETC is done continuously during the 1^st & 2^nd day (technical skills, team membership skills, team leadership skills) and summative assessment on the 3^rd day (team leadership skills).

Data collection

A questionnaire based on Holton’s evaluation model [6, 12] was used to assess whether ETC leads to behavioural changes on the Reanima trainees’ workplace (Google^© form), i.e., is evaluated not only the individual motivation to transfer but also a favourable organisational climate to transfer. Initially created by a Reanima ETC faculty working group, it was modified in light of feedback provided by senior faculty participants. The survey was based on voluntary participation and information disclosure. The study conformed with the principles of The Declaration of Helsinki, and participation was taken as consent. All volunteers were allowed to withdraw from the survey at any time during data collection without any prejudice. Participant data were treated in line with the General Data Protection Regulation. Regarding the study data and analyses, all identifying data were removed. Moreover, no data analysis was performed that allowed for the publication of results that could eventually indirectly identify a participant. The survey was carried out according to the formal approval of the Research Center of Centro Hospitalar Universitário de Santo António.

Forty questions covered six topics and are accessible in Additional file 1:

• Respondents’ biographic data (e.g., age, sex, professional category, workplace, previous experience teaching CCM or trauma)

• Motivation to attend an ETC course

• Previous trauma training

• Trauma experience in the workplace, including a role as a trauma team leader (TTL) or trauma team member (TTM)

• Workplace changes to the management of major trauma patients, including team approach, communication; acquisition of new material and application of new therapeutic interventions

• Obstacles and facilitators to the introduction of new trauma approaches at both the individual and organisational levels.

The questionnaire was published online in a Google^© form from 1^st October until 30^th November 2019. The first email was sent to all Reanima ETC participants (n = 295) upon its inception in Portugal. After the first-round email, three reminders were sent at week 2, week 4 and week 6.

Data analysis

Statistical analysis was conducted using the statistical software package IBM SPSS© Statistics vs. 25.0 (IBM Corp. released 2017, Armonk, NY: IBM Corp.). Descriptive statistical analysis. Categorical variables are depicted as frequencies and proportions/percentages (n and %), and quantitative variables (age and years of practice) are expressed as medians and interquartile ranges (IQRs). Missing answers for new material acquisition, implementation of new therapeutic attitudes and teamwork methodology were categorised with ‘no answer’.

Comparisons of counts for more than two categories of qualitative variables were performed using the chi-square test, and all pairwise comparisons were adjusted using Bonferroni’s correction. Comparisons were only performed for categories with sufficient counts and are presented in Tables 3 and 4. The level of significance was set at p < 0.05.

Results

Out of 295 participants invited to participate in the survey, 126 (42.7%) responded. All presented percentages were calculated on the number of responses to each question.

Respondent characteristics

Respondents were young (median (IQR) of 33 (31–36) years) and early in their careers (median (IQR) of 7 (4–10) years of practice). The majority of respondents (92/119, 77%) had no formal leadership status at their hospital, although they had a previous trauma course (67/126, 53%) and trauma experience (106/126, 84%), as well as familiarity with teaching CCM (75/126, 60%), and 94% (119/126) were still working in the trauma field. Ninety-four percent (118/126) self-reported modifications in their approach to trauma patients. These data are presented in Table 1. The majority (71/119, 60%) of respondents were, on average, involved in managing over three trauma resuscitations each month, and more than half (65/126, 52%) worked in a hospital within a multidisciplinary trauma team composed of surgeons (74/126, 64%), intensivists (62/126, 53%), anaesthesiologists (48/126, 41%), emergency medicine physicians (51/126, 43%) and nurses (106/126, 91%). The respondents’ trauma work place was: emergency room (85/119, 71%), pre-hospital care (58/119, 49%), operation room (52/119, 44%), intensive care unit (48/119, 40%) and others (12/119, 10%).

Table 1 Characteristics of responders (n = 126)

Of those who completed the survey, 82/125 (66%) paid for the course themselves, with 35/125 (28%) having been sponsored by their employer. Eighty (68%) stated that acquiring a systematic approach to trauma patient management was the most important reason for participation in the ETC.

The impact on clinical practice

Ninety-four percent of the respondents (118/126) affirmed that participation in the course modified their approach to trauma patients, as shown in Table 1, and 71.4% self-reported changes in their knowledge, skills and affect, which are presented in Fig. 1. Their nontechnical skills changed, particularly with regard to communication, prioritisation and teamwork, as shown in Table 2 (complete variable information in table 4, Additional file 2). When performing TTL in clinical practice, respondents without formal leadership status reported a significant change in prioritisation (71, 83.5%), and those with previous leadership experience noted significant changes in communication (26, 26.3%). Respondents who had not previously attended a trauma course identified significant changes, with more marked changes in prioritisation compared to communication skills (45, 53.6% vs. 39, 40.6%, respectively, p = 0.001). Those with CCM teaching experience identified significant changes in their communication skills (7, 6.8%, p < 0.05). When performing trauma team membership approach at workplace, respondents managing an average of less than two trauma cases per month significantly valued the structured ABCDE approach (15/72, 20.8%) and communication (16/98, 16.3%) more so than teamwork (p = 0.001), which is presented in Table 2.

Fig. 1

Respondents self-perceived report of the behaviour changes after participation in ETC, using a numeric scale: 0 = no behaviour changes to 5 = remarkably behaviour changes

Table 2 Report of respondents’ characteristics versus their perception of ETC impact on their clinical practice (complete variable information in Table 4, Additional file 2)

Being an ETC instructor (17/35, 48.6%, p = 0.001) and managing two to three trauma cases/month (11/35, 31.4%, p = 0.001) influenced the acquisition of new materials (e.g., pelvic binder) and the implementation of new therapeutic standards, namely, the use of tranexamic acid, which is presented in Table 2.

Factors affecting the transfer of knowledge in the workplace.

At an individual level

Factors that facilitated the transfer of knowledge (no significant differences) included confidence in ETC methodology (83, 45.6%), peer support (50, 27.5%), decision-making experience (35, 19.2%) and organisational support (9, 4.9%), which are presented in Table 3 (complete variable information in table 5, Additional file 2). In contrast, lack of other ETC colleagues (28, 19.4%), low self-efficacy (27, 18.8%) and lack of motivation (26, 18.1%) all hindered knowledge transfer. Respondents with no previous trauma course experience (17, 73.9%) or no teaching experience (17, 63.0%) identified low self-efficacy as a significant obstacle for introducing new behaviour. Those with Advanced Trauma Life Support (ATLS) training (15, 60.0%) found lack of other ETC practitioners to be the main obstacle to change in the workplace, which is presented in Table 3.

Table 3 Report of respondents’ characteristics versus the factors that facilitated or hindered the transfer of learning (complete variable information in Table 5, Additional file 2)

At an organisational level

Change in organisational behaviour was linked to the number of ETC-accredited staff (25, 28.1%), material resource availability (24, 27.0%) and human resources (21, 23.6%), although without statistical significance, which are presented in Table 3. No significant differences were observed for the identification of obstacles in introducing new behaviours.

Discussion

This study provides evidence that participation in the ETC can be one of the elements that can lead to individual behavioural changes in the workplace. However, the ability to influence others and bring about institutional or regional organisational changes was more difficult to achieve. Major factors were the status of the person at the workplace, their previous experience (trauma clinical practice and coursework, as well as in teaching CCM) and self-efficacy. Being an ETC instructor influenced the acquisition of new material and the implementation of new therapeutic attitudes. The Reanima ETC survey identified factors that facilitate or hinder the transfer of experiential learning at the individual, organisational and national levels.

To the best of our knowledge, this is the first study focusing on the ETC impact on candidates’ professional behaviour towards their management of major trauma. A search of the biomedical and medical education databases produced no mentions of the ETC other than those articles written by members on the early development of the ETC group [1,2,3,4]. Over the last three decades, there has been an exponential increase in the number of life-support type courses. Examples include Advanced Cardiac Life Support (ACLS) and ATLS, among others. Although popular and apparently well-received, evidence for their impact in the workplace is heterogeneous [13,14,15]. Hattie et al. (2013) hypothesised that almost any approach to learning might work, but later in 2021, they also identified through a meta-analysis ten factors that effectively influence the process of learning [16, 17]. Despite a paucity of data on the effect of ATLS on trauma mortality, existing evidence supports its practice as a means of decreasing mortality and improving systems of care globally [18,19,20].

To date, the assessment of the ETC, organised by Reanima, has been based on postcourse evaluations by candidates, national faculty and audits by international faculty. For some evaluations, Kirkpatrick model is applied mainly at level 1.

Reaction, also called “customer satisfaction”, i.e., how participants perceived the training; and level 2. Learning, i.e., at what level participants acquire more knowledge, new behaviours, and values and develop skills as a result of their participation in the course. All feedback has been very positive, which encouraged us as an ETC organiser to proceed. The scarcity of evidence on the impact on trainees’ clinical practice challenged us to clarify whether trainees applied the newly acquired ETC knowledge and changed their behaviour in their workplace. Recently, Greif et al. pointed out the importance of ‘educational efficiency’, arguing that educational approaches are critical links between scientific findings and their implementation into practice [21, 22]. A questionnaire based on Holton’s evaluation model [6, 12] was used to assess whether the ETC leads to behavioural changes in the workplace, i.e. if occurred knowledge transfer to the workplace. Our survey results were very promising, with 94% of the respondents reporting that the ETC modified their approach to trauma patients, as shown in Table 1, and 71.4% significantly changed their behaviour, as presented in Fig. 1. These data lead us to infer that ETC methodology, providing a robust educational approach, could be one of the factors favouring knowledge transfer to practice, meeting what 68% of the respondents considered the most important reason for participation in the ETC, i.e., acquiring a systematic approach to trauma patient management.

Research indicates that teamwork and leadership are increasingly recognised as important factors contributing to patient safety and outcomes in health care [23,24,25]. These are important goals of the ETC methodology, where NTS are developed to improve how technical skills, tasks and procedures are carried out by trainees [26]. Post ETC respondents acknowledged behaviour changes regarding the NTS, such as communication, prioritisation and teamwork, which are presented in Table 2. The changes were dependent on formal leadership status at the hospital, previous training in trauma courses, trauma experience and experience with teaching CCM. Respondents without leadership status registered changes in prioritising tasks, acquiring the team leadership skills emphasised in the ETC. In contrast, respondents with leadership status noted communication as the main area of change. What was considered most important varied with the respondent’s current status and frequency of trauma cases managed per month.

ETC instructors held a strategic position as influencers for the acquisition of new institutional materials (e.g., pelvic binder) and implementation of new therapeutic standards in massive haemorrhage (e.g., standard use of tranexamic acid). These results point out the opportunity and ability of instructors to change their workplace.

There are two main aspects of knowledge transfer: the underlying process and the factors that facilitate or hinder it [12]. Reanima ETC study findings have helped to clarify some factors that influence the transfer of learning [27,28,29,30].

At an individual level, confidence in ETC methodology and peer support were the most common facilitators for introducing new behaviour changes, as presented in Table 3. The obstacles to changing behaviour were personality characteristics (for instance, some recognised themselves as lacking self-efficacy, especially those with no previous trauma coursework or teaching experience) and other priorities related to career choices. Respondents with ATLS training found a lack of ETC practitioners in their place of work to be the main obstacle to change, indicating a need for more ETC courses.

A change in organisational behaviour was linked to the number of ETC-accredited professionals and the availability of material and human resources, which is not surprising.

However, the majority of trainees (92, 77% of the 119 that answered the question) had no formal leadership status at their hospital, which could be a reason why we observed fewer dramatic regional organisational changes. Other obstacles to introducing new behaviours included lack of knowledge about ETC methodology by other professionals (69, 31.1%) and limited human resources (65, 29.3%), as presented in Table 3. Again, these findings emphasise the need for ETC dissemination. As ETC has only recently been implemented in Portugal, it would be worth reassessing this in the future when there is a greater share of experiences and knowledge.

Although the minority of respondents had a formal leadership status at their hospital, the positive impact of the ETC has led the Portuguese College of Anaesthesiology to integrate ETC into their trainees’ resident program. This follows their evaluation of it being a good educational model for improving trauma resuscitation training. A further national impact of the ETC has been the prioritisation of the National Trauma Registry, which aims to provide a useful tool in the quality evaluation and improvement of trauma care [31].

Our survey has inherent limitations. First, selection bias cannot be ignored. Given the process of dissemination, it is difficult to provide an estimation of the nonrespondent-to-respondent ratio. Sample size calculations require assumptions about expected event in different groups or, upon expected effect sizes. Therefore, when no guesstimates or expectations are possible (if no assumptions can be made), and no pilot studies were conducted previously, a sample of 126 respondents (which is not large) might be considered reasonable (> 40%) for the field when considering the target population size (N = 295) [32, 33].

Furthermore, any self-report survey is prone to bias; thus, we could not assess the gap between routine practice and self-perception. As such, attitude change in the present study reflects no more than self-perceived and self-reported change.

Effective implementation of a science field, namely trauma education, needs the application of efficient and evidence‐based interventions to improve the health and well‐being of trauma population. It involves: diffusion, dissemination, implementation, adoption and sustainability [34,35,36,37,38].

Conclusion

The investigators acknowledge that ETC can be one of the elements that has impact on Reanima trainees´ recognizing the changes introduced and adopted in their own clinical practice, in their work place and in the end, in the political authorities who manage national healthcare systems. This is highlighted by their behaviour change as they had acquired new knowledge, values, attitudes and skills (practical and procedural). They were able to introduce change into their organizations, disseminating knowledge, i.e., the implementation and adoption of ETC trauma approach. Further improvement to obtain sustainability of ETC project and improve the health and well‐being of trauma population may also be possible through international trauma registry development that will guide us to other achievements, such as the impact of the ETC trauma approach on patient outcomes.

The Reanima ETC survey delivers several key messages. The combination of highly motivated trainees with the ETC educational efficiency methodology was linked with a changing approach to trauma. However, the positive responses from the individual candidates do not automatically translate into organisational changes. Other confounding factors can help or hinder this transformation. Ultimately, both the individual and the organisation need to be in synchrony so that the full impact on the trauma patient’s outcome can be achieved and measured. This is a topic for a future study.