Skip to main content

Flipped learning enhances non-technical skill performance in simulation-based education: a randomised controlled trial



Many efforts of the past years aimed to build a safer health care system and hereby, non-technical skills (NTS) have been recognised to be responsible for over 70 % of preventable medical mishaps. In order to counteract those mishaps, several simulation-based trainings have been implemented in health care education to convey NTS. Still, the best and effective way to foster NTS in simulation-based training is not known. Due to the importance of NTS, this gap in knowledge needs to be filled. A possible approach to convey NTS effectively during simulation-based medical education (SBME), might be the use of the flipped learning approach. The benefits of flipped learning regarding the improvement of human factors (NTS), have not been investigated yet. Therefore, the authors introduced flipped learning as an experimental intervention into their SBME emergency trainings and aimed to analyse, whether flipped learning improved students´ NTS performance compared to lecture-based learning (LBL).


In a randomized controlled trial, 3rd year medical students participated in a SBME training and then received either a further SBME training with integrated flipped learning on NTS (intervention), or a further SBME training and an accompanying lecture on NTS (control). NTS performance was assessed on three skill dimensions with a validated behavioural marker system.


The authors analysed NTS performance of 102 students, prior and after their allocation to each teaching method. The baseline NTS performance of both groups did not differ, whereas the intervention group enhanced significantly on all three skill dimensions (t (44) = 5.63, p < .001; t (44) = 4.47, p < .001; t (44) = 4.94, p < .001).


The integration of flipped learning into SBME yields a significant improvement of NTS performance and therefore medical educators should consider the application of flipped learning to convey complex human factors and skills.

Peer Review reports


Since “The Institute of Medicine” published over 20 years ago the landmark report “To err is human”, many efforts have been done to make healthcare safer and reduce preventable mishaps [1]. In this context, human factors (non-technical skills) have been recognised to be responsible for over 70 % of medical mishaps [2, 3]. Therefore, high quality medical care requires technical skills (TS) alongside with non-technical skills (NTS) [4,5,6,7]. TS refer to medical knowledge and practical procedures (like chest compression, inserting an intravenous line), whereas NTS are defined as “the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance”[7].

Training of NTS should, as recommended by several position papers, should take place as early as possible in health care education [8]. For this purpose, the WHO has even published a safety curriculum guide for medical schools [9]. The emphasis on the importance of early implementation of NTS training is based on the knowledge that NTS are not acquired through clinical practise and routine [10]. Therefore, many medical faculties have adapted their curricula and implemented simulation-based trainings to address NTS in undergraduate education [11].

Although simulation-based medical education (SBME) has been recognised as the ideal instructional design to train NTS, the best and effective way to teach and train NTS during SBME is not known [12, 13].

A possible approach to convey NTS effectively during SBME might be the use of the flipped learning approach, which has been adopted into various undergraduate and postgraduate healthcare curricula [14, 15]. Flipped Learning is an instructional approach in which the traditional concept and idea of classroom-based learning is inverted [16]. New learning contents are first mastered with structured activities in the individual learning space. The face-to-face time (group learning space) is used to accelerate the learning cycle by using more active learning strategies [17]. Therefore, according to the learning cycle model [18, 19], the active content attainment takes place in the individual learning space and the concept application shifts to the class time [16, 20,21,22]. To promote the reflective process and enhance learning achievements, the provision of an adequate learning guidance is important for the individual learning phase. Therefore, flipped learning needs proper planning and organisation and the concept consists not only of the distribution of additional tasks [23, 24]. Flipping a class, like having learners read additional material outside the class, does not necessarily result in flipped learning [16].

The acquisition and the transfer of NTS into behavioural patterns is a complex process which is often not achieved with diligence and learning efforts- it requires deep learning, reflection and the creation of mental models [25, 26]. Therefore, the elements “flexible environment” and the “intentional content” of the flipped learning approach could contribute to enhanced learning achievements of NTS: Students are enabled to control their speed of learning, to reflect as often as possible on the learning contents and hereby activate deep processing, build mental models and develop targets for action plans to improve performance [27].

As suggested by Chen and colleagues in a systematic review [15], as well as in an investigation of Tang and colleagues in ophthalmology teaching [28], one positive side-effect of flipped learning could be enhanced (autonomous) motivation. This phenomenon occurs due to autonomy supported and learner-oriented learning, which results in an enforced identification process with the learning contents [29]. A brief insight into motivational theories explains how this desirable side-effect might come about: The Self-determination theory of motivation describes different motivational qualities that underlie human behaviour (behavioural regulation). When motivation derives from heartfelt interest or personal endorsement, a person is autonomously motivated. Autonomous motivation is composed of the motivational qualities “intrinsic” and “identified “. The opposite concept is controlled motivation (“external”- and “introjected regulation”). Here, motivation derives from external- or internal pressure which is formed by desires for bounty [30]. To support autonomous motivation, three basic psychological needs have to be satisfied: Autonomy, competence and relatedness [31]. Several studies have confirmed autonomous motivation as the type of motivation that leads to better well-being, better learning and greater academic success [32, 33]. The flipped learning approach would help the students to reflect on the importance of non-technical skills during the individual learning space, other than just except them because they are part of their curriculum learning contents. This reflection process shifts the locus of causality (the “why” of engaging in an activity) from the outside (curriculum learning goals) to the inside (identification with the task) [31]. The process of re-location leads to enhanced autonomous motivation, which has been repeatedly emphasized by curriculum developers, to be an important determinant of academic success and learning [32]. Therefore, flipped learning could be one approach to enhance the (desired and demanded) dependent variable of students´ motivation.

Although several studies suggest that the flipped approach improves learning in healthcare education and is superior compared to classical learning approaches [15, 20, 22, 34], to our best knowledge, no published study has investigated if flipped learning yields significant improvements in human factors- like NTS performance.

Therefore, we introduced flipped learning as an experimental intervention into our SBME emergency trainings and aimed to analyse whether flipped learning improved students' NTS performance. Furthermore, we compared the SBME and flipped learning approach with SBME and lecture-based learning (LBL) with respect to NTS performance. We also explored whether the flipped learning approach had an effect on the affectional dimension of learning and analysed students' motivation of both study groups, towards participating in the SBME teaching approaches. We hypothesised, that the flipped learning yielded an improvement of NTS performance itself and compared to the lecture-based conventional training (primary outcome measure). The secondary outcome was students´ motivation to participate in the SBME.


This prospective randomised controlled simulation study with blinded participants was performed at the University Medical Center Hamburg-Eppendorf during Winter semester 2019/20.


To reduce performance bias, 3rd year medical students were chosen as study participants, because they were familiar with simulation trainings.

The students were divided into subgroups and the subgroups were randomized to the intervention- or control group (computer-generated random numbers [35]). To prevent a Hawthorne effect (modification of behaviour due to the knowledge of observation), the students did not know to which group they were randomized to (blinded participants) and they did not know when and what was scored during the trainings [36]. Each group participated in two trainings (first training was for the assessment of the baseline). The trainings were scheduled within one week (Tuesday and Thursday) of the study period for each subgroup.

Participation in the study was voluntary and written informed consent was attained from every study participant.

All students were asked to keep discreet about the details of their training and not to enclose it to their fellow students.

Study setting

After the first training (baseline), the groups received two different teaching approaches on NTS according to their allocation (flipped learning or lecture-based learning) and a further SBME training.

Baseline training

A maximum of 12 students participated per training and the duration of each training was 150 min. The two groups of students were supervised by the same medical teachers (instructor), who were experienced physicians of the department of anaesthesiology with extensive knowledge and training in emergency medicine, cardiac life support, medical education and rating NTS during simulation trainings. The learning objectives of each training were accessible through an online platform of the medical faculty.

The first simulation training of both groups had the same structure and the same procedure and was a conventional SBME training. Likewise, the same simulation scenarios were carried out in both groups with the same learning objectives.

At the beginning of each training a 30-minute seminar was held to refresh theoretical knowledge of emergency medicine and cardiac life support. Principles of Crew Resource Management (CRM) were addressed briefly, using examples from aviation.

Each training consisted of three standardized assigned simulation scenarios (emergency scenario with cardiac arrest) and each scenario was conducted in a different room of the simulation centre of the department of anaesthesiology, using high fidelity manikins (Resusci Anne, Laerdal Medical AS, Stavanger, Norway). The participants of each training (pre-defined subgroups of students by the dean´s office- maximum of 12 students per training) were divided into smaller subgroups (5 to 6 students per subgroup) to rotate through the three scenarios and one instructor supervised each scenario. From these smaller subgroups three undergraduates were assigned randomly to execute the scenario. The assignment of the roles for each scenario (one team leader (physician) and two nurses/paramedics) was also randomly. The remaining students of the smaller subgroup participated as neutral observers.

A debriefing was conducted after each scenario, focusing on three conceptual phases: gathering, analysing, summarising. The debriefing was held in the conventional way and the role of the instructor (medical teacher) during the debriefing was that of a teacher.

Teaching methods of NTS: Interventional and control training

Flipped teaching method (intervention)

Preparation phase

Six months prior to the study period, three medical teachers were engaged in the preparatory phase of the flipped learning approach. According to the backward instructional design, first learning goals (NTS) were identified and then opportunities for pre-learning were discussed and active learning strategies were specified. Afterwards the final design of the flipped learning was determined, which included the four pillars of F-L-I-P ™, as described by the “Flipped Learning Network” (Flexible environment, Learning culture, Intentional content, Professional educator) [37].

The contents which were carried out in flipped learning were the definition and explanation of NTS with reliance to the AS-NTS rating system and relevant benchmarks, which took mostly place in the individual learning space. The students were given the task to prepare themselves in the best possible way on NTS and AS-NTS and its benchmarks, in order to carry out the debriefing in the upcoming training. This step was designed to foster reflection on NTS prior to the second training.

To facilitate learner-centered environment, all the teaching activities were designed based on instructional scaffolding:

Pre-training (class) contents.

  • Brief seminar on NTS held at the end of the baseline training (twenty minutes) This seminar included following contents: History of how NTS were translated from aviation to healthcare. An introduction to the rating system Anaesthesiology Students´ NTS (AS-NTS) [11] with explanatory behavioural benchmarks for each dimension of the AS-NTS. For this purpose, scenarios and situations were talked through with focus on how good NTS performance would be like.

  • At the end of this seminar teaching materials were provided, which included a script on NTS and behavioural benchmarks, a PowerPoint courseware and a detailed description about how AS-NTS was developed and how its application works.

  • Individual learning space (instructional activities): A set of clear defined work assignments, which included to work through the provided teaching materials; to complete a written debriefing of the three scenarios which the students had observed in the baseline training (reflection, cold written debriefing)- hereby the application and description and definition of fitting behavioural benchmarks (targets for action) were encouraged (NTS score based on the AS-NTS).

Furthermore, the students were given the assignment to get as familiar as possible with the AS-NTS, because it would be their responsibility to conduct the debriefings of the following training and to provide an AS-NTS score for each scenario.

Classroom phase (group learning space), second training.

  • At the beginning of the training, outstanding questions were clarified and students were asked if every assignment of the individual learning were completed and if they could define behavioural benchmarks. Then, similar to the first training, the undergraduates were divided into smaller groups (randomly) to rotate through three simulation scenarios. The small groups remained together for all three scenarios. Each scenario and its debriefing had the duration of thirty minutes.

  • The debriefings of each scenario were conducted by the observing students of each small group in terms of collaborative learning by team based learning: Three students conducted the scenario as physician and nurses/paramedics. The observing students (two/three) filled out an AS-NTS score while watching the scenario and took notes. The scenarios contained cardio-pulmonary emergencies, like acute coronary syndrome. After each scenario, the observing students had five minutes to discuss their debriefing in a different room, based on their AS-NTS score. Then they provided the debriefing for their peer-students. The role of the instructors was to act like a facilitator to provide a good learning environment and encourage participation.

The provision of the AS-NTS score and debriefing by the students was not for the outcome analysis. The purpose of this exercise was to enable the students to reflect on NTS and apply what they had prepared in their individual learning space. Therefore, the instructors filled out the AS-NTS as well and complemented the peer-debriefing, because the analysis of NTS performance was based on the instructor ratings.

Eight trainings were conducted as pilots to train the instructors, to close remaining gaps, to standardise the intervention and to detect necessary changes of the study design and to identify an effect size for the sample size calculation.

Lecture-based learning method (control group)

Pre-training (class) contents.

  • After the baseline training, the control group received a 90-minute lecture on NTS called “To err is human”. This lecture addressed the same learning objectives which were conveyed to the intervention group and was held by a professor of anaesthesiology and didactics whose lectures were always evaluated as outstanding.

Classroom phase (group learning space), second training.

  • At the beginning of the training, remaining questions were clarified and a brief summary on NTS and the contents of the lecture were given. Then, similar to the first training, the undergraduates were divided into smaller groups to rotate through three simulation scenarios.

  • After each scenario a debriefing was held by the instructors, based on the AS-NTS scoring (Fig. 1).

Fig. 1
figure 1

Depicts the research flow and procedures

Table 1. compares the two different teaching approaches in the context of our study design.

Table 1 Comparison and differences of the teaching approaches

Measure tools

Primary outcome measure: Performance of NTS

The German version of “Anaesthesiology students´ Non-Technical skills” (AS-NTS) [11] was used by the instructors to rate NTS. AS-NTS is composed of three dimensions:

  1. 1.

    Planning tasks, prioritising and problem-solving.

  2. 2.

    Teamwork and leadership.

  3. 3.

    Team orientation.


Performance is rated on a five-point Likert scale (1 = very good; 5 = very poor). An underlying skill structure is used to give behaviourally anchored rating examples to clarify what a “good” or “poor” performance on each dimension might look like.

Validity, feasibility and sufficient coverage of relevant NTS in the AS-NTS have been previously reported [11].

Secondary outcome measure: Situational Motivation to participate in the trainings

Motivation was measured with the German version of the “Situational Motivation Scale” (SIMS), which measures the underlying motivation to participate in a task or activity at a specific point of time (situational) [38]. The SIMS was developed based on the Self-determination theory (SDT) of motivation [29].

Four subscales (intrinsic motivation, identified- introjected- external regulation and amotivation) are assessed with 20 items. Each item has a 7-point Likert scale (1 = Does not correspond at all; 7 = Corresponds exactly).

Autonomous motivation is computed by adding and averaging intrinsic motivation and identified regulation. Controlled regulation is computed by adding and averaging external- and introjected regulation [38, 39].

Reliability and validity of the SIMS, as well as the German translation has been confirmed in many studies [39, 40].

The undergraduates filled out a paper-based SIMS questionnaire at the beginning of the first (pre-test)- and at the beginning of the second (post-test) training.

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics Version 23.0.

The sample size was calculated based on the effect size (partial η² dimension one: 0.21, dimension two 0.5, dimension three 0.36) which was calculated from the results of the pilot study. A sample size of totally 64 study participants were necessary with a two-sided 5 % significance level and a power of 90 %.

Descriptive statistics were used for the calculation of mean values of each AS-NTS dimension and for computing the subscale scores of the SIMS.

For the primary outcome, first a paired t-test was calculated, followed by a factorial ANOVA (2 way)- to compare the main effects of “Time” (independent variable, within-group, 2 levels) and “Group” (independent variable, between-group, 2 levels) as well as their interaction effects on the AS-NTS scores. Homogeneity of error variances in each group, as assessed by Levene´s test (p > .05) was given.

The dependent variables were the scores of each AS-NTS dimension.

Model-estimated marginal means with 95 % confidence intervals were computed and Bonferroni adjusted pairwise comparisons were conducted. Post-hoc tests were then calculated for planned comparison and exploration of the effects.

For the secondary outcome a paired t-test was calculated. The assumptions of a paired t-test were not violated by our data. There were no outliners in the data and the differences between the pre- and post-test scores were normally distributed as assessed by the Shapiro Wilk test.



After the pilot phase, a total of 102 3rd year medical students were included in the study. Inclusion criteria was the take on of the physician´s role in both trainings, because data was analysed continuously with repeated measures. Therefore, 22 of 102 students had to be excluded from the final analysis because due to the size of the training subgroups, they did not take on the physician´s role in both trainings (Fig. 2).

Fig. 2
figure 2

Depicts a participant flow diagram including all steps of the study.

The demographics of the randomised and analysed undergraduates were not significantly different (Table 2).

Nine instructors conducted the trainings for the interventional and for the control group.

Table 2 Demographic data of the study participants


Primary outcome measure: Performance of NTS

As shown in Table 3., NTS performance, assessed with the AS-NTS rating score, were high on all three dimensions in both groups (this complies with low numeric scores of AS-NTS: 1 = very good-, 5 = bad performance).

The results confirmed our hypothesis, that the flipped learning approach enhances NTS performance: The AS-NTS scores of the intervention group enhanced significantly after the intervention on all dimensions (dimension one: t (44) = 5.63, p < .001; dimension two: t (44) = 4.47, p < .001; dimension three: t (44) = 4.94, p < .001).

Table 3 NTS performance (pre- and post-test) assessed with the AS-NTS

Correlation analysis between the variables indicated that the dependent variables (dimensions of AS-NTS) are not independent: τ b (dimension one and two) = 0.506;

τ b (dimension one and three) = 0.394; τ b (dimension two and three) = 0.555.

The results of the main- and interaction effects of the factorial ANOVA are shown in Table 4; Fig. 3, which also confirmed our first hypothesis (flipped learning enhances NTS) as well our second hypothesis (flipped learning is superior compared to LBL).

Fig. 3
figure 3

Estimated marginal means of NTS performance of both trainings for each group 

Table 4  F-ANOVA results: Main and interaction effects

As shown in Table 4, there was a significant effect of “time” for all AS-NTS dimensions, this indicates that there were changes in NTS performance over the time, averaged across the whole sample. The follow up of these results indicates that these changes were not equivalent across the two groups: There was no significant difference between the groups at baseline and the control group did not change significantly over time (dimension one: F (1, 34) = 2.15, p = .152, partial η² = 0.06; dimension two: F (1,34) = 1.06, p = .310, partial η² = 0.03; dimension three: F (1,34) = 1.52, p = .226, partial η² = 0.04), whereas the mean scores of the intervention group changed significantly over time (dimension one: F (1,44) = 31.70, p < .001, partial η² = 0.42; dimension two: F (1,44) = 20, p < .001, partial η² = 0.31; dimension three: F (1,44) = 24.44, p < .001, partial η² = 0.36).

The comparison of the post-test NTS performance between the groups showed that the scores of the intervention group were significantly different (better) on dimensions two and three (dimension two: F (1, 78) = 5.12, p = .026; dimension three F (1, 78) = 5.20, p = .026) compared to the control group. On dimension one, the post-test scorings of the two groups did not differ significantly (F = (1, 78) = 2.56, p = .114).

Secondary outcome measure: Situational Motivation to participate in the trainings

Autonomous- and controlled motivation, assessed with the SIMS were comparable in both groups: high levels of autonomous- and mediocre to low levels of controlled regulation. In both groups, differences between the pre- and post-test measurements of motivation were negligible and not significant (Table 5).

Table 5 Situational Motivation toward participating in the trainings


In our randomised controlled trial, we confirmed our hypothesis and found that a flipped learning approach to train NTS enhances significantly NTS performance of 3rd year medical students, compared to a conventional lecture-based approach combined with simulation.

Several studies in medical education have confirmed the positive benefits of flipped learning on students´ learning and knowledge acquisition and proved its superiority over traditional teaching approaches [15, 23, 34]. Our results confirm the previous findings and extend them with advantages of flipped learning regarding the improvement of human factors (NTS), which complies with level 3 of Kirkpatrick´s framework [41].

The improvement of human factors is a challenge in medical education and recently a lack of evidence of how to teach and train NTS has been identified [5, 13]. As NTS are gained through the socialisation process of every individual [42], the learning process of NTS is detached from learning processes of factual knowledge. Factual knowledge can be acquired with diligence and studying, whereas the acquisition of NTS is far more complex and requires a transfer of knowledge and behavioural benchmarks into one´s own behavioural patterns [42]. Therefore, many interventions and investigations failed to find the ideal approach to teach and foster NTS in simulation trainings [5, 25]. These investigations mainly focused on post simulation debriefings [5]. The reasons why NTS were not conveyed effectively in the reported studies, can be explained with principles of learning psychology [43, 44]. The training itself as an isolated instructional design, might not be the ideal setting, as learning does not occur in a closed system, where the instructor provides knowledge and the students simply absorb it [45]. Secondly, students who participated in a simulation scenario, directly prior to the debriefing, might not have the emotional absorption capacity for the debriefing input, leading to cognitive overload and resulting in a declining learning process [44]. Certainly, the post simulation debriefing is a crucial component to promote the actual learning process during SBME [12, 46, 47]. Nevertheless, in order to foster active learning and accelerate learning from experience by promoting reflection during SBME, some concepts and factual knowledge have to be reviewed beyond the boundary of formal class time with self-directed instructional activities [45, 48].

In our study, we created these prerequisites of learning for the students, wherein, among other factors, lies the explanation for our results: First, the students had the opportunity to control the learning speed during the individual learning space. Secondly, the instructional activities were designed to connect prior experience (first training) with information and behavioural benchmarks and hereby fostered the process of reflection, which in turn enabled students to develop their own knowledge and mental models of NTS [17, 45, 49, 50]. The reflection on NTS behavioural benchmarks and the building of mental models during the individual learning space, promoted the learning cycle and resulted in a type of formative assessment [51, 52], which accelerated the transfer of factual knowledge to behaviour. Due to its importance, we supported and facilitated the process of mental modelling by illustrating general benchmarks of NTS (provision of the AS-NTS framework) [11]. It was not the peer-debriefing- but far more the responsibility to carry out the debriefings which fostered the reflective process, enhanced deep learning and the building of mental models during the individual learning space. Hereby the transfer of NTS into behaviour and practise was facilitated. Therefore, the task to prepare for the debriefings, might have been the most decisive part of the individual learn space of the flipped learning approach.

The formation of mental models is important to enhance NTS, as it has been demonstrated that mental models help to enhance team effectiveness and to act properly under varying conditions [26, 52,53,54]. According to experimental learning, the building of mental models complies with the generalization stage, in which the students think and reflect critically and analyse what might apply in real clinical practise [55]. Finally, this generalization stage lead to the retrieval of the prior built mental models, during the face-to-face time (second training), resulting in enhanced NTS performance.

Thus, it can be summarised, that conveying the complex topic of NTS with the flipped learning approach enhanced students´ learning experience and fostered positive learning outcomes and behavioural changes. The graduated approach enabled the students to have the ownership of their learning process and prevented cognitive overload [17, 56].

One might argue that according to constructive alignment, the theoretical teaching approach which we designed for the individual learning space, might not be appropriate [57]. Considering that the theoretical instructional activities fostered the reflective process and therefore the transfer of knowledge to action, this objection can be overruled. Furthermore, a first theoretical approach seems to be necessary, in order to foster active learning during SBME.

Interestingly, the baseline scorings of the control group were, even if not statistically significant, better than the scores of the intervention group. One can argue that the intervention group enhanced skills more than the control group, because there was more potential for improvement. However, no ceiling effect occurred in the control group, which indicates, that further improvement is still possible.

Curriculum developers have repeatedly emphasised the positive impact of autonomous motivation on learning success and academic achievement in medical students [30, 33]. Therefore, the integration of autonomy supportive instructional designs and teaching approaches into medical education have been recommended [58]. We hypothesised that the flipped learning approach would enhance students´ autonomous motivation due to its learner oriented and autonomy supportive features. Nevertheless, we did not find any increase of students´ autonomous motivation, although- based on SDT- the pillars of autonomous motivation, which comply with the satisfaction of the three basic psychological needs, were supported by the intervention: The teamwork during each simulation scenario had the potential to create the feeling of connection to their other students or peers (relatedness). The conduction of the debriefing was transferred to the students (competence) and hereby they had the autonomy over their own learning (autonomy) [30, 31]. An increase of autonomous motivation means that the locus of causality, the “why” of doing something, relocates from external to internal [31]. The intervention might have been too short to have such effects on students´ motivation.

Teaching and fostering NTS in healthcare education is inevitable to enhance patient safety and to provide best medical care. Therefore, it is the duty of us medical educators to provide curriculum approaches and facilitate the teaching and learning of NTS.

Our study draws a valuable implication for educational practise: We recommend the application of a flipped learning approach in combination with SBME to complement the learning process of NTS. Our suggested approach is easy assessable, easy to implement and cost-effective. The brief seminar of the intervention can be replaced with a video which can be provided via online platforms.

While we paid plenty attention to the design of our study, some limitations provide a new scope for future research. One limitation is that we did not assess the learning styles of the students, which is an important determinant of learning success. As flipped learning and LBL are two different teaching approaches, it would have been interesting to access learning styles of the students and correlate them with their NTS performance and the teaching approach. Then more differentiated conclusions would have been possible to be drawn on the effect of flipped learning on NTS performance and motivation, with respect to students learning characteristics.

Furthermore, our data does not provide information which component of the flipped learning had the greatest effect on skill enhancement. We can only draw the conclusion that the multifactorial concept and the preparation to conduct the debriefing (individual learning space) resulted in enhanced NTS performance- but our results do no clarify if the conduction itself enhanced NTS as well. For this purpose, a further training and assessment would have been necessary. Future studies should explore the effect of peer-debriefing on NTS performance.

A further limitation of our study is that medical educators were not blinded. However, they were highly motivated to support the study as objectively as possible and not to be influenced by the information of students´ allocation. Furthermore, in order to minimize potential bias, they were instructed to finish their AS-NTS ratings before the students provided feedback.


Flipped learning is an ideal teaching approach to introduce complex teaching contents which include behavioural changes (skills). The combination of flipped learning and SBME leads to NTS enhancement.

Availability of data and materials

Data will be provided from the corresponding author on reasonable request.


  1. Donaldson MS, Corrigan JM, Kohn LT. To err is human: building a safer health system: National Academies Press; 2000.

  2. Cooper JB, Newbower RS, Long CD, McPeek B. Preventable anesthesia mishaps: a study of human factors. Anesthesiology. 1978;49(6):399–406.

    Article  Google Scholar 

  3. Flin R, Patey R. Improving patient safety through training in non-technical skills. British Medical Journal Publishing Group; 2009.

  4. Peltonen V, Peltonen L-M, Salanterä S, Hoppu S, Elomaa J, Pappila T, et al. An observational study of technical and non-technical skills in advanced life support in the clinical setting. Resuscitation. 2020;153:162–8.

    Article  Google Scholar 

  5. Greif R, Bhanji F, Bigham BL, Bray J, Breckwoldt J, Cheng A, et al. Education, Implementation, and Teams: 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations. Circulation. 2020;142(16_suppl_1):S222-S83.

    Article  Google Scholar 

  6. Norris EM, Lockey AS. Human factors in resuscitation teaching. Resuscitation. 2012;83(4):423–7.

    Article  Google Scholar 

  7. Flin RH, O’Connor P, Crichton M. Safety at the sharp end: a guide to non-technical skills: Ashgate Publishing, Ltd.; 2008.

  8. Kiesewetter J, Gutmann J, Drossard S, Salas DG, Prodinger W, Mc Dermott F, et al. The Learning Objective Catalogue for Patient Safety in Undergraduate Medical Education–A Position Statement of the Committee for Patient Safety and Error Management of the German Association for Medical Education. GMS journal for medical education. 2016;33(1).

  9. WHO. WHO Patient Safety Curriculum Guide for Medical Schools 2009 2009 [Available from:

  10. Weinger MB. Experience ≠ ExpertiseCan Simulation Be Used to Tell the Difference? The Journal of the American Society of Anesthesiologists. 2007;107(5):691–4.

    Google Scholar 

  11. Moll-Khosrawi P, Kamphausen A, Hampe W, Schulte-Uentrop L, Zimmermann S, Kubitz JC. Anaesthesiology students’ Non-Technical skills: development and evaluation of a behavioural marker system for students (AS-NTS). BMC medical education. 2019;19(1):205.

    Article  Google Scholar 

  12. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003–2009. Medical education. 2010;44(1):50–63.

    Article  Google Scholar 

  13. Greif R, Lockey A, Breckwoldt J, Carmona F, Conaghan P, Kuzovlev A, et al. European Resuscitation Council Guidelines 2021: Education for resuscitation. Resuscitation. 2021.

  14. Mehta NB, Hull AL, Young JB, Stoller JK. Just imagine: new paradigms for medical education. Academic Medicine. 2013;88(10):1418–23.

    Article  Google Scholar 

  15. Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Medical education. 2017;51(6):585–97.

    Article  Google Scholar 

  16. El Miedany Y. Flipped learning. Rheumatology Teaching: Springer; 2019. p. 285–303.

    Google Scholar 

  17. Hurtubise L, Hall E, Sheridan L, Han H. The flipped classroom in medical education: engaging students to build competency. Journal of Medical Education and Curricular Development. 2015;2:JMECD. S23895.

  18. Heiss ED, Obourn S, Hoffman CW. Modern science teaching. ACS Publications; 1950.

  19. Lawson AE. Science teaching and the development of thinking: Wadsworth Belmont, CA; 1995.

  20. Hwang G-J, Chang C-Y. Facilitating decision-making performances in nursing treatments: a contextual digital game-based flipped learning approach. Interactive Learning Environments. 2020:1–16.

  21. Bergmann J, Sams A. Flipped learning: Gateway to student engagement: International Society for Technology in Education; 2014.

  22. Chang B-Y, Chang C-Y, Hwang G-H, Kuo F-R. A situation-based flipped classroom to improving nursing staff performance in advanced cardiac life support training course. Interactive Learning Environments. 2019;27(8):1062–74.

    Article  Google Scholar 

  23. Al-Zahrani AM. From passive to active: The impact of the flipped classroom through social learning platforms on higher education students’ creative thinking. British Journal of Educational Technology. 2015;46(6):1133–48.

    Article  Google Scholar 

  24. Lai C-L, Hwang G-J. A self-regulated flipped classroom approach to improving students’ learning performance in a mathematics course. Computers & Education. 2016;100:126–40.

    Article  Google Scholar 

  25. Schulte-Uentrop L, Cronje JS, Zöllner C, Kubitz JC, Sehner S, Moll-Khosrawi P. Correlation of medical students’ situational motivation and performance of non-technical skills during simulation-based emergency training. BMC Medical Education. 2020;20(1):1–7.

    Article  Google Scholar 

  26. DeChurch LA, Mesmer-Magnus JR. Measuring shared team mental models: A meta-analysis. Group dynamics: Theory, research, and practice. 2010;14(1):1.

  27. Taylor DC, Hamdy H. Adult learning theories: Implications for learning and teaching in medical education: AMEE Guide No. 83. Medical teacher. 2013;35(11):e1561-e72.

    Article  Google Scholar 

  28. Tang F, Chen C, Zhu Y, Zuo C, Zhong Y, Wang N, et al. Comparison between flipped classroom and lecture-based classroom in ophthalmology clerkship. Medical education online. 2017;22(1):1395679.

    Article  Google Scholar 

  29. Deci EL, Ryan RM. Self-determination theory Handbook of theories of social psychology. 1. London: SAGE Publications Ldt; 2011. p. 416–33.

    Google Scholar 

  30. Kusurkar RA. Autonomous motivation in medical education. Medical teacher. 2019;41(9):1083–4.

    Article  Google Scholar 

  31. Deci EL, Ryan RM. The” what” and” why” of goal pursuits: Human needs and the self-determination of behavior. Psychological inquiry. 2000;11(4):227–68.

    Article  Google Scholar 

  32. van der Burgt SM, Kusurkar RA, Wilschut JA, Tsoi SLTA, Croiset G, Peerdeman SM. Motivational profiles and motivation for lifelong learning of medical specialists. Journal of Continuing Education in the Health Professions. 2018;38(3):171–8.

    Article  Google Scholar 

  33. Kusurkar R, Ten Cate TJ, Van Asperen M, Croiset G. Motivation as an independent and a dependent variable in medical education: a review of the literature. Med Teach. 2011;33(5):e242-e62.

    Article  Google Scholar 

  34. Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC medical education. 2018;18(1):1–12.

    Article  Google Scholar 

  35. Suresh K. An overview of randomization techniques: an unbiased assessment of outcome in clinical research. Journal of human reproductive sciences. 2011;4(1):8.

    Article  Google Scholar 

  36. Wickström G, Bendix T. The” Hawthorne effect”—what did the original Hawthorne studies actually show? Scandinavian journal of work, environment & health. 2000:363–7.

  37. Network FL. The Four Pillars of F-L-I-PTM 2004 [Available from:

  38. Guay F, Vallerand RJ, Blanchard C. On the assessment of situational intrinsic and extrinsic motivation: The Situational Motivation Scale (SIMS). Motivation and emotion. 2000;24(3):175–213.

    Article  Google Scholar 

  39. Gillet N, Vallerand RJ, Lafreniere M-AK, Bureau JS. The mediating role of positive and negative affect in the situational motivation-performance relationship. Motivation and Emotion. 2013;37(3):465–79.

    Article  Google Scholar 

  40. Knörzer L, Brünken R, Park B. Facilitators or suppressors: Effects of experimentally induced emotions on multimedia learning. Learning and Instruction. 2016;44:97–107.

    Article  Google Scholar 

  41. Smidt A, Balandin S, Sigafoos J, Reed VA. The Kirkpatrick model: A useful tool for evaluating training outcomes. Journal of Intellectual and Developmental Disability. 2009;34(3):266–74.

    Article  Google Scholar 

  42. Stewart GL, Barrick MR. Team structure and performance: Assessing the mediating role of intrateam process and the moderating role of task type. Academy of management Journal. 2000;43(2):135–48.

    Article  Google Scholar 

  43. Darling M, Parry C, Moore J. Learning in the thick of it. Harvard business review. 2005;83(7):84.

    Google Scholar 

  44. Sweller J. Cognitive load theory. Psychology of learning and motivation. 55: Elsevier; 2011. p. 37–76.

  45. Dewey J. The reflex arc concept in psychology. Psychological review. 1896;3(4):357.

    Article  Google Scholar 

  46. Issenberg SB MW, Petrusa ER, Lee Gordon D, Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27: 10–28. doi: PubMed: 16147767. I. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach 27: 10–28 2005.

  47. Raemer D, Anderson M, Cheng A, Fanning R, Nadkarni V, Savoldelli G. Research regarding debriefing as part of the learning process. Simulation in Healthcare. 2011;6(7):S52-S7.

    Article  Google Scholar 

  48. Eddy ER, Tannenbaum SI, Mathieu JE. Helping teams to help themselves: Comparing two team-led debriefing methods. Personnel Psychology. 2013;66(4):975–1008.

    Article  Google Scholar 

  49. Reed S, Shell R, Kassis K, Tartaglia K, Wallihan R, Smith K, et al. Applying adult learning practices in medical education. Current Problems in Pediatric and Adolescent Health Care. 2014;44(6):170–81.

    Article  Google Scholar 

  50. Hurtubise L, Martin B, Gilliland A, Mahan J. To play or not to play: leveraging video in medical education. Journal of graduate medical education. 2013;5(1):13.

    Article  Google Scholar 

  51. Rudolph JW, Simon R, Raemer DB, Eppich WJ. Debriefing as formative assessment: closing performance gaps in medical education. Academic Emergency Medicine. 2008;15(11):1010–6.

    Article  Google Scholar 

  52. Sawyer TL, Deering S. Adaptation of the US Army’s after-action review for simulation debriefing in healthcare. Simulation in Healthcare. 2013;8(6):388–97.

    Article  Google Scholar 

  53. Beck S, Doehn C, Funk H, Kosan J, Issleib M, Daubmann A, et al. Basic life support training using shared mental models improves team performance of first responders on normal wards: A randomised controlled simulation trial. Resuscitation. 2019;144:33–9.

    Article  Google Scholar 

  54. Salas E, Burke CS, Stagl KC. Developing teams and team leaders: Strategies and principles. Leader development for transforming organizations: Growing leaders for tomorrow. 2004:325 – 55.

  55. Fry R, Kolb D. Experiential learning theory and learning experiences in liberal arts education. New directions for experiential learning. 1979;6:79.

    Google Scholar 

  56. Sweller J. Cognitive load theory, learning difficulty, and instructional design. Learning and instruction. 1994;4(4):295–312.

    Article  Google Scholar 

  57. Biggs J. Constructive alignment. Assessment@ Bond. 2016;1:25.

    Google Scholar 

  58. Kusurkar RA, Croiset G, Ten Cate OTJ. Twelve tips to stimulate intrinsic motivation in students through autonomy-supportive classroom teaching derived from self-determination theory. Med Teach. 2011;33(12):978–82.

    Article  Google Scholar 

Download references


We acknowledge the contribution of the instructors of the Department of Anaesthesiology, University Medical Center Hamburg-Eppendorf.


Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and Affiliations



PM-K made substantial contributions to conception and design, acquisition of data, analysis and interpretation of data. She has been involved in drafting the manuscript and given final approval of the version to be published. She agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CZ made substantial contributions to acquisition of data. He has been involved in revising the manuscript critically for important intellectual content and has given final approval of the version to be published. He agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. NC made substantial contributions to acquisition of data. She has been involved in revising the manuscript critically for important intellectual content and has given final approval of the version to be published. She agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. LS-U made substantial contributions to conception and design, acquisition of data, analysis and interpretation of data. She has been involved in drafting the manuscript and given final approval of the version to be published. She agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Authors' information

PM-K is a senior consultant at the Department of Anaesthesiology, University Medical Center Hamburg-Eppendorf and holds a Master of Medical Education. Next to her clinical work she is responsible for under- and postgraduate training. In her research, she mainly focusses on how non-technical skills can be assessed and conveyed effectively.

CZ is the chief of the Department of Anaesthesiology, University Medical Center Hamburg-Eppendorf. He supports with great enthusiasm the undergraduate and postgraduate training in anaesthesiology.

NC is a doctoral student, which was fascinated by the work of PM-K and LS-U on the field of non-technical skills. She therefore joined the working group.

LS-U PM-K is a senior consultant at the Department of Anaesthesiology, University Medical Center Hamburg-Eppendorf. Together with PM-K she has dedicated herself and research to the field of non-technical skills in under- and postgraduate education.

Corresponding author

Correspondence to Parisa Moll-Khosrawi.

Ethics declarations

Ethics approval and consent to participate

We contacted the local Ethic Committee of Hamburg (Ethikkommission der Ärztekammer Hamburg, Hamburg, Germany) with a detailed project description and the head of the committee did not see any necessity of deliberation and approved the project (§ 9 des Hamburgischen Kammergesetzes für Heilberufe).

All methods were carried out in accordance with relevant guidelines and regulations, in accordance with the Declaration of Helsinki.

Participation was voluntary and written informed consent was collected from each participant.

Competing interests


Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Moll-Khosrawi, P., Zöllner, C., Cencin, N. et al. Flipped learning enhances non-technical skill performance in simulation-based education: a randomised controlled trial. BMC Med Educ 21, 353 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: