Skip to main content

Non-technical skills of Norwegian medical students at different training sites: a comparative, observational cohort study

Abstract

Purpose

Mastering non-technical skills (NTS) is a fundamental part of the training of new physicians to perform effectively and safely in the medical practice environment. Ideally, they learn these skills during medical school. Decentralized medical education is being implemented increasingly worldwide. Two of the three training sites studied, Bodø (a regional hospital) and Finnmark (a rural local hospital), implemented decentralized medical education. The third training site was the main campus in Tromsø, located at an urban university hospital. The training in Finnmark emphasised training in non-technical skills using simulation to a larger extent than the two other university campuses. This study aimed to compare the NTS performance of medical students in their last year of education at three different training sites of the same university.

Methods

This blinded cohort study included students from the three training sites who participated in identical multi-professional simulations over a six-year period. Eight raters evaluated the video recordings of eight students from each training site using the Norwegian Medical Students Non-Technical Skills (NorMS-NTS) tool. The NorMS-NTS tool, which comprises four categories and 13 elements, assesses the NTS of Norwegian medical students and assigns an overall global score. Pairwise significant differences in the NTS performance levels between the training sites studied were assessed using Tukey’s test.

Results

The overall NTS performance levels of the medical students from Finnmark (mean 4.5) were significantly higher than those of the students from Tromsø (mean 3.8) and Bodø (mean 3.5). Similarly, the NTS performance levels at category-level of the students in Finnmark were significantly higher than those of the students from Bodø and Tromsø. Except for one category, no significant differences were observed between the students from Bodø and Tromsø in terms of the overall or category-level NTS performance.

Conclusion

The NTS performance levels of the medical students from Finnmark, which implements rural, decentralized medical education, were significantly higher than those of the students from Tromsø and Bodø.

Peer Review reports

Introduction

The first medical school in Northern Norway, established at UiT – the Arctic University of Norway (UiT) 50 years ago [1], was the first rural-oriented medical education model in Europe to recruit physicians to the underserved population of northern Norway with the intent to improve the health care standards in the region [1]. The university implemented one year of training in rural general practice and local hospitals outside the campus [1]. Moreover, the university also prioritized applicants from Northern Norway [1] owing to an expected Salmon Effect, which hypothesizes that physicians, similar to salmons, return to the region they grew up [2]. A previous study concluded that the medical education program in Tromsø facilitate the recruitment of physicians to the northern regions [1].

Later, students in Tromsø were able to take the last two years of their medical education program in other parts of Northern Norway; Bodø and Finnmark. These two years consist of both classroom and clinical teaching. In 2009 UiT developed the Bodø model [3], a decentralized model wherein 24 medical students from UiT completed the sixth and last year of undergraduate medical education in Bodø, in addition to the placement in the fifth year of study [4]. The Bodø model aimed to address the limitations of clinical training capacity available in Tromsø [4]. Bodø is home to the second largest hospital in Northern Norway, and these students are located at that hospital [4]. This model was developed on the principle that the students followed the same schedule as that at Tromsø [3]. Academic training schedules at the two centers are largely similar, with only minor variations in learning activities. In 2017 UiT developed the Finnmark model for medical student training, a decentralized model wherein students complete the fifth and sixth year of medical school in the rural county of Finnmark rather than the main training site in Tromsø.

To ensure consistency in medical education, the quality of teaching must be assessed in decentralized education. Students in Bodø, Tromsø, and Finnmark undertake a common final exam. Examination results can be used to assess outcomes of students’ learnings at different training sites. Reports from UiT for the period from 2018–2023 revealed that 87.3% of the 490 students in Tromsø successfully passed the final exam. Moreover, the passing percentage of the students who received decentralized education was higher (91.2% of the 147 students in Bodø and 96% of the 50 students in Finnmark). The differences between the three learning sites are summarized in Table 1.

Table 1 Differences between learning sites

The Finnmark model places an extensive focus on the acquisition of non-technical skills (NTS) via the continuous use of simulations so that the students may achieve high levels of NTS. These NTS includes situational awareness, decision-making, communication, teamwork and leadership [5]. Previous studies have demonstrated the role of NTS of the health care professionals in patient safety [6]. Insufficient NTS have been identified as a contributing factor in 70% of adverse events occurring in hospital settings [7]. NTS include interpersonal skills and complement the necessary technical skills required for clinical practice [8]. In contrast, technical skills are the profession-specific competency possessed by health professionals and students [5, 9].

Researchers have debated the use of the term NTS [10]. Nevertheless, NTS remains the most commonly used description. NTS can be acquired via training [11], and higher levels of NTS have been shown to improve patient safety [5, 12]. Therefore, health professionals and students should undergo NTS training [13].

Evaluating the outcomes of training is an essential element of high-quality training. Because providing feedback to students and health professionals on their NTS levels will aid in increasing the focus on gaining the right skills, it is necessary to develop tools to assess the NTS of health professionals and students. Tools have been used to assess NTS in the field of aviation for decades [14], and have been developed to assess the NTS of health professionals since the beginning of 2000 [8, 15]. Previously, we have developed NorMS-NTS, a tool that assesses the NTS of Norwegian medical students, in 2022 [16]. The tool is specific for the Norwegian context and evidence of validity show single measure ICC levels in the same range as other validated NTS tools [17].

The training in Finnmark emphasised training in non-technical skills using simulation to a larger extent than the two other university campuses. This study aimed to compare the NTS performance of medical students in their last year of education at three different training sites of the same university.

Methods

Overview

This was an observational cohort study [18]. From the three cohorts (Finnmark, Bodø and Tromsø) eight medical students from each of the training sites were studied. The experimental variable was a different training site. Eight raters who were blinded to the training site assessed the NTS performance of the 24 medical students using the NorMS-NTS tool (Table 2). We compared the results of the statistical analysis thereafter.

Table 2 The NorMS-NTS tool

Setting

The students at all three UiT training sites studied participate in InterSim, a simulation-based multi-professional training program that encompasses different acute care situations with standardized scenarios, during the last term of the sixth year of their undergraduate program. The medical students were paired with nursing students forming teams, which also included radiographers and bioengineering students in some cases. The team has to work together to diagnose and treat the patient in a simulated acute care setting. The medical student is the team leader. We chose to compare the students NTS when participating in InterSim, as it is a mandatory one-day course with standardized scenarios at all three campuses.

We video recorded the sessions of the medical students from Finnmark, Bodø, and Tromsø participating in two different scenarios, with each scenario lasting 12–20 min. The first scenario involved a patient with sepsis, whereas the second scenario involved a patient with postoperative dyspnea. One or two trained physicians and nurses facilitated each simulation and debriefing. Although most scenarios employed a simulated patient, some scenarios were performed using a simulation manikin owing to the COVID-19 restrictions. The students performed all measurements, examinations, and tests on the patients and gathered details regarding pain and emotions. The facilitators provided the answers subsequently. The students informed the facilitators regarding the procedures they aimed to conduct if the equipment was missing, and the facilitator provided the results.

Video recordings

All participants were sixth-year medical students from Finnmark, Bodø, or Tromsø. When participating in the mandatory InterSim training they were asked if they wanted to participate in the project. We provided a thorough explanation regarding the objectives and aims of the study to the participants. Participation was voluntary and had no consequences for their education. We randomly selected eight video recordings with sufficient sound and image quality from the video recordings of over 100 teams acquired between 2018 and 2023. We designated identification numbers to the videos and rfnational simulation networks (InterRegSim, a national collaboration for simulation-based learning in the specialist health service in andomized their order. The teaching and training did not vary throughout the years, but there was different emphasis on training non-technical skills and use of simulation between study sites.

Raters

We contacted two national simulation networks (InterRegSim, a national collaboration for simulation-based learning in the specialist health service in Norway [19] and the Better & Systematic Team training [BEST] network, an international multi-professional team training program originating in Finnmark [20]) via email to recruit raters. These were chosen because they work with team training and simulation and are not part of the program. We offered gift cards of NOK 3000 to the 11 raters recruited. We recruited more raters than necessary to compensate for dropouts. Eight raters, comprising four men and four women aged 46–69 years (mean: 55 years), completed the task within a specified timeframe.

The raters had 22–44 (mean: 27.5) years of clinical experience; two raters did not answer this question. Six raters were registered nurses, whereas two were medical doctors. Seven raters reported prior experience with the NTS and/or NTS tools; the last rater did not answer this question.

All raters familiarized themselves with the NorMS-NTS tool and received a presentation of the tool from a researcher (KP) via Microsoft Teams. All raters received secure online access to the 24 videos. We assigned a two-digit study identification code to each video and a number to each rater. The raters who were blinded to the training sites studied rated all videos using the NorMS-NTS tool and returned the ratings to a researcher (KP) via e-mail. The raters were only aware of the identification numbers of the students.

Sample size

The NorMS-NTS is a new tool, and no previous study has assessed the NTS of Norwegian medical students. Consequently, we could not obtain any estimates of the prevalence or standard deviation or calculate the sample size [21]. The practical implications of the selected sample size were also important. Three raters rated 20 videos of the same length as those used in the present study in a previous study [17] and exhibited a nearly identical threshold for the number of videos each rater could rate. The assessment process is time-consuming and requires a focused rater. Therefore, we selected 24 videos and increased the number of raters to eight to increase the likelihood of accurately measuring the NTS performance levels [22].

The NorMS-NTS tool

The NorMS-NTS tool used in this study was developed to assess the NTS of Norwegian medical students via observation in 2022 (Table 2) [16]. This tool consists of four categories comprising 13 elements rated on a 5-point Likert scale. The global overall score is rated on a 7-point Likert scale.

Statistical analysis

We analyzed all data extracted from the NorMS-NTS forms of the raters using Statistical Analysis System (SAS 9.4). We compared the mean element-level, category-level, and overall NTS performance levels at different training sites subsequently. The null hypothesis was that no significant differences would be observed among the NTS performance of the students in the three cohorts. Tukey’s test [23] which is a test that adjust for type I-error was used to calculate significant difference [24].

Results

Overall NTS performance

The NTS performance levels of the medical students from Finnmark (mean 4.52 (0.25)) were significantly higher than those of the students from Bodø (mean 3.53 (0.25)) and Tromsø (mean 3.83 (0.25)) on a scale ranging from 1 to 7 (Table 3 and Fig. 1). However, we observed no significant difference between the NTS performance levels of the medical students from Tromsø and Bodø.

Table 3 Observed differences in NTS performance between the three training sites. Significant when p < 0.05
Fig. 1
figure 1

Overall score of NTS performance between the three training sites (CIM: Confidence interval of the mean)

NTS performance at the category level

The NTS performance levels of the students from Finnmark were significantly higher than those of the students from Bodø and Tromsø in all categories (Fig. 2). We observed no significant difference between the NTS performance levels of the students from Bodø and Tromsø, except in terms of the category of “decision making”. The NTS performance levels of the students from Bodø were significantly lower than those of the students from Tromsø for this category. The categories of “Communication” and “Teamwork” exhibited the most significant differences between Finnmark and the other training sites. These results were also correlated with the overall score.

Fig. 2
figure 2

Category score of NTS performance between the three training sites (CIM: Confidence interval of the mean)

NTS performance when comparing elements

The NTS performance levels of the students from Finnmark were significantly higher than those of their peers in Bodø in all elements (Table 3), except the following elements: “Establish mutual understanding,” “Understanding of team members role” and “Efficient use of team members” (Fig. 3). The element “Patient communication” exhibited the most significant difference. The scores for “Patient communication,” “Professional modesty,” “Flexibility,” “Uncertainty management,” and “Decision analysis” of the students from Finnmark were significantly higher than those of the students from Tromsø. We observed no significant difference between the students from Tromsø and Bodø in terms of these elements.

Fig. 3
figure 3

Element score of NTS performance between the three training sites (CIM: Confidence interval of the mean)

Discussion

The present study demonstrated that the overall and category-level NTS performance levels of the students from Finnmark were significantly higher than those of the students from Bodø and Tromsø. Moreover, we observed no significant difference between the NTS performance levels of the students from Tromsø and Bodø, except in the category “Decision making.” The scores of the students from Bodø for this category were significantly lower than those of the students from Tromsø. The NTS performance level of the students from Finnmark was significantly higher for the elements “Patient communication,” “Professional modesty,” “Flexibility,” “Uncertainty management,” and “Decision analysis.” This finding of consistent results, with no differences between the NTS performance of the students from Bodø and Tromsø, and significantly better NTS performance of the students from Finnmark supports the internal consistency of the tool. The present study is novel in that no previous study has evaluated the NTS performance of Norwegian medical students receiving decentralized medical education. Hence, the results of previous studies cannot be compared with our results.

UiT aimed to use multi-professional team training to enhance the NTS when developing the Finnmark model. The findings of the present study, a follow-up study conducted to evaluate its effect, indicate that the NTS performance levels of the students from Finnmark were superior. However, this finding can be attributed to several reasons.

First, selection bias may have influenced our results. All students in Finnmark and Bodø actively elected to pursue decentralized medical education during the fifth and sixth years of study. The students electing to go to rural areas may possess some distinct features prior to their choice that are associated with higher NTS levels. The student groups in Finnmark are smaller than those in Bodø and Tromsø. Small group teaching optimizes learning in healthcare [25]. The knowledge of the students increases when they can build their understanding with their peers [25]. Small group teaching also promotes team-building skills [25]. The student group in Finnmark is smaller; consequently, the lectures also contained small groups of students. Small-group teaching and small-group lectures are not equivalent [25]. However, the learning experience is correlated with the engagement of the students. Thus, small group lectures may facilitate higher learning outcomes than the bigger groups in Tromsø and Bodø. Furthermore, smaller groups may facilitate active participation, “face-to-face” contact between participants, and purposeful activities, which are the three key elements for small group teaching [25].

More of the learning occurs in general practice in Finnmark, with general practitioners acting as teachers. The students received one-on-one half-day training in gynecology from an experienced general practitioner. The students also participated in general practice group workshops. Three students and one experienced general practitioner consulted with patients requiring a medical interpreter. The patient had regular appointments, and one of the students regularly consulted with a medical interpreter via telephone. The remaining students and general practitioners observed the consultation and participated in discussions. All students conducted one consultation by themselves. These new teaching models may have affected NTS positively as students are more engaged and active in the process and will receive feedback on different level that might help them with NTS. Those effects warrant further studies.

Another noteworthy difference is that the training site located in Finnmark is a small local hospital, whereas the training sites located in Tromsø and Bodø are a large University hospital and a large regional hospital, respectively. The local hospital in Finnmark comprised more generalists, whereas regional and university hospitals comprised more branch specialists. Consequently, more generalists trained the students in Finnmark, which may have affected the results. Students are expected to be skilled professional generalists by the time they graduate from medical school with the ability to become lifelong learners. The outcomes may be affected if education is particularly narrow or specialized. There is also a possibility that the teachers and the whole community in rural Finnmark places higher value on NTS. Further studies should aim to clarify these findings.

Limitations

The scenarios were standardized. However, they were not performed in an identical manner by the facilitators as they had different levels of training and different ways of performing their roles. Facilitators may have affected the ability of the students to perform at their highest level negatively and positively. Notably, several facilitators were involved at each institution to mitigate the influence of individual facilitators.

This study included eight students randomly selected from each training site. Ideally, all students should have been assessed; however, this was not possible owing to practical limitations. We obtained 64 scores for each element, category, and the global score of the NorMS-NTS at each training site as eight raters participated in this study. The wide range of years the students were video recorded may also have influenced the outcome. The teaching and training did not vary throughout the years, but there was different emphasis on training non-technical skills and use of simulation between study sites. However, the generalizability of our findings remains unknown. Further studies must be conducted to validate these results and assess their applicability.

The NorMS-NTS is a novel assessment tool used to evaluate the NTS of Norwegian medical students. The process of collecting evidence for its validity is ongoing. Although not proven optimal for summative assessment, it is the only tool available to assess the NTS of Norwegian medical students. This may have affected the results. However, with one exception, we found no statistical difference between the NTS performance levels of the students from Bodø and Tromsø, which supports the reliability of the tool.

Conclusion

The NTS performance levels of the medical students in Finnmark were significantly higher than that of the students in Bodø and Tromsø. Further studies must explore the reasons for this discrepancy. However, our study demonstrated that rural decentralized medical education may yield better learning outcomes than standard education in large, centralized hospitals.

Availability of data and materials

Data is provided within the manuscript  and supplementary files.

References

  1. Aaraas IJ, Halvorsen PA, Aasland OG. Supply of doctors to a rural region: occupations of Tromsø medical graduates 1979–2012. Med Teach. 2015;37(12):1078–82.

    Article  Google Scholar 

  2. Bertelsen T. Hvor kommer lægene fra og hvor blir de av. Tidsskr Norske Lægeforening. 1963;83:861–70.

    Google Scholar 

  3. Olsen IP, Lillebo B, Ofstad EH, Østlyngen A, Lappegård KT. Time to expand decentralisation of medical education in Norway. Tidsskr Norske Lægeforening. 2021;141(11). https://tidsskriftet.no/en/2021/08/debatt/time-expand-decentralisation-medical-education-norway.

  4. Westlie Å L, Gaski M, Abelsen B, Grimstad H, Ofstad EH. Doctors educated in Bodø - who are they and where do they work? Tidsskr Nor Laegeforen. 2022;142(1). https://tidsskriftet.no/en/2022/01/original-article/doctors-educated-bodo-who-are-they-and-where-dothey-work.

  5. Flin R OCP. Safety at the sharp end: A guide to non-technical skills. Aldershot: Ashgate; 2008.

  6. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiol Scand. 2009;53(2):143–51.

    Article  Google Scholar 

  7. Fletcher G, McGeorge P, Flin R, RJ G, Maran N. The role of non-technical skills in anaesthesia: a review of current literature. Brit J Anaesth. 2002;88(3):418–29.

    Article  Google Scholar 

  8. Fletcher G, Flin R, McGeorge P, et al. Rating non-technical skills: developing a behavioural marker system for use in anaesthesia. Cogn Technol Work. 2004;6(3):165–71.

    Article  Google Scholar 

  9. Prineas S, Mosier K, Mirko C, Guicciardi S. Non-technical skills in healthcare. In: Donaldson L, Ricciardi W, Sheridan S, Tartaglia R, editors. Textbook of Patient Safety and Clinical Risk Management. Cham: Springer International Publishing; 2021. p. 413–34.

    Chapter  Google Scholar 

  10. Nestel D, Walker K, Simon R, Aggarwal R, Andreatta P. Nontechnical skills: An inaccurate and unhelpful descriptor? Simul Healthc. 2011;6(1):2–3.

    Article  Google Scholar 

  11. Hagemann V, Herbstreit F, Kehren C, Chittamadathil J, Wolfertz S, Dirkmann D, et al. Does teaching non-technical skills to medical students improve those skills and simulated patient outcome? Int J Med Educ. 2017;8:101–13.

    Article  Google Scholar 

  12. Neily J, Mills PD, Young-Xu Y, Carney BT, West P, Berger DH, et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304(15):1693–700.

    Article  Google Scholar 

  13. Gordon M, Darbyshire D, Baker P. Non-technical skills training to enhance patient safety: a systematic review. Med Educ. 2012;46(11):1042–54.

    Article  Google Scholar 

  14. Flin R, Martin L. Behavioral markers for crew resource management: a review of current practice. Int J Aviat Psychol. 2001;11(1):95–118.

    Article  Google Scholar 

  15. Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating system for surgeons’ non-technical skills. Med Educ. 2006;40(11):1098–104.

    Article  Google Scholar 

  16. Prydz K, Dieckmann P, Musson D, Wisborg T. The development of a tool to assess medical students’ non-technical skills – The Norwegian medical students’ non-technical skills (NorMS-NTS). Med Teach. 2023-05;45(5):516–23.

  17. Prydz K, Dieckmann P, Fagertun H, Musson D, Wisborg T. Collecting evidence of validity for an assessment tool for Norwegian medical students’ non-technical skills (NorMS-NTS): usability and reliability when used by novice raters. BMC Med Educ. 2023;23(1):865.

    Article  Google Scholar 

  18. Wang X, Kattan MW. Cohort studies: design, analysis, and reporting. Chest. 2020;158((1 Supplement)):S72–8.

    Article  Google Scholar 

  19. Hospital SU. InterRegSim [Available from: https://www.helse-stavanger.no/interregsim.

  20. Wisborg T, Brattebø G, Brattebø J, et al. Training multiprofessional trauma teams in Norwegian hospitals using simple and low cost local simulations. Educ Health (Abingdon). 2006;19(1):85–95.

    Article  Google Scholar 

  21. Kadam P, Bhalerao S. Sample size calculation. Int J Ayurveda Res. 2010;1(1):55–7.

    Article  Google Scholar 

  22. Andrade C. Sample size and its importance in research. Indian J Psychol Med. 2020;42(1):102–3.

    Article  Google Scholar 

  23. Montgomery DC. Design and analysis of experiments. 8th ed. Hoboken, N.J: Wiley; 2013.

    Google Scholar 

  24. Stoll A. Post hoc tests: Tukey honestly significant difference test. The SAGE encyclopedia of communication research methods 4. Los Angeles: SAGE Publications, Inc; 2017.

  25. Burgess A, van Diggele C, Roberts C, Mellis C. Facilitating small group learning in the health professions. BMC Med Educ. 2020;20(Suppl 2):457.

    Article  Google Scholar 

Download references

Funding

Open access funding provided by UiT The Arctic University of Norway (incl University Hospital of North Norway) This work was supported by the Northern Norway Regional Health Authority (Ref: HNF1368-17).

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed to the planning of this study. KP and TW recruited raters. KP did the videorecording of the students. HF and KP did the statistical analysis. HF, KP and TW did the interpretation of the work. KP drafted the manuscript. TW, PD and DM were major contributors to revising the manuscript. All authors contributed to the final version and gave approval for it to be published.

Corresponding author

Correspondence to Katrine Prydz.

Ethics declarations

Ethics approval and consent to participate

Educational studies that do not involve patients are exempt from ethical approval in Norway. The Regional Committee of North Norway for Medical and Health Research waived the requirement for a formal review of this study protocol (Ref: 2016/1539/REK nord) in 2016.

The Norwegian Centre for Research Data (NSD) was entrusted as the Data Protection Official for Research of the Finnmark Hospital Trust and approved the project in 2017 (NSD Ref: 57474/2017). The NSD ensures legal access to necessary personal data for research and provides data-protection services to all Norwegian universities.

We provided written and oral explanations about the purpose and objectives of the study to the students and obtained informed consent from all participants.

Services for sensitive data (TSD) at the University of Oslo provide a platform for public research institutions in Norway. Researchers can collect, store, and analyze sensitive data in a secure project area. We stored all videos used in this study on a TSD platform. We also conducted the rating of the students at TSD facilities.

Consent for publication

Not applicable.

Competing interests

Dieckmann holds a professorship with the University of Stavanger that was established by an unconditional grant by the Laerdal foundation to the university and is today financed by the University itself. He is in the leadership of the EuSim group, providing faculty development courses in healthcare simulation. The other authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Prydz, K., Dieckmann, P., Fagertun, H. et al. Non-technical skills of Norwegian medical students at different training sites: a comparative, observational cohort study. BMC Med Educ 24, 616 (2024). https://doi.org/10.1186/s12909-024-05597-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12909-024-05597-7

Keywords