Skip to content

Advertisement

You're viewing the new version of our site. Please leave us feedback.

Learn more

BMC Medical Education

Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Interprofessional collaboration milestones: advocating for common assessment criteria in graduate medical education

  • Majken T. Wingo1Email author,
  • Rachel DA Havyer1,
  • Nneka I. Comfere2,
  • Darlene R. Nelson3 and
  • Darcy A. Reed1
BMC Medical Education201515:149

https://doi.org/10.1186/s12909-015-0432-0

Received: 11 March 2015

Accepted: 4 September 2015

Published: 14 September 2015

Abstract

Background

Milestone-based assessments of resident physicians inform critical decisions regarding resident competence and advancement. Thus, it is essential that milestone evaluations are based upon strong validity evidence and that consistent evaluation criteria are used across residency programs. A common approach to assessment of interprofessional collaboration milestones is particularly important since standardized measures of individual resident competence in interprofessional collaboration have not been established.

Discussion

We propose that assessments of interprofessional collaboration in graduate medical education meet common criteria, namely, these assessments should: 1) measure competency of an individual resident, 2) occur in the context of an interprofessional team, 3) be ascertained via direct observation of the resident, 4) be performed in a real-world clinical practice setting (such as a hospital ward, outpatient clinic, or operating room). We present the evidence-based rationale for these criteria and cite examples of published assessment instruments that fulfill one or more of the criteria, however further research is needed to ensure fidelity of assessments.

Summary

The proposed criteria may assist residency educators as they endeavor to provide robust and consistent assessments of interprofessional collaboration milestones.

Background

In 2013, the Accreditation Council for Graduate Medical Education introduced the Educational Milestone framework for assessment of residents in the Next Accreditation System [1]. Milestones are a sequence of developmentally-based behaviors that residents are expected to demonstrate throughout training [2]. Milestones are used both to assess the competence of individual residents, as well as to support the accreditation of residency programs [1]. Since milestones may potentially inform high-stakes decisions regarding resident advancement and program accreditation, it is critical that milestone assessments are based on strong validity evidence and are performed according to uniform criteria across residency programs.

Each specialty has published a list of milestones, which implies consensus within specialties regarding what must be assessed. There is no consensus, however, regarding how milestone assessments must be performed. Although there is broad acknowledgement that all assessments should be supported by validity evidence according to established frameworks [3, 4], beyond this, it is left up to individual program directors and teaching faculty to decide how to assess each milestone. In the absence of common criteria for assessment, important decisions may be made using subjective and variable measures.

Lack of a common approach to assessment is particularly problematic for milestones related to interprofessional collaboration and teamwork because, unlike medical knowledge and clinical skills, nationally standardized measures of individual resident competence in interprofessional collaboration have not been established [5]. Although numerous teamwork instruments exist, the most extensively studied instruments with the strongest validity evidence, such as the Safety Attitudes Questionnaire [610] and the Team Climate Inventory [1113], assess clinical teams as a whole rather than individual residents on a team. Yet milestones require assessments of individual residents, including his/her personal contribution to the team function and individual ability to collaborate with various professionals.

Despite these unique assessment challenges, the pressure to rigorously measure collaboration among residents and their interprofessional teams is mounting. An increasing number of authorities cite team-based care as crucial to the practice of medicine and vital to health system reform [1416]. Effective teamwork can enhance patient safety and improve healthcare quality [17, 18], and a common approach to assessment of interprofessional collaboration is critical to ensure that residents are competent in this essential milestone.

Discussion

Given the importance of team-based care and the complexities of its assessment, residency educators need direction regarding best practices for assessing interprofessional collaboration milestones among their trainees. Residents and programs would benefit from consistency in assessments, especially since these data are used to make significant decisions about individuals and programs. Therefore, we propose common criteria for the assessment of interprofessional collaboration milestones that, in addition to the established standards for validity evidence [3, 4], include 4 components:
  1. 1)

    Assessment must measure competency of an individual resident

     
  2. 2)

    Assessment must be in the context of an interprofessional team (includes professions other than the resident’s profession)

     
  3. 3)

    Assessment must be ascertained via direct observation of the resident

     
  4. 4)

    Assessment must occur in a real-world clinical practice setting (such as a hospital ward, outpatient clinic, operating room, emergency room)

     

These criteria were derived from a comprehensive review of the literature regarding Graduate Medical Education (GME) milestones and interprofessional collaboration and teamwork assessment tools. Database search terms and strategy have been previously published [19], and were extended to encompass all specialties. The evidence-based rationale for each criterion is presented in the subsequent sections along with recommendations for existing assessment instruments that fulfill one or more of the criteria. Residency educators may choose to implement and/or adapt these instruments to assess interprofessional collaboration milestones.

Interprofessional collaboration assessment of the individual resident

Milestone assessment necessitates attestation of individual resident competence. Unfortunately, among published instruments measuring interprofessional collaboration, the instruments with the most robust validity evidence measure collaboration of the team as a whole rather than at the individual level [6, 9, 10, 12, 13, 1921]. Competencies for individual interprofessional collaboration have been proposed [16], but these competencies have not been translated into assessment tools. While the overall functioning of the team in clinical practice is of paramount importance, in residency training, milestone assessment requires measurement of the individual resident’s teamwork abilities and contributions to a team. The Ottawa Global Rating Scale [2225] (OGRS) is one example of a published instrument that is well-suited for evaluation of an individual on a team. The OGRS is a 7-point descriptively-anchored scale of directly observed teamwork behaviors (problem solving, situational awareness, leadership, resource utilization, and communication) that has been validated among multilevel trainees from various specialties within simulated scenarios [22]. An important limitation of the OGRS is that it has not been studied among residents in real-world clinical practice settings. It also focuses on crisis resource management, so its applicability to individuals that are not in crisis mode is debatable. Validity studies of the OGRS within various inpatient and outpatient clinical settings are a necessary next step to optimize its use for milestone assessment.

Collaboration assessment within an interprofessional team

Effective collaboration among professions enhances patient safety and healthcare quality [17, 18]. In GME, it is important to capture how well residents collaborate with individuals from multiple professions within the workplace. Tools such as the Nurse-Physician Collaboration Scale [26, 27], the ICU Nurse-Physician Questionnaire [28, 29], and the Jefferson Scale of Attitudes Toward Nurse-Physician Collaboration [3032] have been used among residents to assess attitudes toward collaboration of the nurse-resident physician dyad, yet very few instruments measure resident collaboration with other health professionals such as pharmacists, therapists, social workers, clinical assistants, and administrative staff. Incorporating perspectives from the full spectrum of health professionals would significantly enrich milestone assessment data and provide a more complete picture of resident competence.

Residency educators interested in capturing viewpoints from multiple professions may consider utilizing the Team Climate Inventory (TCI) [1113], which is a 44 to 65-item instrument that evaluates team participation, support, quality, discussion, clarity of objectives, and teamwork climate. Studies of TCI have included diverse non-physician professionals such as advanced practice providers, nurses, therapists, pharmacists, social workers, dieticians, clerical employees, and psychologists [1113, 19]. Validity evidence for the TCI includes content validity, internal structure, relationships to other variables, and has been correlated with patient outcomes [12, 13, 20, 21]. The TCI does not, unfortunately, assess individual teamwork behaviors but rather examines teamwork climate as a whole in a self-reported manner. The Teamwork Mini-Clinical Evaluation Exercise (T-MEX) [33, 34] is a second instrument that incorporates observations from multiple health professionals, however it has only been studied within undergraduate medical education. Validity studies among residents are required to determine the utility of the T-MEX within GME.

Interprofessional collaboration assessment via direct observation

Many evaluations of interprofessional collaboration rely upon resident opinion and self-assessment [3537]. This type of subjective assessment, while valuable to obtain an overall picture of teamwork in many settings [9, 10, 12, 13, 20, 21], requires insight into team functioning and personal performance that some trainees may not possess. The ability to accurately self-assess varies among learners [38], and the extent to which self-assessment skills can be learned and improved is uncertain [39, 40]. To our knowledge, residents’ ability to accurately assess their own milestone performance has not yet been explored, but research among practicing physicians suggests that they may lack insight into their own ability to work in teams [41]. In addition, milestone assessments are designed to measure observable behaviors and skills. For all of these reasons, direct observation is preferable to self-assessment alone for assessing interprofessional collaboration.

Observations may be performed by supervising faculty, resident peers, nurses, pharmacists, allied health professionals, and others who interact with residents in the workplace. Direct observations can also be obtained from trained raters within simulated settings. Several simulation-based instruments have been developed within general surgery and surgical subspecialties. The Observational Teamwork Assessment for Surgery (OTAS) [4246] measures communication and collaboration within the operating room using direct observation by raters trained to use this scale. Multiple studies have demonstrated validity evidence for the OTAS including content validity, response process, internal structure, relationships to other variables, and patient outcomes [4249]. The Nontechnical Skills Evaluation Instrument (NOTECHS) [5052] is similar to the OTAS in that it also assesses teamwork in the operating room using direct observation, but both are more commonly used for assessing teams or subteams rather than individuals. NOTECHS and OTAS have been simultaneously assessed and showed excellent agreement [50]. An additional tool, Non-Technical Skills for Surgeons (NOTSS), was initially validated in simulated operative settings and uses direct observation of individuals on a team [53]. NOTSS has substantial validity evidence and has helped inform the development of the SCORE (Surgical Council for Resident Education) modules for interprofessional education [54, 55]. Although the OTAS, NOTECHS and NOTSS were designed for use in the operating room, many of the content domains in these instruments, such as leadership, teamwork, communication, problem-solving, decision-making, and situation awareness, are applicable to numerous clinical environments including the hospital ward. Tools designed for use in operative simulation must be further adapted into diverse real-world practice settings to assure validity of these observations.

Interprofessional collaboration assessment in a clinical practice setting

The primary purpose for assessing residents using milestones and other measures of competency is to ensure that, by the end of training, residents are ready for unsupervised practice. For this reason, it is essential that interprofessional collaboration is assessed within real-world clinical practice settings [56]. Many assessments of interprofessional collaboration utilize simulated scenarios [22, 36, 57], and while simulation is an excellent environment for residents to prepare for real clinical encounters, performance within simulation may not always mimic performance with a real patient. It is well known that learner behavior is influenced by environmental factors [58, 59] including workload, work intensity, stress and burn-out [60, 61]. As a result, residents who perform well in the controlled environment of an educational simulation may struggle when faced with the numerous challenges inherent in the practice milieu.

The Multi-disciplinary Team Performance Assessment Tool [62, 63] is an example of a direct observation assessment that has been applied among residents in a real-world clinical context, namely an inpatient oncology unit. This instrument focuses on team communication and captures each individual team member’s contributions to the cancer treatment team. Validity evidence includes content validity, response process, internal structure, and relationships to other variables. The Resident Leadership Scale assesses directly observed team leadership skills among internal medicine residents in the setting of an inpatient hospital ward [64]. Established validity evidence includes content, internal structure, and correlations with other measures of teamwork. Both the Multi-disciplinary Team Performance Assessment Tool and the Resident Leadership Scale have each been studied in a single clinical context (inpatient hospital unit), therefore future studies should explore the utility of these instruments in other clinical settings such as outpatient clinics and operating rooms.

Summary

A common approach to the assessment of interprofessional collaboration milestones in GME is imperative. Effective teamwork is necessary to achieve safe, high quality patient care, and milestone evaluations inform crucial decisions within residency programs. Therefore, we suggest that measures of interprofessional collaboration milestones use direct observation of an individual resident on an interprofessional team in a real-world clinical practice setting. While many existing instruments meet one or more of these criteria, these instruments have significant limitations. Future studies should focus on adapting current assessments to more fully encompass these important components within graduate medical education clinical environments.

Declarations

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Division of Primary Care Internal Medicine, Department of Medicine, Mayo Clinic
(2)
Departments of Dermatology and Laboratory Medicine & Pathology, Mayo Clinic
(3)
Division of Pulmonary and Critical Care Medicine, Department of Medicine, Mayo Clinic

References

  1. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system--rationale and benefits. N Engl J Med. 2012;366(11):1051–6.View ArticleGoogle Scholar
  2. Caverzagie KJ, Iobst WF, Aagaard EM, Hood S, Chick DA, Kane GC, et al. The internal medicine reporting milestones and the next accreditation system. Ann Intern Med. 2013;158(7):557–9.View ArticleGoogle Scholar
  3. Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ. 2003;37(9):830–7.View ArticleGoogle Scholar
  4. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166. e167-116.View ArticleGoogle Scholar
  5. Dietz AS, Pronovost PJ, Benson KN, Mendez-Tellez PA, Dwyer C, Wyskiel R, et al. A systematic review of behavioural marker systems in healthcare: what do we know about their attributes, validity and application? BMJ Qual Saf. 2014;23(12):1031–9.View ArticleGoogle Scholar
  6. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6:44.View ArticleGoogle Scholar
  7. Sexton JB, Makary MA, Tersigni AR, Pryor D, Hendrich A, Thomas EJ, et al. Teamwork in the operating room: frontline perspectives among hospitals and operating room personnel. Anesthesiology. 2006;105(5):877–84.View ArticleGoogle Scholar
  8. Poley MJ, van der Starre C, van den Bos A, van Dijk M, Tibboel D. Patient safety culture in a Dutch pediatric surgical intensive care unit: An evaluation using the Safety Attitudes Questionnaire. Pediatr Crit Care Me. 2011;12(6):E310–6.View ArticleGoogle Scholar
  9. Davenport DL, Henderson WG, Mosca CL, Khuri SF, Mentzer RM, Residents WCS. Risk-adjusted morbidity in teaching hospitals correlates with reported levels of communication and collaboration on surgical teams but not with scale measures of teamwork climate, safety climate, or working conditions. J Am Coll Surg. 2007;205(6):778–84.View ArticleGoogle Scholar
  10. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AHS, Dellinger EP, et al. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf. 2011;20(1):102–7.View ArticleGoogle Scholar
  11. Goh TT, Eccles MP, Steen N. Factors predicting team climate, and its relationship with quality of care in general practice. BMC Health Serv Res. 2009;9:138.View ArticleGoogle Scholar
  12. Bower P, Campbell S, Bojke C, Sibbald B. Team structure, team climate and the quality of care in primary care: an observational study. Qual Saf Health Care. 2003;12(4):273–9.View ArticleGoogle Scholar
  13. Hann M, Bower P, Campbell S, Marshall M, Reeves D. The association between culture, climate and quality of care in primary health care teams. Fam Pract. 2007;24(4):323–9.View ArticleGoogle Scholar
  14. Core Principles & Values of Effective Team-Based Health Care. Institute of Medicine. October 2012. [https://www.accp.com/docs/positions/misc/IOM_TeamBased_Care_Principles_Values.pdf. Accessed 3/10/15]
  15. Making the case for interprofessional education. American Medical Association. June 19, 2014. [http://www.ama-assn.org/ama/pub/ama-wire/ama-wire/post/making-case-interprofessional-education. Accessed 3/10/15]
  16. Core Competencies for Interprofessional Collaborative Practice. Interprofessional Education Collaborative. May 2011. [http://www.aacn.nche.edu/education-resources/ipecreport.pdf. Accessed 3/10/15]
  17. Baker DP GS, Beaubien J, Salas E, Barach P. Medical teamwork and patient safety: the evidence-based relation. Rockville, MD: AHRQ Publication No 05–0053 Agency for Healthcare Research and Quality; 2005.Google Scholar
  18. Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care. 2004;13 Suppl 1:i85–90.View ArticleGoogle Scholar
  19. Havyer RD, Wingo MT, Comfere NI, Nelson DR, Halvorsen AJ, McDonald FS, et al. Teamwork assessment in internal medicine: a systematic review of validity evidence and outcomes. J Gen Intern Med. 2014;29(6):894–910.View ArticleGoogle Scholar
  20. Proudfoot J, Jayasinghe UW, Holton C, Grimm J, Bubner T, Amoroso C, et al. Team climate for innovation: what difference does it make in general practice? Int J Qual Health C. 2007;19(3):164–9.View ArticleGoogle Scholar
  21. Gosling AS, Westbrook JI, Braithwaite J. Clinical team functioning and IT innovation: a study of the diffusion of a point-of-care online evidence system. J Am Med Inform Assoc. 2003;10(3):244–51.View ArticleGoogle Scholar
  22. Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J. A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study. Crit Care Med. 2006;34(8):2167–74.View ArticleGoogle Scholar
  23. Plant JL, van Schaik SM, Sliwka DC, Boscardin CK, O'Sullivan PS. Validation of a self-efficacy instrument and its relationship to performance of crisis resource management skills. Adv Health Sci Educ Theory Pract. 2011;16(5):579–90.View ArticleGoogle Scholar
  24. Kim J, Neilipovitz D, Cardinal P, Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (abbreviated as "CRM simulator study IB"). Simul Healthc. 2009;4(1):6–16.View ArticleGoogle Scholar
  25. Balki M, Cooke ME, Dunington S, Salman A, Goldszmidt E. Unanticipated difficult airway in obstetric patients: development of a new algorithm for formative assessment in high-fidelity simulation. Anesthesiology. 2012;117(4):883–97.View ArticleGoogle Scholar
  26. Ushiro R. Nurse-Physician Collaboration Scale: development and psychometric testing. J Adv Nurs. 2009;65(7):1497–508.View ArticleGoogle Scholar
  27. Nair DM, Fitzpatrick JJ, McNulty R, Click ER, Glembocki MM. Frequency of nurse-physician collaborative behaviors in an acute care hospital. J Interprof Care. 2012;26(2):115–20.View ArticleGoogle Scholar
  28. Boyle DK, Kochinda C. Enhancing collaborative communication of nurse and physician leadership in two intensive care units. J Nurs Adm. 2004;34(2):60–70.View ArticleGoogle Scholar
  29. Hamric AB, Blackhall LJ. Nurse-physician perspectives on the care of dying patients in intensive care units: collaboration, moral distress, and ethical climate. Crit Care Med. 2007;35(2):422–9.View ArticleGoogle Scholar
  30. Jones TS, Fitzpatrick JJ. CRNA-physician collaboration in anesthesia. Aana J. 2009;77(6):431–6.Google Scholar
  31. Garber JS, Madigan EA, Click ER, Fitzpatrick JJ. Attitudes towards collaboration and servant leadership among nurses, physicians and residents. J Interprof Care. 2009;23(4):331–40.View ArticleGoogle Scholar
  32. Ward J, Schaal M, Sullivan J, Bowen ME, Erdmann JB, Hojat M. The Jefferson Scale of Attitudes toward Physician-Nurse Collaboration: A study with undergraduate nursing students. J Interprof Care. 2008;22(4):375–86.View ArticleGoogle Scholar
  33. Olupeliyawa A, Balasooriya C, Hughes C, O'Sullivan A. Educational impact of an assessment of medical students' collaboration in health care teams. Med Educ. 2014;48(2):146–56.View ArticleGoogle Scholar
  34. Olupeliyawa AM, O'Sullivan AJ, Hughes C, Balasooriya CD. The Teamwork Mini-Clinical Evaluation Exercise (T-MEX): a workplace-based assessment focusing on collaborative competencies in health care. Acad Med. 2014;89(2):359–65.View ArticleGoogle Scholar
  35. Morgan PJ, Pittini R, Regehr G, Marrs C, Haley MF. Evaluating teamwork in a simulated obstetric environment. Anesthesiology. 2007;106(5):907–15.View ArticleGoogle Scholar
  36. Gettman MT, Pereira CW, Lipsky K, Wilson T, Arnold JJ, Leibovich BC, et al. Use of high fidelity operating room simulation to assess and teach communication, teamwork and laparoscopic skills: initial experience. J Urol. 2009;181(3):1289–96.View ArticleGoogle Scholar
  37. Novy D, Hamid B, Driver L, Koyyalagunta L, Ting J, Perez M, et al. Preliminary evaluation of an educational model for promoting positive team attitudes and functioning among pain medicine fellows. Pain Med. 2010;11(6):841–6.View ArticleGoogle Scholar
  38. Hodges B, Regehr G, Martin D. Difficulties in recognizing one's own incompetence: novice physicians who are unskilled and unaware of it. Acad Med. 2001;76(10 Suppl):S87–9.View ArticleGoogle Scholar
  39. Ward M, Gruppen L, Regehr G. Measuring self-assessment: current state of the art. Adv Health Sci Educ Theory Pract. 2002;7(1):63–80.View ArticleGoogle Scholar
  40. Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(10 Suppl):S46–54.View ArticleGoogle Scholar
  41. Arora S, Miskovic D, Hull L, Moorthy K, Aggarwal R, Johannsson H, et al. Self vs expert assessment of technical and non-technical skills in high fidelity simulation. Am J Surg. 2011;202(4):500–6.View ArticleGoogle Scholar
  42. Undre S, Healey AN, Darzi A, Vincent CA. Observational assessment of surgical teamwork: a feasibility study. World J Surg. 2006;30(10):1774–83.View ArticleGoogle Scholar
  43. Undre S, Sevdalis N, Healey AN, Darzi A, Vincent CA. Observational teamwork assessment for surgery (OTAS): refinement and application in urological surgery. World J Surg. 2007;31(7):1373–81.View ArticleGoogle Scholar
  44. Sevdalis N, Lyons M, Healey AN, Undre S, Darzi A, Vincent CA. Observational teamwork assessment for surgery: construct validation with expert versus novice raters. Ann Surg. 2009;249(6):1047–51.View ArticleGoogle Scholar
  45. Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N. Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg. 2011;212(2):234–43. e231-235.View ArticleGoogle Scholar
  46. Healey AN, Undre S, Vincent CA. Developing observational measures of performance in surgical teams. Qual Saf Health Care. 2004;13 Suppl 1:i33–40.View ArticleGoogle Scholar
  47. Hull L, Arora S, Kassab E, Kneebone R, Sevdalis N. Assessment of stress and teamwork in the operating room: an exploratory study. Am J Surg. 2011;201(1):24–30.View ArticleGoogle Scholar
  48. Russ S, Hull L, Rout S, Vincent C, Darzi A, Sevdalis N. Observational teamwork assessment for surgery: feasibility of clinical and nonclinical assessor calibration with short-term training. Ann Surg. 2012;255(4):804–9.View ArticleGoogle Scholar
  49. Wetzel CM, George A, Hanna GB, Athanasiou T, Black SA, Kneebone RL, et al. Stress management training for surgeons-a randomized, controlled, intervention study. Ann Surg. 2011;253(3):488–94.View ArticleGoogle Scholar
  50. Mishra A, Catchpole K, McCulloch P. The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Health Care. 2009;18(2):104–8.View ArticleGoogle Scholar
  51. Sevdalis N, Davis R, Koutantji M, Undre S, Darzi A, Vincent CA. Reliability of a revised NOTECHS scale for use in surgical teams. Am J Surg. 2008;196(2):184–90.View ArticleGoogle Scholar
  52. Steinemann S, Berg B, DiTullio A, Skinner A, Terada K, Anzelon K, et al. Assessing teamwork in the trauma bay: introduction of a modified "NOTECHS" scale for trauma. Am J Surg. 2012;203(1):69–75.View ArticleGoogle Scholar
  53. Yule S, Flin R, Maran N, Rowley D, Youngson G, Paterson-Brown S. Surgeons' non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg. 2008;32(4):548–56.View ArticleGoogle Scholar
  54. Yule S, Smink D. Competency-based surgical care: Nontechnical skills in surgery. ACS Surgery: Principles and Practice. 2013. http://www.notss.org/literature/. Accessed 7/22/15.
  55. Surgical Council on Resident Education (SCORE). Interpersonal and Communication Skills NOTSS-based modules. http://surgicalcore.org/modules. Accessed 7/22/15.
  56. Teherani A, Chen HC. The next steps in competency-based medical education: milestones, entrustable professional activities and observable practice activities. J Gen Intern Med. 2014;29(8):1090–2.View ArticleGoogle Scholar
  57. Undre S, Koutantji M, Sevdalis N, Gautama S, Selvapatt N, Williams S, et al. Multidisciplinary crisis simulations: the way forward for training surgical teams. World J Surg. 2007;31(9):1843–53.View ArticleGoogle Scholar
  58. Hafferty FW, Franks R. The hidden curriculum, ethics teaching, and the structure of medical education. Acad Med. 1994;69(11):861–71.View ArticleGoogle Scholar
  59. Hafferty FW. Beyond curriculum reform: confronting medicine's hidden curriculum. Acad Med. 1998;73(4):403–7.View ArticleGoogle Scholar
  60. West CP, Shanafelt TD. The influence of personal and environmental factors on professionalism in medical education. BMC Med Educ. 2007;7:29.View ArticleGoogle Scholar
  61. Dyrbye LN, Massie Jr FS, Eacker A, Harper W, Power D, Durning SJ, et al. Relationship between burnout and professional conduct and attitudes among US medical students. Jama. 2010;304(11):1173–80.View ArticleGoogle Scholar
  62. Lamb BW, Wong HW, Vincent C, Green JS, Sevdalis N. Teamwork and team performance in multidisciplinary cancer teams: development and evaluation of an observational assessment tool. BMJ Qual Saf. 2011;20(10):849–56.View ArticleGoogle Scholar
  63. Lamb BW, Sevdalis N, Mostafid H, Vincent C, Green JS. Quality improvement in multidisciplinary cancer teams: an investigation of teamwork and clinical decision-making and cross-validation of assessments. Ann Surg Oncol. 2011;18(13):3535–43.View ArticleGoogle Scholar
  64. Orlander JDWJ, Lew RA. Development of a tool to assess the team leadership skills of medical residents. Medical Education Online Vol. 2006;11:1–6.Google Scholar

Copyright

© Wingo et al. 2015

Advertisement