Research article | Open | Open Peer Review | Published:
Science of health care delivery milestones for undergraduate medical education
BMC Medical Educationvolume 17, Article number: 145 (2017)
The changing healthcare landscape requires physicians to develop new knowledge and skills such as high-value care, systems improvement, population health, and team-based care, which together may be referred to as the Science of Health Care Delivery (SHCD). To engender public trust and confidence, educators must be able to meaningfully assess physicians’ abilities in SHCD. We aimed to develop a novel set of SHCD milestones based on published Accreditation Council for Graduate Medical Education (ACGME) milestones that can be used by medical schools to assess medical students’ competence in SHCD.
We reviewed all ACGME milestones for 25 specialties available in September 2013. We used an iterative, qualitative process to group the ACGME milestones into SHCD content domains, from which SHCD milestones were derived. The SHCD milestones were categorized within the current ACGME core competencies and were also mapped to Association of American Medical Colleges’ Entrustable Professional Activities (AAMC EPAs).
Fifteen SHCD sub-competencies and corresponding milestones are provided, grouped within ACGME core competencies and mapped to multiple AAMC EPAs.
This novel set of milestones, grounded within the existing ACGME competencies, defines fundamental expectations within SHCD that can be used and adapted by medical schools in the assessment of medical students in this emerging curricular area. These milestones provide a blueprint for SHCD content and assessment as ongoing revisions to milestones and curricula occur.
Current and future physicians must acquire and maintain a broad array of knowledge and skills to maximize health outcomes, many of which are not included in formal curricula or accreditation requirements. Historically, complexities of modern health care systems have not been addressed during traditional medical training, and physicians are not adequately prepared to practice within and improve complex health care systems . With the rapid change and reform in health care delivery today, physicians need to learn new knowledge and skills  in order to help achieve the Triple Aim outlined by the Institute for Healthcare Improvement . To address these gaps, physician education in novel domains within the Accreditation Council for Graduate Medical Education (ACGME) Systems-Based Practice and Practice-Based Learning and Improvement competencies is needed [4,5,6,7]. The content of these novel domains is broad and includes topics such as systems improvement, clinical informatics, high-value care, systems and human factors engineering, health policy, and population health [4, 8]. Taken together, this content has been referred to as the third science , healthcare delivery science , health systems science , and herein, the science of health care delivery (SHCD) . Although the terms vary, all refer to a new set of physician capabilities essential for practice that extend beyond the basic sciences and clinical skills traditionally taught in medical schools .
In order to engender public trust and confidence in physicians’ abilities, educators must be able to meaningfully measure physicians’ capabilities in SHCD. The ACGME has adopted milestones for measuring competence in graduate medical education (GME) , and the Association of American Medical Colleges (AAMC) has defined core Entrustable Professional Activities (EPAs) to ensure that medical students are prepared for residency training [14, 15].
With this in mind, we aimed to develop a set of SHCD milestones based on published ACGME milestones and corresponding to AAMC EPAs that can be used by medical schools to determine students’ competence in SHCD. These milestones define fundamental expectations within SHCD that are needed for all physicians to help optimize patients’ health outcomes within complex modern healthcare systems. The SHCD milestones may provide a blueprint for SHCD content and assessment, and inform future revisions to ACGME milestones. The authors report no competing interests.
We reviewed key references and consulted with experts in the areas of health systems, quality, and medical education to identify core topics related to the SHCD [2, 4, 16,17,18,19,20,21,22]. These topics included patient-centered care (individualized care, cultural competence, shared decision making, health behavior change), population-oriented care (public and population health, advocacy), high-value care (quality, safety, cost, health systems, practice improvement, information technology, clinical informatics), team-based care (inter-professional practice, care transitions), and healthcare policy, law, and regulation.
We reviewed the “Milestone Project” documents available on the ACGME Next Accreditation System website on September 11, 2013 for all 25 available specialties . These documents included 601 sub-competencies and 6667 individual milestone elements, which served as our unit of analysis. Each milestone element was reviewed by two authors, and elements that were felt to be related to the key SHCD topics by one or more of the authors were identified. A third author then reviewed and resolved any discrepancies with additional input from the group if uncertainties remained. All included milestone elements were then sorted via group discussion into subsets sharing a common theme. These themes informed the subsequent definition of SHCD sub-competencies which were categorized within the existing six ACGME core competencies based on established descriptions in the ACGME Common Program Requirements . If SHCD milestones mapped to more than one ACGME competency, the most relevant competency was selected.
The SHCD milestones were also mapped to AAMC Core EPAs based on descriptions provided in the AAMC Core EPAs Curriculum Developers’ Guide . Select ACGME milestone elements were incorporated into the SHCD milestones as appropriate. Milestone elements appropriate for an early UME level (e.g. ‘novice’ level) that did not exist in the ACGME milestones documents were developed. Finally, additional milestones were created to supplement SHCD topics that were missing or insufficiently robust in the ACGME milestones.
The resulting set of SHCD milestones is provided in Additional file 1. Each includes references to representative ACGME milestones upon which the SHCD milestones were based. Table 1 shows the categorization of the 15 sub-competencies for these SHCD milestones within the existing ACGME core competencies, along with corresponding AAMC EPAs.
The SHCD milestones are organized within 5 levels, similar to most ACGME specialty milestones . Level 1 milestone elements describe a novice level student and level 5 elements describe aspirational levels of performance typically achieved during residency or beyond.
Table 2 provides an example of the SHCD milestones related to shared decision making and motivating behavior change. Shared decision making is the process of understanding the patient’s values, exchanging information about available options and their consequences, and coming to a decision based on consensus . The aforementioned sub-competency incorporates these components of shared decision making between levels 1 to 5 by 1) supporting patient autonomy and eliciting patient values (“Uses directive style of guiding patient decisions” to “Develops care plans jointly with the patients and caregivers”); 2) utilizing appropriate communication strategies (“Uses technical terms and jargon” to “Consistently demonstrates communication strategies to ensure patient understanding and matches communication modality to the patient needs, health literary and context”); and 3) utilizing tools to assist in decision making and health behavior change (“Conducts interview without inquiring about social and behavioral factors that affect the health of individuals” to “Adapts decision aids and health behavior coaching techniques for patients who are not motivated to change”). The terms used to describe the target knowledge/skill level within these milestones progress according to Bloom’s taxonomy , with lower-level objectives at the critical deficiency and novice stages (e.g. ‘awareness’, ‘identify’) and higher-level objectives at the competent and aspirational stages (e.g. ‘demonstrates’, ‘implements’, ‘adapts’).
We propose this novel set of SHCD milestones, derived from published ACGME milestones and linked to AAMC EPAs, for use by medical schools to assess students’ knowledge and skills in SHCD. These milestones define fundamental expectations within SHCD and provide a blueprint for SHCD education and assessment that can be adaptable and responsive to ongoing milestone revisions. These milestones may serve as a starting point for implementation, further development, and refinement by medical schools based on the local curriculum, learning environment, and school-specific needs. While a unique SHCD curriculum is being implemented at our institution, these SHCD milestones were intentionally designed to be devoid of institution-specific contexts to enable broad applicability across educational settings. Until there is consensus among educators regarding a minimal set of SHCD student outcomes for UME, medical schools should individually determine whether and how to implement SHCD milestones and where along the spectrum of performance (e.g. from novice to aspirational) their desired graduation target would lie.
During our review of initial ACGME milestones, we identified several SHCD topics that were infrequently or never mentioned. These areas included patient advocacy, motivational interviewing, health behavior coaching, clinical informatics, and data analytics (beyond typical clinical use of electronic health records). As such, we developed new milestones for these topics in order to assist medical schools in measuring student capabilities in these emerging areas. Likewise, some SHCD topics did not readily map to any of the core EPAs for entering residency, such as healthcare costs and policy. We propose that these important topics also be considered for inclusion among ACGME milestones and EPAs as these undergo future revision.
To operationalize these SHCD milestones, medical schools will need to find time and space within their already established curricula to deliver relevant SHCD content as well as identify appropriate settings in which students’ performance in SHCD milestones can be observed and assessed . Rationale for incorporating SHCD content into existing curricula include not only preparing future physicians for the changing landscape of healthcare but also utilizing medical education as a driver for change in order to add value to the healthcare system [2, 6, 7, 27]. In ranking the importance of the gaps in physician competencies, the American Hospital Association report identified that the top physician gaps were in content areas of SHCD . Since formal, comprehensive SHCD curricula have not traditionally been integrated into medical education, faculty experts and content may not exist for many aspects of SHCD. In such cases, content creation and faculty development will be required. Since few tools for assessing SHCD knowledge and skills exist in the medical literature, it will be incumbent upon educators to develop new assessment tools, ideally those that are generalizable and can be easily adapted to different settings, and to collect validity evidence supporting their use.
These SHCD milestones have several limitations. First, we reviewed ACGME milestones at a single point in time. Yet, revisions have occurred over time and ongoing revisions are anticipated . The SHCD milestones included here are not intended to be static, and should be modified over time as experience with their use accrues. Second, our authorship team represents a single institution. However, the SHCD milestones are not specific to our organization and can be adapted for local use. Third, our team was comprised of the medical disciplines of internal medicine, pediatrics, and neurology, but did not include members of other disciplines such as surgical and procedural specialties. Fourth, we utilized all milestones available at the time of study (which included 25 specialties at that time), however milestones have subsequently been developed for the full complement of ACGME accredited specialties. Thus the SHCD milestones may have limited generalizability to the other sub-specialties until undergoing further revision by those sub-specialties. Finally, we recognize that the scope of SHCD represented by these milestones, while not completely comprehensive, is broad, making implementation of the entire set a challenge. Schools may opt to use a smaller set of SHCD milestones based on their individual priorities.
SHCD is an emerging curricular area within medical education, and a common set of SHCD milestones may help facilitate the successful preparation of medical students for residency and twenty-first century clinical practice. Valid assessments of students’ capabilities in SHCD may offer helpful metrics to determine their preparedness for residency in these domains. As a medical student would not be expected to demonstrate full competency in all domains by graduation, assessment of SHCD milestones will need to continue during GME and beyond. This novel set of SHCD milestones provides a blueprint for SHCD content and assessment, and a starting point for the medical education community as ongoing revisions to milestones and curricula occur within the emerging area of SHCD.
Association of American Medical Colleges
Accreditation Council of Graduate Medical Education
Entrustable Professional Activities
Graduate Medical Education
Science of Health Care Delivery
Undergraduate Medical Education
Skochelak SE. A decade of reports calling for change in medical education: what do they say? Acad Med. 2010;85(Suppl):26–33.
Combes JR and Arespacochaga E. Lifelong Learning Physician Competency Development. American Hospital Association’s physician leadership forum, Chicago, IL. June 2012.
Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff. 2008;27(3):759–69.
Lucey CR. Medical education: part of the problem and part of the solution. JAMA Intern Med. 2013;173:1639–43.
Pershing S, Fuchs VR. Restructuring medical education to meet current and future health care needs. Acad Med. 2013;88(12):1798–801.
Lin SY, Schillinger E, Irby DM. Value-added medical education: engaging future doctors to transform health care delivery today. J Gen Intern Med. 2015;30(2):150–1.
Sklar DP. How medical education can add value to the health care delivery system. Acad Med. 2016;91(4):446–7.
Gonzalo JD, Haidet P, Papp KK, Wolpaw DR, Moser E, Wittensten RD, et al. Educating for the 21st-century health care system: an interdependent framework of basic, clinical, and systems sciences. Acad Med. 2017;92(1):35–9.
Master of Health Care Delivery Science at Dartmouth. http://mhcds.dartmouth.edu/the-program/. Accessed 11 May 2017.
Gonzalo JD, Dekhtyar M, Starr SR, Borkan J, Brunett P, Fancher T, et al. Health systems science curricula in undergraduate medical education: identifying and defining a potential curricular framework. Acad Med. 2017;92(1):123–31.
Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery www.mayo.edu/research/centers-programs/robert-d-patricia-e-kern-center-science-health-care-delivery. Accessed 11 May 2017.
Gonzalo JD, Baxley E, Borkan J, Dekhtyar M, Hawkins R, Lawson L, et al. Priority areas and potential solutions for successful integration and sustainment of health systems science in undergraduate medical education. Acad Med. 2017;92(1):63–9.
Holmboe ES, Edgar L, Hamstra S. The Milestones Guidebook, Version 2016. www.acgme.org/Portals/0/MilestonesGuidebook.pdf. Accessed 11 May 2017.
Association of American Medical Colleges. Core Entrustable Professional Activities for Entering Residency: Curriculum Developers’ Guide. 2014. https://members.aamc.org/eweb/upload/Core%20EPA%20Curriculum%20Dev%20Guide.pdf. Accessed 11 May 2017.
Carraccio C, Englander R, Gilhooly J, Mink R, Hofkosh D, Barone MA, et al. Building a framework of entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med. 2017;92(3):324–30.
Institute of Medicine (U.S.) Crossing the quality chasm. A new health system for the 21st century. Washington, D.C.: The National Academies Press; 2001.
Smoldt RK, Cortese DA. Pay-for-performance or pay for value? Mayo Clin Proc. 2007;82:210–3.
Starfield B. Is patient-centered care the same as person-focused care? Perm J. 2011;15:63–9.
Lewis N. Populations, population health, and the evolution of population management: making sense of the terminology in US health care today. 2014. www.ihi.org/communities/blogs/_layouts/ihi/community/blog/itemview.aspx?List=81ca4a47-4ccd-4e9e-89d9-14d88ec59e8d&ID=50. Accessed 11 May 2017.
Mitchell P, Wynia M, Golden R, McNellis B, Okun S, Webb CE, et al. Core principles & values of effective team-based health care. 2012. https://www.nationalahec.org/pdfs/VSRT-Team-Based-Care-Principles-values.pdf. Accessed 11 May 2017.
Armstrong G, Headrick L, Madigosky W, Ogrinc G. Designing education to improve care. Jt Comm J Qual Patient Saf. 2012;38:5–14.
Interprofessional Education Collaborative Expert Panel. Core competencies for interprofessional collaborative practice: report of an expert panel. Washington, D.C.: Interprofessional Education Collaborative; 2011. https://www.aamc.org/download/186750/data/core_competencies.pdf. Accessed 23 Aug 2017.
Accreditation Council for Graduate Medical Education. Next Accreditation System: Milestones. www.acgme.org/acgmeweb/tabid/430/ProgramandInstitutionalAccreditation/NextAccreditationSystem/Milestones.aspx. Accessed 11 May 2017.
Accreditation Council for Graduate Medical Education. Common Program Requirements. 2017. http://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf. Accessed 23 Aug 2017.
Mayo Clinic Shared Decision Making National Resource Center. Shared decision making philosophy. 2016. http://shareddecisions.mayoclinic.org/decision-aid-information/decision-aids-for-chronic-disease/. Accessed 11 May 2017.
Anderson LW, Krathwohl DR. A taxonomy for learning, teaching, and assessing : a revision of Bloom's taxonomy of educational objectives. Complete ed. New York: Longman; 2001. p. 352.
Cooke M. Cost consciousness in patient care—what is medical education’s responsibility? N Engl J Med. 2010;362(14):1253–4.
Nasca TJ, Philibert I, Brigham T, Flyyn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051–6.
We thank Ling Werner for help organizing the data for this study.
This study was prepared with financial support from the American Medical Association as part of the Accelerating Change in Medical Education initiative, the Robert D. and Patricia E. Kern Family Foundation, and the Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery. The content reflects the views of the authors.
Availability of data and materials
The datasets analysed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Science of Health Care Delivery Milestones Developed for Undergraduate Medical Education with Corresponding Representative ACGME Milestones. This file contains the complete set of Science of Health Care Delivery milestones developed for undergraduate medical education. The milestones are grouped into 15 Science of Health Care Delivery sub-competencies based upon content domains and include references to representative ACGME milestones upon which the milestones were based. (DOCX 78 kb)