Skip to content


  • Research article
  • Open Access
  • Open Peer Review

Science of health care delivery milestones for undergraduate medical education

  • 1Email author,
  • 1,
  • 1,
  • 2,
  • 3,
  • 2 and
  • 2
BMC Medical EducationBMC series – open, inclusive and trusted201717:145

  • Received: 25 October 2016
  • Accepted: 22 August 2017
  • Published:
Open Peer Review reports



The changing healthcare landscape requires physicians to develop new knowledge and skills such as high-value care, systems improvement, population health, and team-based care, which together may be referred to as the Science of Health Care Delivery (SHCD). To engender public trust and confidence, educators must be able to meaningfully assess physicians’ abilities in SHCD. We aimed to develop a novel set of SHCD milestones based on published Accreditation Council for Graduate Medical Education (ACGME) milestones that can be used by medical schools to assess medical students’ competence in SHCD.


We reviewed all ACGME milestones for 25 specialties available in September 2013. We used an iterative, qualitative process to group the ACGME milestones into SHCD content domains, from which SHCD milestones were derived. The SHCD milestones were categorized within the current ACGME core competencies and were also mapped to Association of American Medical Colleges’ Entrustable Professional Activities (AAMC EPAs).


Fifteen SHCD sub-competencies and corresponding milestones are provided, grouped within ACGME core competencies and mapped to multiple AAMC EPAs.


This novel set of milestones, grounded within the existing ACGME competencies, defines fundamental expectations within SHCD that can be used and adapted by medical schools in the assessment of medical students in this emerging curricular area. These milestones provide a blueprint for SHCD content and assessment as ongoing revisions to milestones and curricula occur.


  • Science of health care delivery
  • Milestones
  • Undergraduate medical education
  • Assessment


Current and future physicians must acquire and maintain a broad array of knowledge and skills to maximize health outcomes, many of which are not included in formal curricula or accreditation requirements. Historically, complexities of modern health care systems have not been addressed during traditional medical training, and physicians are not adequately prepared to practice within and improve complex health care systems [1]. With the rapid change and reform in health care delivery today, physicians need to learn new knowledge and skills [2] in order to help achieve the Triple Aim outlined by the Institute for Healthcare Improvement [3]. To address these gaps, physician education in novel domains within the Accreditation Council for Graduate Medical Education (ACGME) Systems-Based Practice and Practice-Based Learning and Improvement competencies is needed [47]. The content of these novel domains is broad and includes topics such as systems improvement, clinical informatics, high-value care, systems and human factors engineering, health policy, and population health [4, 8]. Taken together, this content has been referred to as the third science [8], healthcare delivery science [9], health systems science [10], and herein, the science of health care delivery (SHCD) [11]. Although the terms vary, all refer to a new set of physician capabilities essential for practice that extend beyond the basic sciences and clinical skills traditionally taught in medical schools [12].

In order to engender public trust and confidence in physicians’ abilities, educators must be able to meaningfully measure physicians’ capabilities in SHCD. The ACGME has adopted milestones for measuring competence in graduate medical education (GME) [13], and the Association of American Medical Colleges (AAMC) has defined core Entrustable Professional Activities (EPAs) to ensure that medical students are prepared for residency training [14, 15].

With this in mind, we aimed to develop a set of SHCD milestones based on published ACGME milestones and corresponding to AAMC EPAs that can be used by medical schools to determine students’ competence in SHCD. These milestones define fundamental expectations within SHCD that are needed for all physicians to help optimize patients’ health outcomes within complex modern healthcare systems. The SHCD milestones may provide a blueprint for SHCD content and assessment, and inform future revisions to ACGME milestones. The authors report no competing interests.


We reviewed key references and consulted with experts in the areas of health systems, quality, and medical education to identify core topics related to the SHCD [2, 4, 1622]. These topics included patient-centered care (individualized care, cultural competence, shared decision making, health behavior change), population-oriented care (public and population health, advocacy), high-value care (quality, safety, cost, health systems, practice improvement, information technology, clinical informatics), team-based care (inter-professional practice, care transitions), and healthcare policy, law, and regulation.

We reviewed the “Milestone Project” documents available on the ACGME Next Accreditation System website on September 11, 2013 for all 25 available specialties [23]. These documents included 601 sub-competencies and 6667 individual milestone elements, which served as our unit of analysis. Each milestone element was reviewed by two authors, and elements that were felt to be related to the key SHCD topics by one or more of the authors were identified. A third author then reviewed and resolved any discrepancies with additional input from the group if uncertainties remained. All included milestone elements were then sorted via group discussion into subsets sharing a common theme. These themes informed the subsequent definition of SHCD sub-competencies which were categorized within the existing six ACGME core competencies based on established descriptions in the ACGME Common Program Requirements [24]. If SHCD milestones mapped to more than one ACGME competency, the most relevant competency was selected.

The SHCD milestones were also mapped to AAMC Core EPAs based on descriptions provided in the AAMC Core EPAs Curriculum Developers’ Guide [14]. Select ACGME milestone elements were incorporated into the SHCD milestones as appropriate. Milestone elements appropriate for an early UME level (e.g. ‘novice’ level) that did not exist in the ACGME milestones documents were developed. Finally, additional milestones were created to supplement SHCD topics that were missing or insufficiently robust in the ACGME milestones.


The resulting set of SHCD milestones is provided in Additional file 1. Each includes references to representative ACGME milestones upon which the SHCD milestones were based. Table 1 shows the categorization of the 15 sub-competencies for these SHCD milestones within the existing ACGME core competencies, along with corresponding AAMC EPAs.
Table 1

Sub-competencies for science of health care delivery milestones in undergraduate medical education

ACGME core competency [23]

Sub-competencies for SHCD milestones

Related AAMC core EPA functions or behaviors [14]

Systems-Based Practice

Develops skills in advocacy theory, execution and communication in order to advocate effectively for individual patients and at-risk populations

• Advocates for patient access to community resources (EPA 9)

Partners with the community to improve individual and population health

• Understands that population health issues affect the health of patients and therefore identifies sources of information about the needs and resources of the community; Interacts and begins to work collaboratively with community agencies, professionals, and others to address population health issues (EPA 3 and 7)

Defines value in health care and applies high value care strategies for individual patients and populations

• Incorporate cost awareness and principles of cost-effectiveness and pre-test/post-test probability in developing diagnostic plans (EPA 3)

Identifies systems failures and errors and contributes to a culture of safety and quality improvement

• Recognize and avoid errors by using safety alerts (e.g., drug-drug interactions) and information resources to place the correct order and maximize therapeutic benefit and safety for patients (EPA 4)

• Identify system failures and contribute to a culture of safety and improvement (EPA 13)

Describes United States’ healthcare financing and related effects on patient care and quality

• None

Analyzes current United States’ healthcare policy and its impact on health care delivery systems

• None

Describes the roles of clinical informatics, healthcare IT, and technology assessment in improving patient outcomes

• Uses tools and information technology to support decision making and adopt strategies to decrease cost and risk to individuals (EPA 3)

• Compose orders efficiently and effectively, such as by identifying the correct admission order set, selecting the correct fluid and electrolyte replacement orders, and recognizing the needs for deviations from standard order sets (EPA 4)

• Identify and demonstrate the use of information technology to access accurate and reliable online medical information (EPA 7)

• Document, and update, an electronic handover tool (EPA 8)

Interpersonal and Communication Skills

Develops patient-centered perspective and interviewing skills

• Demonstrate patient-centered interview skills (EPA 1)

• Apply the primary findings of one’s information search to an individual patient or panel of patients (EPA 7)

Collaborates as a member of an inter-professional team and demonstrates effective, team-based patient care

• Demonstrates an understanding of the roles of various team members by interacting with appropriate team members to accomplish assignments. Actively works to integrate into team function and meet or exceed the expectations of his or her given role (EPA 9)

Effectively leads an intra−/ inter-professional team in the clinical, educational, or research setting

• Collaborate as a member of an inter-professional team (EPA 9)

Effectively gives and receives a patient handover to transition care responsibility

• Give or receive a patient handover to transition care responsibility (EPA 8)


Individualizes care by engaging patients in shared decision making and motivating behavior change

• Elicit and take into account patient preferences in making recommendations (EPA 3)

• Discuss the planned orders and prescriptions… with patients and families and use a nonjudgmental approach to elicit health beliefs (EPA 4)

• Document patient preferences to allow their incorporation into clinical decision making (EPA 5)

Practices contextual and cultural humility, curiosity, and awareness

• Consider cultural and other factors that may influence the patient’s description of symptoms; Demonstrate cultural awareness and humility (EPA 1)

Practice-Based Learning and Improvement

Forms clinical questions and retrieves, appraises and assimilates evidence from the scientific literature

• Form clinical questions and retrieve evidence to advance patient care (EPA 7)

Patient Care

Applies principles of preventive health and strategies for population health management

• Understands that population health issues affect the health of patients and therefore identifies sources of information about the needs and resources of the community (EPA 3 and 7)

• Apply the primary findings of one’s information search to an individual patient or panel of patients (EPA 7)

Abbreviations: AAMC Association of American Medical Colleges; ACGME Accreditation Council for Graduate Medical Education; EPA Entrustable Professional Activities; IT Information Technology; SHCD Science of Health Care Delivery

The SHCD milestones are organized within 5 levels, similar to most ACGME specialty milestones [23]. Level 1 milestone elements describe a novice level student and level 5 elements describe aspirational levels of performance typically achieved during residency or beyond.

Table 2 provides an example of the SHCD milestones related to shared decision making and motivating behavior change. Shared decision making is the process of understanding the patient’s values, exchanging information about available options and their consequences, and coming to a decision based on consensus [25]. The aforementioned sub-competency incorporates these components of shared decision making between levels 1 to 5 by 1) supporting patient autonomy and eliciting patient values (“Uses directive style of guiding patient decisions” to “Develops care plans jointly with the patients and caregivers”); 2) utilizing appropriate communication strategies (“Uses technical terms and jargon” to “Consistently demonstrates communication strategies to ensure patient understanding and matches communication modality to the patient needs, health literary and context”); and 3) utilizing tools to assist in decision making and health behavior change (“Conducts interview without inquiring about social and behavioral factors that affect the health of individuals” to “Adapts decision aids and health behavior coaching techniques for patients who are not motivated to change”). The terms used to describe the target knowledge/skill level within these milestones progress according to Bloom’s taxonomy [26], with lower-level objectives at the critical deficiency and novice stages (e.g. ‘awareness’, ‘identify’) and higher-level objectives at the competent and aspirational stages (e.g. ‘demonstrates’, ‘implements’, ‘adapts’).
Table 2

Example of one of the 15 science of health care delivery sub-competencies with associated milestones

Individualizes care by engaging patients in shared decision making and motivating behavior change


Level 1

Level 2

Level 3

Level 4

Level 5

Uses directive style of guiding patient decisions

Respects patient autonomy in healthcare decisions

Discusses potential challenges associated with integrating patient values and beliefs with those of self, society, and the core values of medicine

Recognizes opportunities for shared decision making

Engages patients in shared decision making

Develops care plans jointly with patients and caregivers

Uses technical terms and jargon


Uses easy-to-understand language in all phases of communication

Assesses patient understanding of health information and invites questions

Consistently demonstrates communication strategies to ensure patient understanding and matches communication modality to the patient needs, health literacy and context

Conducts interview without inquiring about social and behavioral factors that affect the health of individuals

Identifies social and behavioral factors that affect the health of individuals and may be modifiable

Describes effective approaches to modifying individual health behaviors, such as shared decision making aids, motivational interviewing and health behavior coaching

Implements decision aids, motivational interviewing and/or health behavior coaching to modify individual health behaviors for patients who are motivated to change

Adapts decision aids and health behavior coaching techniques for patients who are not motivated to change

aRepresentative Corresponding ACGME Milestones [23]

Derm ICS1

Aero Med MK1, FM ICS1, TY Prof2, Occ Med MK1

Aero Med MK1, Occ Med MK1, Prev MK1, Psych ICS2, Derm ICS1, FM ICS2

Aero Med MK1, FM ICS2, Oph ICS1, Occ Med MK1, Prev MK1, Psych ICS2, Uro ICS2, IM ICS1

Aero Med MK1, FM ICS2, Neuro ICS, Occ Med MK1, Peds SBP1, Prev MK1, Psych ICS2 & Prof1,PS Prof1

aReferenced milestones are representative, not comprehensive. Reference Milestone Abbreviations: GME Graduate medical education, ICS interpersonal and communication skills, MK medical knowledge, Prof professionalism, SBP systems-based practice. Specialty Abbreviations: Aero Med aerospace medicine, FM family medicine, Neuro neurology, Occ Med occupational medicine, Oph ophthalmology, Peds pediatrics, PS plastic surgery, preventative/public health, Psych psychiatry, TY transitional year, Uro urology


We propose this novel set of SHCD milestones, derived from published ACGME milestones and linked to AAMC EPAs, for use by medical schools to assess students’ knowledge and skills in SHCD. These milestones define fundamental expectations within SHCD and provide a blueprint for SHCD education and assessment that can be adaptable and responsive to ongoing milestone revisions. These milestones may serve as a starting point for implementation, further development, and refinement by medical schools based on the local curriculum, learning environment, and school-specific needs. While a unique SHCD curriculum is being implemented at our institution, these SHCD milestones were intentionally designed to be devoid of institution-specific contexts to enable broad applicability across educational settings. Until there is consensus among educators regarding a minimal set of SHCD student outcomes for UME, medical schools should individually determine whether and how to implement SHCD milestones and where along the spectrum of performance (e.g. from novice to aspirational) their desired graduation target would lie.

During our review of initial ACGME milestones, we identified several SHCD topics that were infrequently or never mentioned. These areas included patient advocacy, motivational interviewing, health behavior coaching, clinical informatics, and data analytics (beyond typical clinical use of electronic health records). As such, we developed new milestones for these topics in order to assist medical schools in measuring student capabilities in these emerging areas. Likewise, some SHCD topics did not readily map to any of the core EPAs for entering residency, such as healthcare costs and policy. We propose that these important topics also be considered for inclusion among ACGME milestones and EPAs as these undergo future revision.

To operationalize these SHCD milestones, medical schools will need to find time and space within their already established curricula to deliver relevant SHCD content as well as identify appropriate settings in which students’ performance in SHCD milestones can be observed and assessed [12]. Rationale for incorporating SHCD content into existing curricula include not only preparing future physicians for the changing landscape of healthcare but also utilizing medical education as a driver for change in order to add value to the healthcare system [2, 6, 7, 27]. In ranking the importance of the gaps in physician competencies, the American Hospital Association report identified that the top physician gaps were in content areas of SHCD [2]. Since formal, comprehensive SHCD curricula have not traditionally been integrated into medical education, faculty experts and content may not exist for many aspects of SHCD. In such cases, content creation and faculty development will be required. Since few tools for assessing SHCD knowledge and skills exist in the medical literature, it will be incumbent upon educators to develop new assessment tools, ideally those that are generalizable and can be easily adapted to different settings, and to collect validity evidence supporting their use.

These SHCD milestones have several limitations. First, we reviewed ACGME milestones at a single point in time. Yet, revisions have occurred over time and ongoing revisions are anticipated [28]. The SHCD milestones included here are not intended to be static, and should be modified over time as experience with their use accrues. Second, our authorship team represents a single institution. However, the SHCD milestones are not specific to our organization and can be adapted for local use. Third, our team was comprised of the medical disciplines of internal medicine, pediatrics, and neurology, but did not include members of other disciplines such as surgical and procedural specialties. Fourth, we utilized all milestones available at the time of study (which included 25 specialties at that time), however milestones have subsequently been developed for the full complement of ACGME accredited specialties. Thus the SHCD milestones may have limited generalizability to the other sub-specialties until undergoing further revision by those sub-specialties. Finally, we recognize that the scope of SHCD represented by these milestones, while not completely comprehensive, is broad, making implementation of the entire set a challenge. Schools may opt to use a smaller set of SHCD milestones based on their individual priorities.


SHCD is an emerging curricular area within medical education, and a common set of SHCD milestones may help facilitate the successful preparation of medical students for residency and twenty-first century clinical practice. Valid assessments of students’ capabilities in SHCD may offer helpful metrics to determine their preparedness for residency in these domains. As a medical student would not be expected to demonstrate full competency in all domains by graduation, assessment of SHCD milestones will need to continue during GME and beyond. This novel set of SHCD milestones provides a blueprint for SHCD content and assessment, and a starting point for the medical education community as ongoing revisions to milestones and curricula occur within the emerging area of SHCD.



Association of American Medical Colleges


Accreditation Council of Graduate Medical Education


Entrustable Professional Activities


Graduate Medical Education


Science of Health Care Delivery


Undergraduate Medical Education



We thank Ling Werner for help organizing the data for this study.


This study was prepared with financial support from the American Medical Association as part of the Accelerating Change in Medical Education initiative, the Robert D. and Patricia E. Kern Family Foundation, and the Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery. The content reflects the views of the authors.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions

RDH, SMN, ANLH, SRS, TRL, and DAR reviewed all ACGME milestone elements and participated in data gathering and the iterative process of authoring the novel SHCD sub-competencies and milestones. RDH, SMN, ANLH, SRS, APW and DAR participated in writing/editing the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

Mayo Clinic, 200 First St. SW, Rochester, MN 55905, USA
Mayo Clinic School of Medicine, Mayo Clinic, Rochester, MN, USA
Gundersen Health System, LaCrosse, WI, 1900 South Avenue, La Crosse, WI 54601, USA


  1. Skochelak SE. A decade of reports calling for change in medical education: what do they say? Acad Med. 2010;85(Suppl):26–33.View ArticleGoogle Scholar
  2. Combes JR and Arespacochaga E. Lifelong Learning Physician Competency Development. American Hospital Association’s physician leadership forum, Chicago, IL. June 2012.Google Scholar
  3. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff. 2008;27(3):759–69.View ArticleGoogle Scholar
  4. Lucey CR. Medical education: part of the problem and part of the solution. JAMA Intern Med. 2013;173:1639–43.View ArticleGoogle Scholar
  5. Pershing S, Fuchs VR. Restructuring medical education to meet current and future health care needs. Acad Med. 2013;88(12):1798–801.View ArticleGoogle Scholar
  6. Lin SY, Schillinger E, Irby DM. Value-added medical education: engaging future doctors to transform health care delivery today. J Gen Intern Med. 2015;30(2):150–1.View ArticleGoogle Scholar
  7. Sklar DP. How medical education can add value to the health care delivery system. Acad Med. 2016;91(4):446–7.View ArticleGoogle Scholar
  8. Gonzalo JD, Haidet P, Papp KK, Wolpaw DR, Moser E, Wittensten RD, et al. Educating for the 21st-century health care system: an interdependent framework of basic, clinical, and systems sciences. Acad Med. 2017;92(1):35–9.View ArticleGoogle Scholar
  9. Master of Health Care Delivery Science at Dartmouth. Accessed 11 May 2017.
  10. Gonzalo JD, Dekhtyar M, Starr SR, Borkan J, Brunett P, Fancher T, et al. Health systems science curricula in undergraduate medical education: identifying and defining a potential curricular framework. Acad Med. 2017;92(1):123–31.View ArticleGoogle Scholar
  11. Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery Accessed 11 May 2017.
  12. Gonzalo JD, Baxley E, Borkan J, Dekhtyar M, Hawkins R, Lawson L, et al. Priority areas and potential solutions for successful integration and sustainment of health systems science in undergraduate medical education. Acad Med. 2017;92(1):63–9.View ArticleGoogle Scholar
  13. Holmboe ES, Edgar L, Hamstra S. The Milestones Guidebook, Version 2016. Accessed 11 May 2017.
  14. Association of American Medical Colleges. Core Entrustable Professional Activities for Entering Residency: Curriculum Developers’ Guide. 2014. Accessed 11 May 2017.
  15. Carraccio C, Englander R, Gilhooly J, Mink R, Hofkosh D, Barone MA, et al. Building a framework of entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med. 2017;92(3):324–30.View ArticleGoogle Scholar
  16. Institute of Medicine (U.S.) Crossing the quality chasm. A new health system for the 21st century. Washington, D.C.: The National Academies Press; 2001.Google Scholar
  17. Smoldt RK, Cortese DA. Pay-for-performance or pay for value? Mayo Clin Proc. 2007;82:210–3.View ArticleGoogle Scholar
  18. Starfield B. Is patient-centered care the same as person-focused care? Perm J. 2011;15:63–9.View ArticleGoogle Scholar
  19. Lewis N. Populations, population health, and the evolution of population management: making sense of the terminology in US health care today. 2014. Accessed 11 May 2017.Google Scholar
  20. Mitchell P, Wynia M, Golden R, McNellis B, Okun S, Webb CE, et al. Core principles & values of effective team-based health care. 2012. Accessed 11 May 2017.Google Scholar
  21. Armstrong G, Headrick L, Madigosky W, Ogrinc G. Designing education to improve care. Jt Comm J Qual Patient Saf. 2012;38:5–14.View ArticleGoogle Scholar
  22. Interprofessional Education Collaborative Expert Panel. Core competencies for interprofessional collaborative practice: report of an expert panel. Washington, D.C.: Interprofessional Education Collaborative; 2011. Accessed 23 Aug 2017.
  23. Accreditation Council for Graduate Medical Education. Next Accreditation System: Milestones. Accessed 11 May 2017.
  24. Accreditation Council for Graduate Medical Education. Common Program Requirements. 2017. Accessed 23 Aug 2017.
  25. Mayo Clinic Shared Decision Making National Resource Center. Shared decision making philosophy. 2016. Accessed 11 May 2017.
  26. Anderson LW, Krathwohl DR. A taxonomy for learning, teaching, and assessing : a revision of Bloom's taxonomy of educational objectives. Complete ed. New York: Longman; 2001. p. 352.Google Scholar
  27. Cooke M. Cost consciousness in patient care—what is medical education’s responsibility? N Engl J Med. 2010;362(14):1253–4.View ArticleGoogle Scholar
  28. Nasca TJ, Philibert I, Brigham T, Flyyn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051–6.View ArticleGoogle Scholar


© The Author(s). 2017