Table 1 A “fit for purpose” framework for medical education accreditation system design

Mandate

The role and purpose of the accrediting body in reviewing the quality of educational programs, institutions, or systems.

Mandate

• Type of education or stage in the education continuum (undergraduate, postgraduate, or continuing professional development)

• Role in the education system (mandatory vs. optional) and implications for learner certification, licensure, or maintenance of certification

• Role of the accrediting body vis-à-vis government (legislated vs. non-legislated)

• Focus of accreditation (QA vs. QI vs. mixed model)

• Focus and scope of the accreditation system (national/local regulation vs. international comparison or benchmarking)

Accreditation standards (criteria, requirements)

Measures or generally accepted benchmarks used in making decisions about the quality of a program, institution, or system

Standards taxonomies

• Types of criteria or benchmarks included in the standards (structures vs. processes vs. outcomes vs. mixed model)

• Level of expectations (minimum standards vs. aspirational vs. mixed model)

• Framework as a basis for standards content, including international taxonomies such as that used by WFME

Process of standards development or renewal

• Content used for standards development and renewal (e.g., expert consensus, research evidence, government or regulatory directive, mixed model)

• Cycle of standards evaluation and renewal (planned vs. unplanned)

• Process of review (top down versus bottom up, who undertakes the review/evaluation, and who is involved [e.g., medical professionals, policy-makers, government, regulatory authorities, learners and/or graduates])

Application for accreditation

The process of reviewing an initial request for accreditation by a program seeking to demonstrate compliance with established standards, and which results in a decision about whether to grant new (first-time) accreditation

Application process

• Timing (before learners enter the program vs. once they have entered or completed the program)

• Application process (whether it differs from the regular accreditation process) and process elements (e.g., paper-based review, review by external assessors on site or by telephone)

• Learner input (required vs. optional, or not included as part of the process)

• Required follow-up after a successful application (a regular cycle of follow-up vs. shorter cycles for new programs)

Requirements/ benchmarks for new accreditation

• Thresholds for a successful application (same threshold as established programs vs. a lower or higher threshold for new programs)

• Focus of initial application (initial focus on structures/processes vs. focus on outcomes for more established programs)

• Support to achieve new accreditation (education programs, coaching, or advisory services)

Self-study (self-evaluation, self-assessment)

The internal process of reflection undertaken by a program, institution, or system to evaluate compliance with externally established standards

Self-study requirement

• Whether self-study is optional or mandatory

Process of self-study

• Focus of self-study if required (on compliance with standards vs. descriptive narrative of program vs. emphasis on action plans to address identified areas for improvement)

• Tools used for self-study (checklists vs. qualitative questionnaires vs. objective data [e.g., learner and/or graduate outcome data, survey data])

• Use of self-study if required (submission of the self-study to the accrediting body vs. use for the program’s own improvement only)

External assessment of standards

The process of determining the level of compliance of a program, institution, or system with established accreditation standards, undertaken by individuals external to the program, institution, or system

Documentation

• Documentation available to inform the external assessment, such as results of program self-study, objective learner and graduate data (e.g., learner and/or graduate data, stakeholder feedback) or required documentation such as program policies

• Timing of the documentation’s availability (before a site visit vs. at the site visit)

External assessment process

• Basis of standards evaluation (validation of program’s self-study vs. objective review of program against standards)

• Type of review (paper-based vs. review by tele- or video-conference vs. in-person site review)

• Types of activities included in the review (review of documentation, site tours, meetings with key stakeholders, including learners and/or graduates)

Accreditation report

The final report by external evaluators regarding the level of compliance of the program, institution, or system with established standards

Report content

• Tools to evaluate standards compliance (checklists, rating scales, narrative descriptions)

• Type of report content (narrative/qualitative, vs. numerical/quantitative, vs. hybrid)

• Process of report creation (automatically generated on the basis of standards evaluation vs. written report by external assessors or accrediting body staff)

Accreditation decision

The final decision on accreditation status, and its associated follow-up, as determined by the accrediting body

Categories (types) of accreditation decisions (statuses)

• Types of accreditation status decisions, namely

 - binary (e.g., accredited/not accredited),

 - levels tied to degree of compliance attained (e.g., gold, silver, bronze)

 - levels tied to cycle length (e.g., 3 vs. 4 years),

 - levels tied to types of follow-up (e.g., regular accreditation review vs. focused review of follow-up report)

• Whether there is a process to recognize and/or share innovations or program best practices

Process of decision-making

• Criteria by which the decision is made (holistic judgment against overall criteria vs. established thresholds of compliance vs. evaluation of compliance with high-risk/minimum standards)

• Who makes the decision (expert committee vs. accreditation body staff vs. computer) and how (group consensus vs. computer algorithm)

• Information considered in making the decision (all available information vs. the accreditation only)

Impact of and follow-up after the accreditation decision

• Whether the accreditation decision is considered final, versus an opportunity to improve the accreditation decision on the basis of new information or progress made

• Impact of the accreditation decision and follow-up on learners within the program, e.g. on admissions, credentialing, certification, and maintenance of certification (see Mandate)

• Process of appeal, including

 - what information is considered in the appeal, and

 - at what level an appeal is possible (individual standards vs. overall decision)

• Transparency of the accreditation decision (public reporting/transparency vs. full confidentiality vs. hybrid [e.g., confidential details with published list of accredited programs])

Accreditation cycle

The phases of an accreditation process dictating how often each program, institution, or system is re-evaluated for compliance with the standards, including the types of phases and activities in the process and any follow-up activities that must occur between external assessments

Types of accreditation cycles

• Length of cycle (e.g., 4, 6, or 8 years)

• Whether the cycle is static for all programs or tied to the accreditation decision (longer for programs that demonstrate better compliance with standards)

• Type of follow-up required (follow-up or special visit, regular accreditation process, progress report)

• Types of activities throughout the cycle, ranging from a purely episodic cycle (once per cycle for full accreditation review) to more continuous cycles (e.g., those requiring regular submission of reports, activities, and/or data for ongoing monitoring)

Site review model

The approach used by the accrediting body in determining the composition of its external site review team, as well as processes for recruiting, assigning, training, and assessing team members

Types of external reviewers (site reviewers, external assessors) and team composition

• Type of external site reviewer or external assessor used (volunteer/part-time vs. professional/full-time)

• Expertise required by the site reviewer, e.g., peer review by physician or other health care professional, lay person with experience in accreditation, learners (students, residents, physicians in practice), mixed model

• Size of the survey team (number of reviewers dedicated to each program or review)

• Roles (whether there is a team lead or survey chair)

• Role of the accrediting body’s staff in the site visit (observer only vs. full participant vs. process expert)

Reviewer (assessor) training / professional development and assessment

• Types of training/professional development (workshops, mentorship programs, any formal training program and/or certification)

• Whether surveyors are formally assessed and, if so,

 - types of assessment used (review by survey chair or team members, review by staff or a committee, and implications of negative assessments)

 - what is done with the assessment results (used to inform reviewers’ own development vs. used to inform future accreditation assignments)

Accreditation system administration

The approaches used by the accrediting body to support the administration and operationalization of accreditation process; this component includes the business model, the technology used (if any), system review and improvement (including research and scholarship), and oversight and risk management

Technological infrastructure

• Little to no automation vs. systems with “electronic” paper vs. fully automated and digitized accreditation processes

 - automation may include internal scheduling/workflows, surveyor software, program software/portals, report generation software, technology to support ongoing data monitoring

 - different approaches to automation (“off-the-shelf” technological solutions vs. customized development)

System improvement

• Approach to system improvement, from ad hoc to systematic, regular process of system review and improvement

• Includes any focus on research and scholarship to improve the system of accreditation as well as that which is known about accreditation more generally

Oversight and risk management

• Different approaches to oversight and risk management, including

 - committee governance

 - public accountability

 - legal risk management

 - regulation and established standards (accreditation of accrediting agencies, e.g. WFME, International Society for Quality in Healthcare)

Business model

• Costs to accredited program or organization (free vs. annual fee or fee(s) associated with each accreditation activity)

• Business model for the accrediting agency (funded by government or external funding vs. cost recovery vs. revenue generation, vs. mixed model)