Table 2 List and Description of Themes Identified from Qualitative Content Analysis

1.

Identification of residents in difficulty [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21]

A. Assessment

Includes articles discussing the association between assessment tool type, accuracy or frequency, and the identification of residents in difficulty / in need of remediation

The utility of Standardized Direct Observation Tools (SDOT) and OSCEs to identify deficiencies in clinical performance (Medical Expert)

B. Faculty Development

Includes articles discussing how faculty development / training could help improve faculty’s ability to diagnose resident difficulties / deficiencies

Through improving the knowledge and skills of faculty around proper assessment and feedback methods; teaching faculty how to define and classify deficiencies so they can be more readily identified in practice

C. Other

Other topics discussed in relation to the identification of resident deficiencies

Factors predictive of resident deficiencies / their need for remediation in the future (e.g. age of application to residency, having transferred from another institution) [18]; identification of deficiencies by creating and having a better understanding of the taxonomy of the “problem resident” [19, 20]

2.

Discusses the importance of creating a framework for defining and classifying resident problems in an effort to design appropriate programs tailored to the issue(s) at hand. Addresses the questions of “What is the problem or deficiency?”; “What are its causes?”; “What signals a deficiency in this competency domain?”; etc.

Defining the problem through subjective and objective measures

Classifying problems, e.g. by rating their severity, determining whether the problem is inherent or contextual, what the contributing factors are

3.

Improving the accuracy and/or frequency of assessment to better track residents’ progress throughout their remediation

Advocating for more frequent formal and informal evaluations and feedback

Often found to improve resident learning outcomes

4.

Individualizing or tailoring of the remediation plan / program

A. To the resident

Many articles discuss the importance of tailoring remediation plans to the resident

Customizing the approach and structure and the resources needed for the remediation plan for each resident, with consideration given to the problem type and severity, and the resident’s characteristics (such as their learning style, personality, and level of insight).

B. To the specialty

Different specialties may require special consideration when developing a remediation plan, due to the unique nature of their training program and/or special clinical/training environment(s)

Emergency Medicine interns are found to have a significantly higher chance of under-performing than those in other disciplines, thought to be possibly related to differences in assessment practices and/or training environments

5.

Demonstrates the importance of having institutional-level consistency in the definitions of terms relating to remediation. This can help Program Directors (PDs), their faculty, and residents to better understand the expectations for training and the repercussions of not meeting them

Need to better differentiate and understand the difference between “need for improvement,” “need for remediation” (formal vs. informal), and “need for probation” (notice of potential for dismissal), e.g. CanMEDS, ACGME Roles

6.

Any strain on faculty as a result of their participation in the remediation of residents, including their time and effort spent and the complexity of their role(s)

Most refer to how time-consuming it is for faculty to participate in remediation programs and the great effort usually required

7.

Discusses the “hidden curriculum” in terms of attending role modelling, which can either positively or negatively impact residents

Faculty and residents need to be held to the same standards of professionalism, e.g. studies find that residents are given passes on their behaviour relative to the learner’s level of training

8.

Correlations observed between past performance (e.g. in medical school) and performance during residency

Performance in medical school may be predictive of performance in residency

9.

Discusses the pilot testing of a novel remediation plan or program, including program description and program effectiveness and/or outcomes

Looks at the effects of a

4-month training program implemented in an Emergency Medicine residency program to improve residents’ American Board of Emergency Medicine exam scores

10.

Understanding and identifying the many “players” involved in remediating residents, in identifying those in difficulty, and defining their roles

Identifying the key individuals to be involved in the remediation process and specifying their roles for varying severities of resident problems, e.G. minor problems can be managed by the ward or department, whereas more serious problems might merit a formal investigation

11.

Topic areas that were less prominent in the literature reviewed

Critique of literature on remediation (review); PD survey of incidence / prevalence of resident problems and possible predictors; plan for improved self-reflection integrated into remediation programs; general description of remedial process; and benchmark scale for residency training