Table 6 Assessment methods and outcome measures
Kirkpatrick Levels | Outcome measured | Assessment method |
|---|---|---|
Level 1: Participation in training | • Usefulness of the course [6, 34, 107, 108, 116, 139, 143, 145, 147, 171, 177, 188, 199,200,201] • Feedback on course structure and teaching methods [32, 37, 110, 116, 139, 143, 174, 199] • Satisfaction with the course [16, 131, 139, 143, 177, 179, 184, 201] | • Post-course survey [6, 16, 37, 108, 129, 131, 147, 167, 168, 178, 188, 189, 201] ◦ Using a Likert scale [6, 34, 107, 110, 116, 139, 143, 171, 177,178,179, 184] • Focus group session [37] |
Level 2a: Attitudes and perceptions | • Attitude and perceived importance towards patient-physician communication [16, 35, 99, 103, 126, 139, 146, 153, 161] • Attitude towards communication skills training [36] • Attitude on applying the skills learnt to regular practice [105] • Self-rated confidence in own communication skills [6, 7, 13, 16, 34,35,36,37, 103,104,105, 111, 116, 124,125,126, 128, 139, 141, 143, 145, 152, 177, 179, 184, 188, 199, 202,203,204] | • Pre- and post -course surveys [16, 36, 99, 105, 111, 139, 143, 145, 151, 153, 179, 188, 204] ◦ Post-course survey only [177] ◦ Using a Likert scale [34, 35, 99, 105, 116, 119, 184, 202] |
Level 2b: Knowledge and skill levels | • Self-rated skill levels [7, 35, 104, 107, 111, 112, 125, 128, 131, 141, 143, 146, 149, 151, 153, 155, 161, 165, 170, 189, 191, 199,200,201] • Self-rated knowledge level [37, 107, 139, 146] • Knowledge and skill levels as rated by: ◦ Experienced physician or psychologist or faculty staff or communication trainer [11, 13, 22, 32, 103, 104, 112, 141, 145, 154, 179, 189, 192, 195, 198, 200, 204,205,206,207,208] ◦ Simulated patients [12, 36, 131, 137, 145, 149, 163, 179, 186, 189, 203, 204, 208, 209] ◦ Patients and caregivers [140, 141, 160, 170, 173] • Patient satisfaction with doctor’s communication skills [138, 143] ◦ In-role simulated patient (SP) feedback [164] • Physician’s ability to detect and identify emotion ◦ Self-rated [33] ◦ Comparison between physician’s ability to detect patient's distress and patient’s self-reported distress [118] • Communication scenarios that physicians found difficulty in [116] • Content specific skills [99, 126, 137, 152] ◦ Preparedness to break bad news [34, 112, 116, 139, 141, 199, 200] ◦ Addressing end of life matters [35, 105, 109, 122, 125, 175, 178, 179, 184, 200, 204] ◦ Showing empathy [138] ◦ Discussing patient’s spiritual concerns [184] ◦ Health literacy [163] | • Pre- and post-course surveys and quizzes [34, 104, 107, 116, 125, 126, 139, 141, 143, 151, 153, 165, 193, 199] ◦ Post-course quiz only [189, 201] • Structured self-assessments [37, 191] • Clinical vignettes and case scenarios [139, 142] • Debrief and feedback session [36, 150, 155, 175, 209] ◦ Written feedback [9, 32, 114, 195] • Role playing scenarios ◦ Videotaped or audiotaped [32, 103, 104, 112, 114, 136, 141, 142, 163, 175, 179, 193, 199, 203, 205, 207, 210, 211] ◦ Conducted using simulated patients [11, 32, 34, 36, 103, 118, 129, 133, 137, 149, 163, 177, 179, 186, 189, 199, 203,204,205, 207, 212,213,214] • Video or audio recording of the physician’s interaction with real patients [9, 114, 118, 120, 134, 173, 178, 211] • Communication skills tools/ checklists [11, 34, 36, 103, 104, 131, 141,142,143, 145, 160, 173, 177, 189, 202, 206, 207] ◦ Specifically graded by patients [118, 137, 138, 140, 143, 173, 202] ◦ Tools to identify and detect emotion [33] |
Level 3: Change in behaviour | • Communication with patients and caregivers [153, 200] • Application of communication skill techniques taught [92, 124, 151] ◦ Frequency of skills practice after the course [6, 103, 124, 139, 146, 149, 173] ◦ Commitment to continued practice of the skills learnt [124] ◦ Rated according to a checklist by trained raters [169] | • Surveys [119, 124, 132, 167, 215] • Direct workplace observation [120, 153, 195, 200, 213, 216] ◦ Feedback from an interprofessional clinician and the patient [92] ◦ Unannounced SP visits [165] ◦ Videotaped encounters [169, 217] • Portfolio to record real life scenarios [151] • Role playing scenarios ◦ Annual evaluation with the use of an assessment form [7] |
Level 4: Long term change in performance and effect on patient care | • Patient’s satisfaction [14, 152, 153, 185, 215] • Post-consultation anxiety [152] • Communication about cancer screening [165] • Self-rated self-efficacy in challenging communication scenarios [124, 202] | • Survey/questionnaires [14, 185, 202] • Patient family surveys and semi structured interviews [153] • Direct observation ◦ Unannounced SP visits [165] |