Skip to main content

Table 2 Themes & subthemes found in think-alouds

From: Pre-clerkship EPA assessments: a thematic analysis of rater cognition

STUDENT PERFORMANCE (504, 65%)
 Observable or inferred student activities - often described as skills.
  1) Student behavior (362, 72%)
   These statements describe what the student is doing. They typically include an evaluation of the performance as good or bad - but are occasionally neutral observations.
  2) Inferred clinical reasoning (100, 20%)
   How or what the assessor explicitly perceives the student to be thinking. This is frequently based on a conclusion about what the student is doing / has done.
  3) Patient rapport (42, 8%)
   This represents the assessor’s understanding of the relationship between the patient and the student. It often appears that out of concern for a compromised relationship or rapport that more supervision would be needed, and that a good relationship can compensate for poor performance in other domains.
FRAME OF REFERENCE (123, 16%)
 How the assessor understands the task at hand to include personal context or differences in understanding related to the purposes of the assessment.
  4) Future training needs (40, 33%)
   The assessor considers how much supervision the student will require in the future rather than how much supervision they currently require.
  5) Assessor preference/self (20, 16%)
   These assessments/impressions are based on self - the assessor’s personal opinion or their preferred/historical way of doing the task.
  6) Affective response (20, 16%)
   When the assessor references their emotional state/reaction as part of their decision making.
  7) Phase of training (17, 14%)
   The assessor is using a phase of training to determine what level of entrustment/supervision is appropriate rather than the student’s performance.
  8) Previous exposure to the same student (13, 11%)
   The assessor incorporates knowledge and impressions regarding performance based on previous experiences with that same student.
  9) Comparison with another student’s performance (7, 06%)
   The assessor is utilizing a normative style of assessment, comparing the student to another student’s performance.
  10) The curriculum (6, 5%)
   The assessor believes that the student is doing something incorrectly and ascribes this to the curriculum (i.e., the assessors believe that the students was taught incorrectly). In light of this, they consider the behavior acceptable.
ASSESSOR UNCERTAINTY (88, 12%)
 When the assessor questions their ability observe the student adequately.
  11) Assessor confidence in their ability to assess (40, 45%)
   The assessor questions their own observations, but not due to an actual compromise of information or insufficient number of assessments. This often seems to be an attempt to couch a judgment. Note that assessor commented at least every 4 min, so it is unlikely that they actually had recall issues.
  12) Compromised Information (27, 31%)
   The assessor was not able to adequately observe a student due to camera position or time running out.
  13) Insufficient number of assessments (21, 24%)
   The assessor mentions that the number of times they have worked with the student is compromising their assessment, namely, that they haven’t worked with the student enough to make an entrustment assessment.
THE PATIENT (49, 6%)
 Details specific to the patient, like acuity and risk associated with care.
  14) Patient characteristics (27, 45%)
   The assessor considers patients characteristics, complexity, symptomatology in entrustment decisions without consideration of the student’s ability or patient safety.
  15) Patient safety (22, 55%)
The assessor considers patient safety in their entrustment rating of the student. This has less to do with how the student is performing, and more to do with the potential risk to the patient associated with their signs/symptoms. For example, if an assessor reports concern because a student missed a critical question or physical exam maneuver, this would be coded as student behavior. This subtheme represents a fear that the patient may suffer (despite being simulated) regardless of the student’s performance
  1. N refers to how many times the subtheme/theme appeared in the transcripts. Theme percentage refers to total frequency. Subtheme percentages refers to the frequency within a theme