Table 1 Exploratory rating framework for written feedback – Feedback Quality Evaluation Form provided to blinded raters

▢ Weak

■ Lacking performance content altogether

o E.g.“Saw many cases of lung cancer”

■ Blank

■ Nonspecific

o E.g. “A pleasure to work with”, “Functions at PGY-1 level”

■ Irrelevant

o E.g.“Spends much time studying after clinic”

■ Based on second-hand information

o E.g. “Caused pt in Dr. X’s clinic to cry”

■ Predominantly evaluative without specific aim of performance improvement

o E.g.“above average level of knowledge”

▢ Neither weak nor strong

■ Mentions points of good performance in general

o E.g.“Good communicator”

■ Mentions areas for improvement in general

o E.g. “Should take more time with patients”

▢ Strong

■ Specific areas for improvement

o E.g.“Explore symptoms in more depth during review of systems”

■ Based on direct observation

o E.g.“Did not respond to patient comments about anxieties/worries on several occasions – work on questioning further to validate and explore patient concerns”

■ Relevant to course goals

o “When presenting case history, lung cases were better presented than prostate cases. Review prognostic features of CA prostate important in initial consultation discussion”

■ Explains the gap [between observed performance and explicit standard]

o “At this point in training would be expected to develop a differential diagnosis of at least 3 conditions or etiologies underlying a presenting symptom. Tends to focus only on the most likely cause – encouraged to think of other potential causes as well”