We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
Communicating wisely: teaching residents to communicate effectively with patients and caregivers about unnecessary tests
BMC Medical Educationvolume 17, Article number: 248 (2017)
With rising healthcare costs and a focus on quality, there is a growing need to promote resource stewardship in medical education. Physicians need to be able to communicate effectively with patients/caregivers seeking tests and treatments that are unnecessary.
This study aimed to evaluate the impact of an interactive workshop on residents’ knowledge of resource stewardship and communication skills when counseling patients/caregivers about requests for unnecessary testing.
Participants were 83 Internal Medicine and Pediatrics residents at the University of Toronto in 2014–15. The evaluation compared resource stewardship knowledge and communication skills of 57 (69%) residents that attended the resource stewardship workshop to 26 residents (31%) who did not. Knowledge and communication skills assessment consisted of a written test and a structured assessment using standardized patient raters, respectively. A linear regression was applied to determine predictors of overall communication skills performance.
Workshop attendance resulted in better performance on the knowledge test (4.3 ± 1.9 vs. 3.1 ± 1.7 out of 8, p = 0.01), but not better performance on the communication skills assessment (4.1 ± 0.8 vs. 4.0 ± 0.9 out of 5, p = 0.56). Higher training level (p = 0.01) and knowledge test scores (p = 0.046) were independent predictors of better overall communication skills, after adjusting for gender, training level, workshop attendance, knowledge and self-reported prior feedback on communication skills.
An interactive workshop can improve knowledge of resource stewardship, but improving communication skills with patients/caregivers about unnecessary testing may require additional training or reinforcement in the clinical learning environment. These teaching and assessment approaches can support the integration of education on resource stewardship into medical education.
Physicians have a professional obligation to ensure appropriate use of healthcare resources [1, 2]. Unnecessary tests and treatments that do not enhance medical care have significant implications for the well-being of patients as they can expose patients to potential harm including procedural complications, side effects such as radiation or nosocomial infection, and unnecessary follow-up testing, and can also have significant health system impact by contributing to rising healthcare costs [3,4,5,6,7]. Examples include imaging for uncomplicated headaches and lower back pain, computed tomography abdominal scan as an initial evaluation of a child with suspected appendicitis, pap smears for women who have had hysterectomies or are under the age of 21 or routine check-ups for healthy adults . Yet, current training does not adequately prepare physicians to deliver high-value care . In 2012, less than 15% of programs surveyed had a formal resource stewardship curriculum . Furthermore, residents who trained in clinical environments with high resource utilization tend to utilize more resources in future practice [11,12,13,14]. Therefore, there is a critical need to develop teaching and assessment strategies for resource stewardship education .
One driver of healthcare resource overuse is patient/caregiver request for unnecessary tests/treatments [16, 17]. The Choosing Wisely® campaign promotes conversations between physicians and patients/caregivers about unnecessary tests/treatments as a key strategy to ensure the delivery of high-value care [18,19,20]. Developing effective communication skills to address patient preference is essential for educating about resource stewardship, as demonstrated by a shared decision-making program with patients that led to fewer unnecessary antibiotic prescriptions .
A recent systematic review identified a number of curricula that teach key concepts related to resource stewardship, and identified the following factors that contribute to successful learning of resource stewardship concepts: 1) effective transmission of knowledge related to healthcare costs, practice guidelines, benefits and harms of healthcare, and patient preferences/values; 2) facilitation of reflective practice, including provision of feedback on laboratory ordering or prescribing practices; and 3) creation of a supportive environment with appropriate role modeling to promote a culture of high-value, cost-conscious care . While there are published examples of curricula focused on resource stewardship [23,24,25], only one of the included studies in the systematic review had a small component focused specifically on communication skills training related to unnecessary tests and treatments . Our study aims to evaluate the impact of a resource stewardship workshop on residents’ knowledge of resource stewardship and communication skills when counseling patients/caregivers about requests for unnecessary testing.
Setting and participants
This study included Internal Medicine (IM) and Pediatric residents across 6 medical centers affiliated with the University of Toronto (UofT) in 2014–15. All third-year IM residents (N = 64) and first to third year Pediatric residents (N = 75) were eligible to participate. The resident inclusion criteria were based on how the individual program directors chose to implement the curriculum. The IM program targeted their third-year residents, whereas the pediatric program introduced the curriculum to first to third year residents during their academic half-days. There were no exclusion criteria. Prior to this study, there was no formal resource stewardship training. The UofT research ethics board approved this study.
Resource stewardship workshop
The 2.5-h interactive workshop focused on developing communication skills to counsel patients/caregivers about unnecessary testing, and was delivered to IM and Pediatric residents during their respective academic half-days (Table 1). Attendance at academic half-days is strongly encouraged but not mandatory, and some cannot attend due to vacation, illness or being post-call. We reviewed the literature and solicited input from the Choosing Wisely Canada® leadership team to inform our workshop. Specifically, we incorporated content relating to overcoming barriers to high-value care from the Alliance for Academic Internal Medicine and the American College of Physicians High Value Care curriculum , elements from the VALUE Framework that teaches residents to assess whether an intervention provides value , and a framework for communicating with patients /caregivers who are requesting unnecessary tests/treatments taken from the Choosing Wisely® Communication Modules developed in partnership with Drexel University College of Medicine . Members of the research team who were clinicians with an academic interest in promoting high-value care and medical educators with expertise in assessment tools were involved in development and delivery of the curriculum.
We applied the Kirkpatrick model to support the evaluation of educational outcomes . The knowledge assessment was a delayed knowledge test and the skills assessment was a structured communication skills assessment station integrated into the IM and Pediatric annual objective structured clinical examination (OSCE) (both Level 2 Kirkpatrick outcomes). Both assessments occurred approximately 3 months after the workshop. Program directors would not agree to random allocation of residents to being exposed or not to the curriculum. Therefore, to evaluate the impact of curriculum exposure on knowledge and skills, we compared residents who attended our workshop to those who did not.
Delayed knowledge test
We developed multiple-choice and short-answer questions to assess the knowledge taught during the workshop. We piloted questions with residents and faculty with resource stewardship expertise and iteratively modified questions for content and readability [see Additional file 1].
We generated an overall score based on their answers to the one short answer question and seven multiple-choice questions (total potential score of 8). For the short answer question, two study authors (GM and AW) independently scored the response and resolved discrepancies by consensus. We counted unanswered questions as incorrect.
Structured communication skills assessment
We developed hypothetical scenarios based on published recommendations from the Choosing Wisely® campaign and input from study authors who have prior experience creating OSCE stations. The IM scenario was a patient with a history of mild aortic stenosis, without any worrisome symptoms or signs, who is requesting an unnecessary repeat echocardiogram [see Additional file 2]. The Pediatric scenario was a parent of an infant who had a simple febrile seizure and a normal neurological examination, who is requesting an unnecessary magnetic resonance imaging (MRI) brain [see Additional file 3]. We piloted these scenarios with faculty and resident volunteers to ensure that they were realistic, readable, and feasible to complete in 10 min. At the time of the structured assessment, each resident read the clinical scenario and then counseled the standardized patient (SP) (playing either the patient or caregiver) about their request. The SP then used a rating scale to score the resident’s performance immediately after the encounter. The study team members were not involved in completing the rating scale to assess communication skills.
We created a novel rating scale that mapped specifically to the communication skills construct that we sought to assess. The scale includes the five components (i.e., clear recommendations, elicit patient/caregiver concerns, empathy, confirm agreement and general communication skills) of the structured communication framework that was taught as part of our workshop [see Additional file 4]. The SP rated performance on each of the 5 components and also provided a global rating using a five-point Likert scale. The SP also scored individual sub-items within each component using a three-point scale (1-not done at all; 2-attempted but either not complete or not effective; and 3-excellent).
We chose to use SPs as opposed to faculty raters because we sought to assess a resident’s ability to communicate about unnecessary tests from the perspective of the patient/caregiver. There is also high inter-rater reliability between ratings of trained SPs and trained faculty to assess other types of communication skills such as disclosing medical errors . To promote consistency of SP ratings, we used the same 3 SPs for all skills assessments. SP training occurred on two separate occasions for each scenario. During the first session, the SPs familiarized themselves with the rating scale and practiced using it by viewing the training videos used during the workshop. During the second session, they met with study investigators, and familiarized themselves with the scenario and practiced it in pilot interactions with a volunteer resident. This allowed SPs to clarify details regarding the scenario and items on the rating scale, and ensured that there was agreement between the ratings of the three SP raters. To establish inter-rater reliability of SP ratings for this tool, we audiotaped the assessments on the day of the OSCE and had a second trained SP independently rate resident communication skills.
We obtained informed consent from participating residents prior to their formative OSCE. The residents received no incentive to participate. In addition to the knowledge test and structured assessment data, we also collected baseline demographic characteristics from participating residents including gender and training level. We also asked residents about their prior experiences with communicating with patients/caregivers about unnecessary tests/treatments and to rate their own comfort and skills in this area, as well as whether they have ever received feedback on these skills.
We summarized participant characteristics descriptively, using means and standard deviations for continuous data and count and percentages for categorical data. We calculated a Cronbach’s alpha to determine the internal consistency of the scale items on the communication skills assessment.
We compared knowledge test scores and global ratings on the structured communication skills assessment for residents who attended the workshop versus those who did not attend using Wilcoxon rank sum test to better account for the non-normal distribution of scores. We also calculated intraclass correlation coefficients (ICC) to assess inter-rater reliability of the communication skills assessment ratings. To identify predictors of overall performance on the structured assessment, we incorporated five pre-specified variables of interest (gender, training level, workshop attendance, delayed knowledge assessment scores and self-reported prior feedback on resource stewardship communication skills) into a mixed-effects linear regression analysis. The model accounted for the correlation among observations from the same residency training program due to possible differences in case difficulty. The analyses were carried out using SAS version 9.3 (SAS Institute, Cary, NC, USA).
Of the 111 eligible IM and Pediatric residents, 98 (88%) participated in the structured assessment, of which 83 (75%) provided consent to have their assessment data used for analysis. Of the 83 study participants, 57 (69%) attended the resource stewardship workshop. Table 2 summarizes participant characteristics stratified by workshop attendance. There were no significant differences in participant characteristics comparing residents who attended the workshop to those who did not.
Over 97% of participants reported having prior experience communicating with patients/caregivers about unnecessary tests. Most self-rated their comfort and skill level with such communication scenarios as average or above average (Table 2), and 70% reported having received prior feedback on their communications skills with patients/caregivers about unnecessary tests.
Workshop attendance resulted in better performance on the knowledge test (4.3 ± 1.9 vs. 3.1 ± 1.7 out of 8, p = 0.01), but not better performance on the communication skills assessment (4.1 ± 0.8 vs. 4.0 ± 0.9 out of 5, p = 0.56). With regards to specific communication components, residents performed better on providing clear recommendations (4.1 out of 5) as compared to eliciting patient/caregiver concerns (3.7 out of 5) (p < 0.01) (Table 3).
The Cronbach’s alpha for the overall communication skills assessment was 0.88, indicating good to excellent internal consistency. ICC values comparing dual rater assessments on aggregate means of the subdomains and overall scores demonstrated moderate agreement, ranging from 0.384 to 0.574.
In the mixed-effects analysis, both higher training level (p = 0.01) and higher knowledge assessment scores (p = 0.046) were independent positive correlates of better global ratings on communication skills assessment, after adjusting for gender, training program, workshop attendance, and self-reported prior feedback on communication skills (Table 4).
We taught IM and Pediatric residents how to communicate with patients/caregivers who are seeking unnecessary tests/treatments as a way to enhance education on resource stewardship. Despite incorporating known approaches to promote learning of effective communication skills like role-play and observing videotaped patient and physician encounters,  our workshop only improved resident knowledge but did not significantly improve communication skills.
The most logical explanation for our findings is that a single workshop may not be sufficient to improve communication skills specific to resource stewardship. Additional reinforcement of the communication framework in the clinical learning environment is likely required to ensure delayed skills retention. In addition, it is possible that residents who tend to be effective communicators would perform well irrespective of workshop attendance.
Our study helps to advance education on resource stewardship in several important ways. Beyond providing an example of an educational workshop to teach effective communication when patients/caregivers request unnecessary tests, our findings also identified specific educational gaps that require closer attention. Residents in our study performed less well at eliciting patient/caregiver concerns compared to other communication domains. Prior research has demonstrated the importance of eliciting patient concerns to promote effective physician-patient interactions and decrease patient-related complaints [3, 33,34,35]. Educators should therefore emphasize the importance of eliciting patient/caregiver concerns when discussing unnecessary tests and/or treatments.
Second, with the recent focus on competency-based medical education [13, 14], we developed a tool to support assessments of communication skills about unnecessary tests and/or treatments that could be adopted by other programs. The design of our scale was methodical and attended to issues of validity in its construction . To represent our construct, we chose content items to measure using a communication framework developed by Choosing Wisely®, with additional input from experts, and subsequently checked response processes of our raters through rigorous training. With respect to internal structure, our scale demonstrated moderate inter-rater reliability and good internal consistency. Skills ratings also correlated with knowledge test performance, suggesting a relationship between these variables within the resource stewardship construct.
This study has limitations to note. Workshop attendance is strongly encouraged but not mandatory. Therefore, volunteer bias may have influenced the results, since residents who attended the workshop may have a higher level of intrinsic interest, knowledge or skills in resource stewardship than those who did not. Since we did not assess real interactions with patients, additional research is needed to determine whether skills demonstrated on the structured assessment translate into real-world practice. There may also be value in engaging patients to further refine the construct of the tool  and validating the knowledge and skills assessment tools in different settings.
An interactive workshop can improve knowledge of resource stewardship, but improving communication skills with patients/caregivers about unnecessary testing may require additional training or reinforcement in the clinical learning environment. These teaching and assessment approaches can support the integration of resource stewardship education into medical education.
Intraclass correlation coefficients
Magnetic resonance imaging
Objective structured clinical examination
University of Toronto
The Internal Medicine Milestone Project American Board of Internal Medicine 2015. Available from: http://www.acgme.org/portals/0/pdfs/milestones/internalmedicinemilestones.pdf.
Physician Competency Framework The Royal College of Physicians and Surgeons of Canada, 2015. Available from: http://www.royalcollege.ca/rcsite/documents/canmeds/canmeds-full-framework-e.pdf.
Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307(14):1513–6.
Crossing the Quality Chasm: A New Health System for the 21st Century. Institute of Medicine. 2001.
Welch HG, Schwartz LM, Woloshin S. Overdiagnosed- making people sick in the pursuit of health. Boston: Beacon Press; 2012.
McGregor MJ, Martin D. Testing 1, 2, 3: is overtesting undermining patient and system health? Can Fam Physician. 2012;58(11):1191–3. e615-7
Gibson RS, Janardan P. The treatment trap: how the overuse of medical care is wrecking your health and what you can do to prevent it. Chicago: Rowman & Littlefield Publishing Group; 2010.
Choosing Wisely Canada. Choosing wisely recommendations by specialty. Toronto: Choosing Wisely Canada; Available from: https://choosingwiselycanada.org/wp-content/uploads/2017/02/Choosing-Wisely-Canada-collection-of-lists.pdf. Accessed 21 Aug 2017.
Mehrotra A, Reid RO, Adams JL, Friedberg MW, McGlynn EA, Hussey PS. Physicians with the least experience have higher cost profiles than do physicians with the most experience. Health Aff. 2012;31(11):2453–63.
Patel MS, Reed DA, Loertscher L, McDonald FS, Arora VM. Teaching residents to provide cost-conscious care: a national survey of residency program directors. JAMA Intern Med. 2014;174(3):470–2.
Chen C, Petterson S, Phillips R, Bazemore A, Mullan F. Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for Medicare beneficiaries. JAMA. 2014;312(22):2385–93.
Sirovich BE, Lipner RS, Johnston M, Holmboe ES. The association between residency training and internists' ability to practice conservatively. JAMA Intern Med. 2014;174(10):1640–8.
Fraser AB, Stodel EJ, Chaput AJ. Curriculum reform for residency training: competence, change, and opportunities for leadership. Can J Anaesth. 2016;63(7):875–84.
Weinberger SE, Pereira AG, Iobst WF, Mechaber AJ, Bronze MS. Competency-based education and training in internal medicine. Ann Intern Med. 2010;153(11):751–6.
Hackbarth G, Boccuti C. Transforming graduate medical education to improve health care value. N Engl J Med. 2011;364(8):693–5.
Lurie P. DTC advertising harms patients and should be tightly regulated. J Am, Law, Med Ethics. 2009;37(3):444–50. 396
Mintzes B, Barer ML, Kravitz RL, Kazanjian A, Bassett K, Lexchin J, et al. Influence of direct to consumer pharmaceutical advertising and patients' requests on prescribing decisions: two site cross sectional survey. BMJ (Clinical research ed). 2002;324(7332):278–9.
Levinson W, Huynh T. Engaging physicians and patients in conversations about unnecessary tests and procedures: choosing wisely Canada. CMAJ : Canadian Medical Association Journal. 2014;186(5):325–6.
Smith CD. Teaching high-value, cost-conscious care to residents: the alliance for academic internal medicine-American College of Physicians Curriculum. Ann Intern Med. 2012;157(4):284–6.
Dine CJ, Miller J, Fuld A, Bellini LM, Iwashyna TJ. Educating physicians-in-training about resource utilization and their own outcomes of Care in the Inpatient Setting. Journal of Graduate Medical Education. 2010;2(2):175–80.
Legare F, Labrecque M, Cauchon M, Castel J, Turcotte S, Grimshaw J. Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ : Canadian Medical Association Journal. 2012;184(13):E726–34.
Stammen LA, Stalmeijer RE, Paternotte E, Oudkerk Pool A, Driessen EW, Scheele F, et al. Training physicians to provide high-value, Cost-Conscious Care. A Systematic Review JAMA. 2015;314(22):2384–400.
Moriates C, Soni K, Ranji S. The value in the evidence: teaching residents to “choose wisely”. JAMA Intern Med. 2013;173(4):308–10.
Post J, Reed D, Halvorsen AJ, Huddleston J, McDonald F. Teaching high-value, cost-conscious care: improving residents’ knowledge and attitudes. Am J Med. 2013;126(9):838–42.
Smith CD. Teaching high-value, cost-conscious care to residents: the alliance for academic internal medicine–American College of Physicians Curriculum. Ann Intern Med. 2012;157(4):284–6.
High Value Care: Curriculum for Educators, Residents, and Students. Available from: http://hvc.acponline.org/curriculum.html.
Patel MS, Davis MM, Lypson ML. The VALUE framework: training residents to provide value-based care for their patients. J Gen Intern Med. 2012;27(9):1210–4.
ABIM Foundation. Physician communication modules. Available from: http://www.choosingwisely.org/resources/modules/.
ABIM Foundation. Patient with Back Pain who requests an MRI. Available from: https://www.youtube.com/watch?v=cJLuxDbBs1w.
Kirkpatrick D. Training and development handbook. New York: McGraw-Hill; 1967.
Stroud L, McIlroy J, Levinson W. Skills of internal medicine residents in disclosing medical errors: a study using standardized patients. Academic Medicine : Journal of the Association of American Medical Colleges. 2009;84(12):1803–8.
Levinson W, Lesser CS, Epstein RM. Developing physician communication skills for patient-centered care. Health Aff. 2010;29(7):1310–8.
Halpert A, Godena E. Irritable bowel syndrome patients' perspectives on their relationships with healthcare providers. Scand J Gastroenterol. 2011;46(7–8):823–30.
Shields CG, Coker CJ, Poulsen SS, Doyle JM, Fiscella K, Epstein RM, et al. Patient-centered communication and prognosis discussions with cancer patients. Patient Educ Couns. 2009;77(3):437–42.
Teutsch C. Patient-doctor communication. The Medical Clinics of North America. 2003;87(5):1115–45.
Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119(2):166. e7-16
Hodges BD, McNaughton N. Who should be an OSCE examiner? Acad Psychiatry. 2009;33(4):282–4.
The authors would like to thank Alex Kiss for his support on statistical analysis and Elisa Hollenberg for support with study recruitment and data entry.
This study was supported by an Educational Development Funding grant from the University of Toronto.
Availability of data and materials
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
Ethics approval was granted by the University of Toronto Research Ethics Board (Protocol Reference: # 30049). Informed consent was obtained from all participants in this study.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Post-OSCE Resource Stewardship Knowledge Test with Correct Answers Bolded. Copy of post-OSCE resource stewardship knowledge test with correct answers bolded. (PDF 165 kb)
Resource stewardship OSCE communication scenario for Internal Medicine residents. Copy of the resource stewardship OSCE communication scenario for Internal medicine residents. (PDF 217 kb)
Resource stewardship OSCE communication scenario for Pediatric residents. Copy of the resource stewardship OSCE communication scenario for Pediatric residents. (PDF 260 kb)
Rating scale used to assess resident communication skills. Copy of rating scale used to assess resident communication skills. (PDF 200 kb)