Evaluation of a communication skills seminar for students in a Japanese medical school: a non-randomized controlled study
© Mukohara et al; licensee BioMed Central Ltd. 2004
Received: 13 May 2004
Accepted: 18 November 2004
Published: 18 November 2004
Little data exist for the effectiveness of communication skills teaching for medical students in non-English speaking countries. We conducted a non-randomized controlled study to examine if a short intensive seminar for Japanese medical students had any impact on communication skills with patients.
Throughout the academic year 2001–2002, a total of 105 fifth-year students (18 groups of 5 to 7 students) participated, one group at a time, in a two-day, small group seminar on medical interviewing. Half way through the year, a five-station objective structured clinical examination (OSCE) was conducted for all fifth-year students. We videotaped all the students' interaction with a standardized patient in one OSCE station that was focused on communication skills. Two independent observers rated the videotapes of 50 students who had attended the seminar and 47 who had not. Sixteen core communication skills were measured. Disagreements between raters were resolved by a third observer's rating.
There was a statistically significant difference in proportions of students who were judged as 'acceptable' in one particular skill related to understanding patient's perspectives: asking how the illness or problems affected the patient's life, (53% in the experimental group and 30% in the control group, p = .02). No differences were observed in the other 15 core communication skills, although there was a trend for improvement in the skill for asking the patient's ideas about the illness or problems (60% vs. 40%, p = .054) and one of the relationship building skills; being attentive and empathic nonverbally (87% vs. 72%, p = .064).
The results of this study suggest that a short, intensive small group seminar for Japanese medical students may have had a short-term impact on specific communication skills, pertaining to understanding patient's perspectives.
The literature from English-speaking countries indicates that teaching communication skills is effective in improving learners' communication skills with patients . However, the evidence from non-English speaking countries is sparse . In addition, the conceptual frameworks for communication skills teaching are based on research evidence from English-speaking countries . There is an ongoing debate about whether the principles and methods for teaching communication skills developed in English-speaking countries could be applied to other places with different languages and cultures [2–4].
Teaching communication skills is gaining popularity and proliferating for Japanese health professional students . Yoshida et al. conducted a controlled study to examine the effects of such training with 16 Japanese dental students and had a positive result . A few reports have been published on Japanese medical students [7–9]. However, to our best knowledge, no controlled studies for communication skills teaching have been conducted for that population.
In many traditional medical schools in Japan, communication skills teaching is limited in time and scope, and isolated from other formal curricula. Thus it is important to know whether such type of training make a difference, at least in the short run. This should also be of interest to educators elsewhere who similarly work in settings where there is not enough formal curricular time for communication skills teaching.
The objective of this study was to evaluate the impact of a short, intensive small group seminar, which was based on Western educational principles, on Japanese medical students' communication skills with patients.
Medical schools in Japan last six years with the last two years consisting of clerkships. Before the fifth-year, Japanese students typically have few direct interactions with patients. Throughout the academic year 2001–2002, a total of 105 fifth-year students from the Nagoya University School of Medicine rotated through the various clinical services of the Nagoya University Hospital. Students divided themselves into 18 groups of 5 to 7, but the sequential order of rotations is set by the medical school officials.
In September 2001, half way through the academic year, a five-station objective structured clinical examination (OSCE) was conducted for all fifth-year students. The primary purpose of the OSCE was to provide trainees with the opportunity to receive feedback on their clinical skills from the faculty in a safe and structured environment. One OSCE station focused on the medical interview. Students engaged in a 5-minute interaction with a standardized patient presenting with cough. A total of 10 fourth-year students were trained to serve as standardized patients in a series of 3 small group sessions, each lasting 60 minutes . During the interview, the fifth-year students were observed by faculty and evaluated for both station-specific and general communication skills on the pre-defined rating scale. The faculty gave students a 3-minute feedback immediately after the encounter. Standardized patients did not give feedback. All interactions were videotaped and subsequently reviewed by faculty members to provide students with additional written feedback.
Placed at the mid point of the academic year, the OSCE provided us with the opportunity to evaluate the short-term effectiveness of the small group seminar on students' communication skills. We reviewed the videotapes of 52 students who had attended the seminar (the experimental group) and 53 students who had not at the time of the OSCE (the control group). The group assignment was based on the sequential order of clinical rotations, arbitrarily set by the medical school officials. The time intervals between the seminar and the OSCE ranged from 1 week to 5 months. Students were asked to provide informed consent using a form that had been approved by the Institutional Review Board at the Nagoya University Hospital.
The interview rating form was created by one of the authors (KM) and includes 16 essential communication skills items. They are grouped into 6 communication tasks that should be accomplished during the initial 5 minutes of an encounter (establish initial rapport, survey patient's reason(s) for the visit, determine the patient's chief concern, elicit and understand the patient's perspective, manage flow – provide the structure for the interview, and use of relationship building skills). The performance was rated on a 4-point scale labelled as good, satisfactory, insufficient and poor. The skills items were selected for their association with improved patient outcomes. They were derived from evidence-based communication assessment tools (i.e., the Calgary-Cambridge observation guide, the SEGUE framework, and the checklist developed by the investigators of the Macy Initiative in Health Communication) [2, 10, 11]. These instruments are based on the same conceptual models for patient-physician communication we referred to during our teaching seminar.
Two staff members (KK, HW) were trained to serve as raters. The tapes were independently reviewed and scored using the students' communication skills rating scale. Ten arbitrarily selected videotapes of students' role-plays of a doctor-patient encounter during the small group seminar were used to ensure accuracy and inter-rater reliability. At the time of the research the raters primarily worked outside the University and did not participate in the teaching seminars. Thus they were blinded to the students' group assignments.
From the 105 students who attended the OSCE, 2 did not return the consent form, 5 did not give permission for the video review, for 1 the videotape quality was too poor to be analyzed. Thus, a total of 97 videotapes were available for the analysis.
A skill item was considered 'acceptable' if both raters scored the students' performance as 'good' or 'satisfactory.' It was labeled as 'unacceptable' if both raters scored the performance as 'insufficient' or 'poor.' When these two raters disagreed over the judgment about the students' performance (e.g., one rater scored the performance of a skill item as 'acceptable' and the other scored the performance of the same item as 'unacceptable'), a communication educator and researcher (KA) served as the tiebreaker. The overall disagreement rate between the two raters (KK, HW) was 21%. The raters disagreed more often on inviting the patient to tell the story chronologically (41%), actively responding to the patient's concerns and nonverbal cues (34%), and being attentive and empathic nonverbally (31%).
We compared baseline characteristics of the two groups using t-tests for a continuous variable (age) and chi-square tests for categorical variables. To evaluate the effect of the educational intervention, the proportion of students with 'acceptable' performance was compared with those whose performance was unacceptable using chi-square tests for all 16 skills items. All statistical analyses were done by JS using Stat View Version 5.0 (SAS Institute Inc. North Carolina).
Baseline characteristics of the students
Intervention Group (N = 47)
Control Group (N = 50)
Mean age (SD)
36% (N = 17)
40% (N = 20)
Did a self-study preparing for OSCE
34% (N = 16)
56% (N = 28)
Took an elective on communication in medicine at the 4th year
43% (N = 20)
54% (N = 27)
Interested in becoming a generalist
13% (N = 6)
26% (N = 13)
Student performance of the skill judged as 'acceptable'
Communication Tasks and Related Skills
Intervention Group (N = 47)
Control Group (N = 50)
Establish Initial Rapport
Greet patient and obtain patient's name
Introduce self and clarify the role
Survey Patient's Reason(s) for the Visit
Allow the patient to complete his/her opening statement
Invite the patient to tell the story chronologically
Actively listen, using verbal and nonverbal techniques
Summarize. Check for understanding. Invite more questions?
Determine the Patient's Chief Concern
Ask closed-questions that are non-leading, one at a time
Define the concern completely
Elicit and Understand the Patient's Perspective
Explore contextual factors (e.g., job, family, hobbies)
Ask the patient's ideas about the illness or problems
Explore how the problem affects the patient's life
Manage Flow – Provide the Structure to the Interview
Summarize periodically throughout the interview
Use of Relationship Building Skills
Be attentive and empathic nonverbally
Actively respond to patient's concerns and nonverbal cues
Use appropriate language
A short, intensive small group seminar on medical interviewing appeared to have had an impact on some specific skills, pertaining to "eliciting and understanding the patient's perspectives." It did not seemed to have improved the skills associated with the other tasks: establishing initial rapport, surveying the patient's reason(s) for the visit, determining the patient's chief concern, and managing flow – providing the structure for the interview, and the skills for building relationships.
There are several strengths of our study. First, this is one of the few empirical, controlled studies from a non-English speaking country. Even though the students were not strictly randomized into intervention and control groups, the assignment occurred arbitrarily by the administration, without regard to students' preferences or interests in medical interviewing. Thus, it is unlikely that the higher scores in the intervention group are attributable to self-selection. Although there was a significant difference between the groups in proportions of students who did a self-study for the OSCE, which might have caused the results of no difference in most of the skills, the other characteristics such as age and gender were similarly distributed (Table 1). Second, interventions and evaluations were guided by the conceptual framework, modelled after the 3 widely used theoretical models that are based on rigorous, empirical research in the field of patient-physician communication [2, 10, 11]. Third, the communication skills evaluation instrument was matched with the competencies taught in the small group sessions . By carefully delineating and defining specific communication skills that should be addressed in the teaching session and by evaluating the effect of the teaching intervention on these individual skills, we sought to examine whether some skills were more teachable than others in such a brief, small group sessions.
Our study also has weaknesses that should be addressed. First, our teaching method was based on the research findings in Western world, and this is based on the untested assumption that these findings are equally valid in Japan. There is evidence that patient-physician communication patterns in Japan are different from those in the West. Previous research by Ohtaki and colleagues compared patient-physician communication patterns in Japan and the USA . It included 20 outpatient consultations of four physicians in Japan and 20 outpatient consultations of five physicians in the USA. Japanese physicians spent less time on social talk than the USA counterparts (5% vs. 12%). Japanese patient-physician encounters included more pauses than those in the USA (30% vs. 8.2% of the total consultation length). There is a need for more empirical studies linking physicians' communication skills to patient outcomes specifically for Japanese population. Second, our assessment of students' communication skills was based on observations of a single, five-minute OSCE station. The reliability of which as a measure of communication skills is known to be low . Third, because we assessed the students' skills at only one time, we could not assess the change in students' performance before and after the intervention. Fourth, the use of junior students as standardized patients may have influenced the performance of the examinees. The accuracy of student-standardized-patients' (student-SPs') portrayal would be a critical issue especially when the OSCE is used to grade students. Although we did not objectively investigate the consistencies of the portrayal by student-SPs, our examinees rated highly the fidelity of student-SPs, i.e., the degree to which they were acting as if they were real patients (mean score, 3.9 on a 5-point Likert scale) . Fifth, our study might have only shown that the intervention was effective in improving students' skills for eliciting 'expert' observations of patient perspectives, not actual patient perspectives. We did not ask student-SPs whether examinees elicited their perspectives. Rather, we judged examinees' ability to elicit patient perspectives through their 'observable' behaviours from the experts' point of view. The role of student-SPs in evaluating fellow students' communication skills, particularly skills for eliciting patient perspectives should be addressed in future studies. Finally, the statistically significant difference observed for only 1 skill among a total of 16 skills could be due to chance alone. It is certainly possible that our intervention was too weak to influence any of the 16 communication skills.
One can hypothesize the reasons why the intervention appeared to make a difference to some communication skills competencies but not to others. One could speculate that the competencies that were not influenced by the intervention were either very easy in general or too difficult to acquire in such a short teaching session. For example, the skills for establishing initial rapport (greet patient and obtain the patient's name, introduce self and clarify roles) and skills for determining the patient's chief concern (ask closed-ended questions that are non-leading and one at a time, define the concern completely) may be already present from the outset or so easy to acquire that a self-study just before the OSCE would make no differences in scores between the groups regardless of the intervention. On the other hand, the skills for surveying the patient's reason(s) for the visit, which requires being open at the beginning of the interview, may be too difficult for students to demonstrate, with or without the intervention. In particular, only 9% in the intervention group and 6% in the control group demonstrated an acceptable performance for the skills of allowing patients to complete their opening statements. These very low scores may also indicate that during small group sessions, we did not emphasize enough the importance of not interrupting patients at the beginning of the interview. Another explanation is that 'content' skills (i.e., what we communicate) are easier for students to acquire than 'process' skills (i.e., how we communicate). Kurtz at al. noted that the skills for understanding patient's perspectives, which our intervention made a difference, are actually 'content' skills, not 'process' skills . One could argue that the intervention was just too short to influence other 'process' skills. These interesting hypotheses should require further investigations.
The results of this study suggest that a short, intensive small group seminar for Japanese medical students may have had an impact on specific communication skills, namely, skills for exploring how the illness or problem affected the patient's life, asking the patient about ideas concerning the illness or problem, and being attentive and empathic nonverbally at least in the short term. Further studies should be done to confirm this preliminary finding and to clarify the skills for which educational interventions could make a difference.
We thank students who participated in the study. We thank Elizabeth Kachur, Ph.D. in New York for useful comments on earlier drafts.
- Aspegren K: BEME Guide No. 2: Teaching and learning communication skills in medicine – a review with quality grading of articles. Med Teach. 1999, 21: 563-570. 10.1080/01421599978979.View ArticleGoogle Scholar
- Silverman J, Kurtz S, Draper J: Skills for communicating with patients. 1998, Radcliffe Medical PressGoogle Scholar
- Nestel D: Evaluation of a communication skills course: cultural relevance of the patient-centered interview in a Hong Kong Chinese setting. Med Teach. 2001, 23: 212-214. 10.1080/01421590020031147.View ArticleGoogle Scholar
- Nestel D, Betson C: An evaluation of a communication skills workshop for dentists: cultural and clinical relevance of the patient-centered interview. Br Dent J. 1999, 187: 385-8. 10.1038/sj.bdj.4800286a.Google Scholar
- Tsuda T: The evaluation of education for communication skills. In: A manual for clinical education. Edited by: Japan Society of Medical Education. 1994, Tokyo: Shinohara Publications Inc, 214-220. (in Japanese)Google Scholar
- Yoshida T, Itadani C, Tsubouchi J, Matsumura S, Miyaji J, Okamoto Y, Shimono T: Effects of training with simulated patient for dental school students in clinical interviewing. Med Educ (Japan). 2001, 32: 153-158.Google Scholar
- Saito S, Shimizu Y, Kita K, Osawa K, Watanabe A: Evaluations of a program for teaching basic clinical skills to medical students. Med Educ (Japan). 2001, 32: 451-457.Google Scholar
- Isobe M, Sakamoto A, Inubushi M, Komiya R, Takayama K, Takei H: Practice in medical interviewing during clinical clerkship. Med Educ (Japan). 2001, 32: 47-52.Google Scholar
- Inoue S, Fujita H, Takahashi M, Kakeda K, Yamauchi Y, Kataoka K, Fukuzawa K: Trial of videotape feedback to train fifth-year undergraduate in medical interviewing. Med Educ (Japan). 2003, 34: 21-28.Google Scholar
- Makoul G: The SEGUE Framework for teaching and assessing communication skills. Pat Edu Counsel. 2001, 45: 23-34. 10.1016/S0738-3991(01)00136-7.View ArticleGoogle Scholar
- Yedidia MJ, Gillespie CC, Kachur E, Schwartz MD, Ockene J, Chepaitis AE, Snyder CW, Lazare A, Lipkin M: Effect of communications training on medical student performance. JAMA. 2003, 290: 1157-65. 10.1001/jama.290.9.1157.View ArticleGoogle Scholar
- Kitagawa M, Ban N, Shimada Y: A trial of medical students playing standardized patients during an Objective Structured Clinical Examination. Med Educ (Japan). 2000, 31: 247-254.Google Scholar
- Cegala DJ, Broz SL: Physician communication skills training: a review of theoretical backgrounds, objectives, and skills. Med Educ. 2002, 36: 1004-1016. 10.1046/j.1365-2923.2002.01331.x.View ArticleGoogle Scholar
- Ohtaki S, Ohtaki T, Fetters MD: Doctor-patient communication: a comparison of the USA and Japan. Fam Pract. 2003, 20: 276-82. 10.1093/fampra/cmg308.View ArticleGoogle Scholar
- van Thiel J, Kraan HF, Van Der Vleuten CP: Reliability and feasibility of measuring medical interviewing skills: the revised Maastricht History-Taking and Advice Checklist. Med Educ. 1991, 25: 224-9.View ArticleGoogle Scholar
- Kurtz S, Silverman J, Benson J, Draper J: Marrying content and process in clinical method teaching: enhancing the Calgary-Cambridge guides. Acad Med. 2003, 78: 802-9.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/4/24/prepub