- Research article
- Open Access
Undergraduate educational environment, perceived preparedness for postgraduate clinical training, and pass rate on the National Medical Licensure Examination in Japan
BMC Medical Education volume 10, Article number: 35 (2010)
We investigated the views of newly graduating physicians on their preparedness for postgraduate clinical training, and evaluated the relationship of preparedness with the educational environment and the pass rate on the National Medical Licensure Examination (NMLE).
Data were obtained from 2429 PGY-1 physicians-in-training (response rate, 36%) using a mailed cross-sectional survey. The Dundee Ready Education Environment Measure (DREEM) inventory was used to assess the learning environment at 80 Japanese medical schools. Preparedness was assessed based on 6 clinical areas related to the Association of American Medical Colleges Graduation Questionnaire.
Only 17% of the physicians-in-training felt prepared in the area of general clinical skills, 29% in basic knowledge of diagnosis and management of common conditions, 48% in communication skills, 19% in skills associated with evidence-based medicine, 54% in professionalism, and 37% in basic skills required for a physical examination. There were substantial differences among the medical schools in the perceived preparedness of their graduates. Significant positive correlations were found between preparedness for all clinical areas and a better educational environment (all p < 0.01), but there were no significant associations between the pass rate on the NMLE and perceived preparedness for any clinical area, as well as pass rate and educational environment (all p > 0.05).
Different educational environments among universities may be partly responsible for the differences in perceived preparedness of medical students for postgraduate clinical training. This study also highlights the poor correlation between self-assessed preparedness for practice and the NMLE.
One of the principal roles of medical schools is to equip newly graduating physicians for postgraduate clinical training [1–3]. Lack of preparedness for this training has been linked to greater stress in junior physicians and may also lead to poorer quality of patient care . To improve clinical skills that can increase readiness for postgraduate clinical training, undergraduate medical education in Japan has recently begun implementing the Model Core Curriculum. This is proposed to restructure all areas of undergraduate medical education by requiring the implementation of the core curriculum for 70% of an entire curriculum and unique curriculum for remaining 30% in each medical school in order to facilitate learning by active participation in clinical clerkships . The core curriculum emphasizes clinical practice through use of a problem-based learning (PBL), organ-based integrated curriculum (rather than a department-based approach), and implementation of the objective structured clinical examination (OSCE) .
In UK studies, self-perceived preparedness for postgraduate clinical training was found to differ substantially among medical schools [1, 2]. Crucial factors influencing these differences may include a good educational environment for medical students, curricular changes with greater attention to preparedness for practice, adherence to modern educational theory, integration of basic and clinical sciences, and appropriate selection of students who will be successful medical graduates . However, the extent, to which newly graduating physicians in Japan are prepared for postgraduate clinical training and differences among medical schools in terms of preparedness, has not been evaluated.
Preparedness for postgraduate clinical training of medical students is likely to depend on the engagement of medical students for learning clinical medicine. Educational environment is now considered as one of the most important factors determining engagement of medical students [8, 9]. The educational environment is also considered to be the most significant manifestation and conceptualization of a curriculum, since it embraces everything that happens in a medical school . Factors such as self-perceptions of learning, self-perceptions of teachers, academic self-perceptions, self-perceptions of atmosphere, and social self-perceptions are currently considered as major domains that comprise the educational environment of medical schools . For instance, competitive, authoritarian, stressful, or intimidating environments may de-motivate students and weaken their engagement for learning clinical medicine. Environments that are collaborative, collegial, and supportive may enhance greater engagement of medical students for learning clinical medicine and the greater engagement may lead to improved preparedness for postgraduate clinical training. Thus, it is important to investigate whether a better environment is likely to help graduating physicians to be better prepared for postgraduate clinical training.
Preparedness for postgraduate clinical training could be assessed by different measures and self-perceived preparedness may be one for conducting nationwide studies like the previous UK report . Thus, in the current study, we first sought the views of newly graduating physicians on their preparedness for postgraduate clinical training to obtain an understanding of this preparedness at different medical schools. Next, we evaluated the relationship between the educational environment and students' self-perceived preparedness at all 80 medical schools in Japan. We also evaluated the relationship between the pass rate on the National Medical Licensure Examination (NMLE) and self-perceived preparedness for postgraduate clinical training.
In November 2008, a cross-sectional survey was mailed to 6725 1st-year resident physicians at 427 teaching hospitals with five or more 1st-year resident physicians, including 80 university hospitals and 347 non-university teaching hospitals throughout Japan. The program director at each hospital was asked to encourage the residents to complete the self-administered questionnaire. The academic calendar in Japan starts on April 1 and ends on March 31 of the following year; thus, the survey was conducted at the midpoint (November) of the 2008 academic year. We obtained ethical approval for the study from the ethics committee of St. Luke's International Hospital (the institution that the principal investigator was affiliated with at the time of the study) in Tokyo, Japan.
Data collected in the survey included the demographics of the respondents and their responses to the Dundee Ready Education Environment Measure (DREEM) inventory, which were used to assess the learning environment of the medical schools from which the respondents had graduated . The earlier Japanese version of the DREEM was developed by Nishigori et al [12, 13]. The original British version of the DREEM is in the public domain and was translated forward and backward by native-English speakers and native Japanese speakers, respectively, for development of a version for this study, the item translation of which was similar to that of the earlier version [12, 13]. The DREEM inventory consists of 50 items, each of which is scored from 1 to 4 on a Likert scale, giving a maximum score of 200. A score of 100 or lower indicates a poor environment for learning medicine. The five DREEM domains include students' perception of learning, perception of teachers, academic self-perceptions, perception of atmosphere, and social self-perceptions. The DREEM has been validated as a measure of learning environment effectiveness in several previous studies [8, 14].
The survey also included six questions related to preparedness for clinical competencies taken from the Association of American Medical Colleges Graduation Questionnaire (AAMC-GQ) after forward and backward translation into Japanese, for which we received permission from the AAMC. This questionnaire has been validated in several studies [15–17]. The questions represented a broad range of competencies for the following six clinical areas: 1) general clinical skills, 2) basic knowledge of diagnosis and management of common conditions, 3) communication skills, 4) skills for applying evidence-based medicine to clinical care, 5) professionalism, and 6) basic skills for physical examination. Participants rated their confidence in each of these areas using a five-point Likert scale: 1 (strongly agree), 2 (agree), 3 (neither agree nor disagree), 4 (disagree), and 5 (strongly disagree).
Participant responses were grouped by university that they graduated from. Spearman correlation coefficients and linear regression were used for analysis of relationships between the mean DREEM score for each medical school and the proportion of respondents from that school with confidence in their preparedness for each of the six clinical skills (as indicated by an answer of "strongly agree" or "agree"). Similar analyses were performed for the relationships between the pass rates on the 2008 NMLE (data were obtained from the Ministry of Health, Welfare and Labor) and the proportion of respondents with confidence in their preparedness. Individual performance scores were not available for analysis. Statistical analyses were conducted using SPSS 15.0J (Tokyo, Japan), with a two-tailed value of p < 0.05 considered to be statistically significant.
A questionnaire was sent to 6725 1st-year residents in 427 teaching hospitals in November 2008. A total of 2429 PGY-1 physicians-in-training (927 women, 38%) completed the questionnaire (response rate, 36%). Among the 80 medical schools from which the participants in the survey had graduated, the mean DREEM scores ranged from 95 to 137 (pooled mean 112; median 111; standard deviation 7.2). Three medical schools had a mean DREEM score of less than 100. The highest mean DREEM score was 137 (Table 1). The five subscales of the DREEM were highly inter-correlated (Table 2).
Of the 2429 respondents to the survey, 406 (17%) agreed or strongly agreed that they were confident that they had acquired the general clinical skills required to begin postgraduate training. Overall, confidence in preparedness for postgraduate training in other clinical areas was indicated by 686 respondents (29%) for basic knowledge of diagnosis and management of common conditions, 1173 (48%) for communication skills, 465 (19%) for skills for applying evidence-based medicine to clinical care, 1317 (54%) for professionalism, and 904 (37%) for basic skills for physical examination.
Based on data for each school, the proportions of participants in the survey who were confident in their preparedness for postgraduate training ranged from 0% to 43% for general clinical skills; from 3% to 56% for basic knowledge of diagnosis and management of common conditions; from 19% to 77% for communication skills; from 3% to 42% for skills for applying evidence-based medicine to clinical care; from 27% to 83% for professionalism; and from 9% to 70% for basic skills for physical examination (Table 1).
A Spearman correlation analysis of the relationships between mean DREEM scores for medical schools and the proportions of participants with confidence in preparedness indicated significant relationships (all p < 0.01) in all six clinical areas (Figures 1, 2, 3, 4, 5 and 6), with positive coefficients of 0.425 for general clinical skills (R2 = 0.218), 0.377 for basic knowledge of diagnosis and management of common conditions (R2 = 0.160), 0.452 for communication skills (R2 = 0.169), 0.402 for skills for applying evidence-based medicine to clinical care (R2 = 0.236), 0.522 for professionalism, R2 = 0.294), and 0.568 for basic skills for physical examination (R2 = 0.393). A Spearman correlation analysis of the relationships between each of all five DREEM subscales for medical schools and the proportions of participants with confidence in preparedness also indicated significant relationships (all p < 0.05) in all six clinical areas.
The pass rates for the 2008 NMLE ranged from 85% to 99%. A Spearman correlation analysis of the relationships between these pass rates and the proportion of participants with confidence in their preparedness indicated no significant relationship in any of the six clinical areas, with coefficients of 0.035 for general clinical skills (p = 0.758), 0.039 for basic knowledge of diagnosis and management of common conditions (p = 0.733), 0.123 for communication skills (p = 0.277), -0.077 for skills for applying evidence-based medicine to clinical care (p = 0.496), 0.078 for professionalism (p = 0.492), and -0.012 for basic skills for physical examination (p = 0.917). The mean DREEM scores and pass rates for the universities also showed no significant relationship (Spearman correlation coefficient = 0.209, p = 0.063).
The current study showed that only 17% of our responding newly graduated physicians from all medical schools in Japan felt prepared by their schools in general clinical skills for their first postgraduate clinical training. Perceived preparedness for other clinical areas was similarly unsatisfactory. These results are in line with those of our recent pilot report indicating that Japanese medical graduates perceive that they are not well prepared clinically to start working as physicians-in-training, and that self-reported preparedness is significantly lower in Japanese graduates than in their US counterparts . However, because of a limited ability to accurately self-assess among professionals , external assessment is needed for evaluating a real difference in preparedness between Japan and US.
The Model Core Curriculum has been introduced in undergraduate medical education throughout Japan. But medical schools in Japan vary greatly in the extent to which they have adopted the Model Core Curriculum, and there are still considerable differences in the duration and contents of clinical clerkships among medical schools [20, 21]. In many schools, students have limited opportunities to examine and interact with patients and little direct responsibility for patient care . In this study, we found substantial differences among Japanese medical schools in self-perceived preparedness of their graduates, with significant positive correlations between self-assessed preparedness for all clinical areas and a better educational environment. These findings suggest that differences in educational environments among medical schools may be partly responsible for the differences in perceived preparedness of graduates for postgraduate clinical training. The implications of this finding and the need for improvement of the educational environment for medical students should be of concern to medical educators.
Engagement is a crucial step in learning that depends not only on the motivation and learning style of students, but also on the environment or "climate" in which the learning is taking place [23, 24]. Genn and Harden suggested that the educational environment is the soul and spirit of the medical curriculum and that establishing an effective environment is the most important single task of medical educators [10, 25]. Components of the educational environment related to the curriculum include the style (e.g., PBL vs. a traditional approach) and quality of teaching, signposting and clarity of the process, outcomes, assessment, and support mechanisms for students. Components of the educational environment related to individual teachers include teaching style, enthusiasm, physical environment, and role modeling . Thus, our finding of significant positive correlations between perceived preparedness for all clinical areas and a better educational environment in medical schools reflects the importance of this environment in affecting the extent of engagement of medical students. Further research is needed to determine the components in an educational environment that are most strongly related to preparedness.
We found no significant association between the pass rate on the NMLE and perceived preparedness for any of the six clinical areas addressed in our survey. Traditionally in Japan, medical students have been required to attend lectures that were aimed at equipping the students to pass the NMLE (a paper-only test with a major emphasis on cognitive domains), even in the last two years of clinical training in a six-year medical school program. Some schools still emphasize this traditional curriculum, with little emphasis on the new Model Core Curriculum [20, 21]. However, the result of our study showing lack of association between the pass rate on the NMLE and perceived preparedness may indicate poor predictive validity of students' perceived preparedness for their first postgraduate clinical training against the NLME outcomes. This may also reflect relative insensitivity for identifying any association between the NMLE and perceived preparedness when using data about the pass rate. Thus, this result should be cautiously interpreted and there would be a need for further investigations such as a study using actual scores of the exam rather than pas-fail rates.
If our results would be confirmed using actual scores of the NMLE, a change of the contents and assessment method of the NMLE, such as introduction of clinical skills assessment or an OSCE-type test, could be considered for improving preparedness by helping schools to focus their training on this aspect. Another possibility for a change of the exam would be to administer the cognitive domain tests of the exam prior to entry into the clinical clerkship period in the last two years of medical school, similar to the United States Medical Licensing Exam (USMLE) Step 1.
There are several limitations in this study. First, our results might have been influenced by sampling bias, since the response rate was relatively low and many residents may have been too busy to respond to the survey. As we have conducted a cross-sectional survey for 1st-year resident physicians at teaching hospitals with five or more 1st-year resident physicians, we did not obtained data from teaching hospitals with four or less 1st-year resident physicians. Thus, we did not know the range of response rates across schools and gender ratio in all PGY-1 population. However, our sample size was large and we were able to examine a nationwide sample. Although Japanese Ministry of Health, Welfare and Labor does not open data related to individual residents, based on data about 46,800 medical students in 2006, 33% were women . So our sample (38% were women) seemed not much different from the total population.
Second, our survey only investigated each resident's perception of their preparedness, rather than using clinical or objective assessment of preparedness. Thus, we cannot determine the differences in outcomes between universities objectively. Finally, because of the cross-sectional observational nature of the survey, we are unable to determine a causal link between the educational environment and preparedness, since confounding factors such as personal characteristics of students that may be related to both environment and preparedness may have influenced the results.
In summary, Japanese medical graduates perceive that they are not well prepared clinically to start working as physicians-in-training. There were substantial differences among Japanese universities in the perceived preparedness of their graduates. Significant positive correlations were found between perceived preparedness for all clinical areas and a better educational environment, but there was no significant association between the pass rate on the National Medical Licensure Examination and perceived preparedness for any clinical areas. The link between self-perceived and independently observed preparedness needs to be established. Then, further research will be required to evaluate the most important components in the educational environment that influence preparedness. Japanese medical schools may need to consider establishment of minimum requirements for clinical competencies for their medical graduates.
Goldacre MJ, Lambert T, Evans J, Turner G: Preregistration house officers' views on whether their experience at medical school prepared them well for their jobs: national questionnaire survey. BMJ. 2003, 326 (7397): 1011-1012. 10.1136/bmj.326.7397.1011.
Illing J, Morrow G, Kergon C, Burford B, Spencer J, Peile E, Davies C, Baldauf B, Allen M, Johnson N: How prepared are medical graduates to begin practice? A comparison of three diverse UK medical schools. Final summary and conclusions for the GMC Education Committee. GMC Annual Report; London. 2008
GMC: Tomorrow's Doctors. 2009, [http://www.gmc-uk.org/education/undergraduate/tomorrows_doctors_2009.asp]
Paice E, Rutter H, Wetherell M, Winder B, McManus IC: Stressful incidents, stress and coping strategies in the pre-registration house officer year. Med Educ. 2002, 36 (1): 56-65. 10.1046/j.1365-2923.2002.01101.x.
Abe Y: The Model Core Curriculum. Igaku-Kyoiku (Japanese). 2002, 33: 77-82.
Teo A: The current state of medical education in Japan: a system under reform. Medical Education. 2007, 41 (3): 302-308. 10.1111/j.1365-2929.2007.02691.x.
Cave J, Goldacre M, Lambert T, Woolf K, Jones A, Dacre J: Newly qualified doctors' views about whether their medical school had trained them well: questionnaire surveys. BMC Med Educ. 2007, 7: 38-10.1186/1472-6920-7-38.
Whittle SR, Whelan B, Murdoch-Eaton DG: DREEM and beyond; studies of the educational environment as a means for its enhancement. Educ Health (Abingdon). 2007, 20 (1): 7.
Bassaw B, Roff S, McAleer S, Roopnarinesingh S, De Lisle J, Teelucksingh S, Gopaul S: Students' perspectives on the educational environment, Faculty of Medical Sciences, Trinidad. Med Teach. 2003, 25 (5): 522-526. 10.1080/0142159031000137409.
Genn JM: AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med Teach. 2001, 23 (4): 337-344. 10.1080/01421590120063330.
Roff S: The Dundee Ready Educational Environment Measure (DREEM)--a generic instrument for measuring students' perceptions of undergraduate health professions curricula. Med Teach. 2005, 27 (4): 322-325. 10.1080/01421590500151054.
Nishigori H, Yoshida I: DREEM, PHEEM, ATEEM and STEEM in Japanese. Igaku-Kyoiku (Japanese). 2006, 37 (2): 4-14.
Nishigori H, Nishigori M, Yoshimura H: DREEM, PHEEM, ATEEM and STEEM in Japanese. Med Teach. 2009, 31 (6): 560-10.1080/01421590903136488.
de Oliveira Filho GR, Vieira JE, Schonhorst L: Psychometric properties of the Dundee Ready Educational Environment Measure (DREEM) applied to medical residents. Med Teach. 2005, 27 (4): 343-347. 10.1080/01421590500046387.
Dial TH, Lindley DW: Predictive validity of specialty choice data from AAMC graduation questionnaire. Association of Medical Colleges. J Med Educ. 1987, 62 (12): 955-958.
Coates WC, Crooks K, Slavin SJ, Guiton G, Wilkerson L: Medical school curricular reform: fourth-year colleges improve access to career mentoring and overall satisfaction. Acad Med. 2008, 83 (8): 754-760. 10.1097/ACM.0b013e31817eb7dc.
Patel MS, Lypson ML, Davis MM: Medical student perceptions of education in health care systems. Acad Med. 2009, 84 (9): 1301-1306. 10.1097/ACM.0b013e3181b17e3e.
Obara H, Tokuda Y, Orlander JD: Perceived Preparedness of Japanese Medical Graduates for Postgraduate Training: a Comparison to US Graduates. 2008, Society of General Internal Medicine; Pittsburgh, USA
Balcetis E, Dunning D, Miller R: Do collectivists "know themselves" better than individualists? Cross-cultural studies of the "holier than thou" phenomenon. Journal of Personality and Social Psychology. 2008, 95 (6): 1252-1267. 10.1037/a0013195.
Sasaki H, Arimura K, Itoyama Y, Kwak S, Kira J, Nakashima K, Amano T, Inoue K, Uozumi T, Kohara N, et al: Nationwide questionnaire study in "the Model Core Curriculum" and current status for the undergraduate education in neurology. Rinsho Shinkeigaku(Japanese). 2008, 48 (8): 556-562.
Onishi H, Yoshida I: Rapid change in Japanese medical education. Med Teach. 2004, 26 (5): 403-408. 10.1080/01421590412331270492.
Tierney L: An experience in Japanese academic medicine. Western Journal of Medicine. 1994, 160 (2): 139.
Hutchinson L: Educational environment. BMJ. 2003, 326 (7393): 810-812. 10.1136/bmj.326.7393.810.
McAleer S, Soemantri D, Roff S: Chaper 9. Educational environment. A practical guide for medical teachers. 2009, Edinburgh; New York: Elsevier Churchill Livingstone, 64-72. 3
Genn JM, Harden RM: What is medical education here really like? Suggestions for action research studies of climates of medical education environments. Med Teach. 1986, 8 (2): 111-124. 10.3109/01421598609010737.
Kozu T: Medical education in Japan. Acad Med. 2006, 81 (12): 1069-1075. 10.1097/01.ACM.0000246682.45610.dd.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/10/35/prepub
We thank Dr Sue Roff, Center for Medical Education, Dundee University, for allowing us to translate the DREEM into Japanese. We are grateful to Dr Hiroshi Yoshimura and Dr Akira Naito for their valuable advice and to Ms. Sayoko Yamauchi and Ms. Kayo Ichikawa for their excellent secretarial work. We also thank Dr Sue Roff, Dr Edward Krupat, Dr Phillip Cotton, and Dr Yasuyuki Suzuki for their excellent reviews.
This study was supported by a research grant from the Japanese Ministry of Health, Welfare, and Labor. The funding agency had no role in data acquisition, analysis and manuscript preparation.
The authors declare that they have no competing interests.
YT conceived of the study, analyzed the data, and drafted the manuscript. EG, JO, JJ, and FO participated in the design of the study and assisted writing of the manuscript. HO participated in its design and coordination. MS, KS, YI, SO, OT, and TF participated in the design of the study and assisted the data analysis. All authors read and approved the final manuscript.
Authors’ original submitted files for images
About this article
Cite this article
Tokuda, Y., Goto, E., Otaki, J. et al. Undergraduate educational environment, perceived preparedness for postgraduate clinical training, and pass rate on the National Medical Licensure Examination in Japan. BMC Med Educ 10, 35 (2010). https://doi.org/10.1186/1472-6920-10-35
- Pass Rate
- Objective Structure Clinical Examination
- Educational Environment
- Clinical Area
- Undergraduate Medical Education