- Research article
- Open Open Peer Review
Educational climate seems unrelated to leadership skills of clinical consultants responsible of postgraduate medical education in clinical departments
BMC Medical Educationvolume 10, Article number: 62 (2010)
The educational climate is crucial in postgraduate medical education. Although leaders are in the position to influence the educational climate, the relationship between leadership skills and educational climate is unknown. This study investigates the relationship between the educational climate in clinical departments and the leadership skills of clinical consultants responsible for education.
The study was a trans-sectional correlation study. The educational climate was investigated by a survey among all doctors (specialists and trainees) in the departments. Leadership skills of the consultants responsible for education were measured by multi-source feedback scores from heads of departments, peer consultants, and trainees.
Doctors from 42 clinical departments representing 21 specialties participated. The response rate of the educational climate investigation was moderate 52% (420/811), Response rate was high in the multisource-feedback process 84.3% (420/498). The educational climate was scored quite high mean 3.9 (SD 0.3) on a five-point Likert scale. Likewise the leadership skills of the clinical consultants responsible for education were considered good, mean 5.4 (SD 0.6) on a seven-point Likert scale. There was no significant correlation between the scores concerning the educational climate and the scores on leadership skills, r = 0.17 (p = 0.29).
This study found no relation between the educational climate and the leadership skills of the clinical consultants responsible for postgraduate medical education in clinical departments with the instruments used. Our results indicate that consultants responsible for education are in a weak position to influence the educational climate in the clinical department. Further studies are needed to explore, how heads of departments and other factors related to the clinical organisation could influence the educational climate.
Postgraduate medical education (PGME) is a work-based education where learning and teaching takes place in a clinical context. On the one hand the young doctor (trainee) is under education and on the other hand he is a member of the staff in the clinical department. The clinical departments face the challenge of creating an educational environment that is supportive and learning-oriented  and at the same time meeting the demands from society to deliver efficient clinical services and research [2, 3]. A common feature in PGME is the need for supervision and feedback among trainees and for being engaged in the responsibility for patients [4, 5]. Furthermore, trainees need to be appreciated and valued as team members . The challenge is to find a proper balance between involving trainees in patient treatment, and at the same time ensure patient safety and meet demands for a high production . As the educational climate is perceived to have major influence on how trainees learn and perform, the quality of the educational climate has received increasing attention in the literature on PGME [1, 5, 7].
However, the concept educational climate has been used indiscriminately with culture, environment or learning context . Nevertheless, all of these concepts include the same elements: atmosphere [7–9], the connection and personal relation with colleagues [5, 9, 10], openness to questions and appropriateness of supervision and feedback [1, 5, 9], and eventually the working conditions and organisation of the work [5, 7, 9]. The educational climate may differ considerably across clinical departments. Hence, in order to be able to change the educational climate, it is relevant to find out who contributes to this climate and to the specific ways of behavior and "living" in the clinical departments.
The organisational culture, according to Schein , is closely connected to leadership. Although all employees take part in developing the culture in a department, the leaders are in a position to deliberately influence the culture . In PGME leaders can influence the educational climate by prioritising and attending a variety of activities in the department and by role-modelling. Various authors have described the "good physician leader" as a necessity to the survival of teaching hospitals [10, 12–14]. And since the responsibilities and obligations in managing PGME in the clinical departments are many, and has become a growing business , the nomination of leaders of PGME in clinical departments has been introduced in many countries [15–18].
However we do not know much of these educational leaders in PGME. In a previous study on the clinical consultant responsible for education (CRE) in clinical departments, we found that stakeholders expected the CRE to develop and improve the educational climate , which is in accordance with the description of desired competencies for a leader of PGME provided by Wong et al. . They also indicated that structure of the educational program in the department was a necessary but not sufficient prerequisite for the educational climate. The CREs were expected to take the lead regarding educational matters and stakeholders perceived the CREs to have fairly good administrative and leadership skills . Hence, a positive relation between leadership skills of the CRE and the quality of the educational climate might be expected. Likewise, a positive relation between the CRE's administrative skills and the organisation of the daily clinical work could be expected. However, this inference cannot be drawn from current literature on PGME.
The purpose of this study was to explore the relationship between the educational climate in clinical departments and leadership skills in clinical consultants responsible for PGME.
The study was a trans-sectional correlation study on the relationship between the educational climate in clinical departments and leadership performance of CREs. The unit of analysis is the clinical department.
Context of the study
Postgraduate medical education in Denmark is governed by the Danish National Board of Health. For clinical departments participating in PGME, it is mandatory to nominate one of the clinical consultants in the department to be leader of PGME in the department (CRE). The CRE has responsibility for a highly diverse group of trainees undergoing a number of different PGME programs at each clinical department. At the same time the position of a CRE is an important link between the administrative line and the educational line as shown in Figure 1. The CRE manages PGME in the clinical department and has both administrative and leader responsibilities ranging from organising the work in the department to assure all trainees get the proper education, through monitoring the evaluation of the trainees, to appointing and supervising the clinical teachers and supervisors in the department.
This investigation took place in the Northern Educational Region in Denmark. This region covers one third of the country and includes both university and non-university hospitals. CREs from clinical departments with more than three consultants in addition to the head of the department and more than three trainees were eligible for inclusion.
CREs who had previously participated in a leadership course for CREs were excluded as they were included in another study also collecting multi-source feedback (MSF) data. Thus, a total of 79 CREs and their departments were eligible for inclusion. Participants were contacted by phone and informed about the study, the questionnaire on the educational climate and the MSF procedure. All gave informed consent. Confidentiality was guaranteed and participants were assured that it would be impossible to trace findings to individual participants, clinical departments or hospitals. The study was presented to the ethical committee for Viborg and Aalborg County. In our jurisdiction studies of this kind do not need approval.
Questionnaire on the educational climate
Various instruments have been introduced to measure educational climate in PGME [9, 21]. We chose a previously validated Danish instrument, derived from the Postgraduate Hospital Educational Environment Measure (PHEEM) instrument [9, 21]. In this study we adjusted the instrument to comprise answers from all doctors in the department, thus including both specialists and trainees. The assumption was that all doctors in a department have an opinion about the educational climate . The questionnaire comprised 36 statements divided into three main areas concerning the educational climate: 1) Learning opportunities, 2) Supervision and feedback, and 3) Organisation of work. A five-point Likert scale was used (1 = "totally disagree" and 5 = "totally agree" or best score). An e-mail based electronic system (Enalyzer®) was used to collect data.
MSF on leadership skills
Multi-source feedback is a widely accepted tool for measuring leadership skills [22–24]. In this study we used an MSF instrument previously developed to specifically evaluate leadership skills of CREs . Multi-source feedback scores were collected from heads of department, consultants and trainees . The MSF instrument comprised 69 statements divided into four categories: 1)Technical skills, referring to the proficiency in specific leadership methods and processes; 2) Human skills, including the ability to work with and through people to meet goals; 3) Citizenship behaviour referring to professionalism regarding interpersonal, organisational and job/task performance; 4) Administrative skills, involving knowledge of the planning, organizing and coordinating of tasks . Leaders' skills are normally perceived as two-dimensional with a leadership dimension and a management dimension. Accordingly, the statements in technical skills, human skills and citizenship behavior comprise leadership skills while administrative skills refer to management. Each statement was scored on a seven-point Likert scale (1 = "not at all" and 7 = "always" or best score). The option "not able to answer" was provided. The CRE chose at least three consultants and three trainees in the department to secure anonymity in addition to the head of department. Enalyzer® was used to collect data
Mean scores of educational climate and MSF were calculated. If an item score was missing it was replaced by a mean of all other scores in the same category from the same respondent, provided that more than half of the items in the category were scored. If scores on more than half of the items were missing the respondent was excluded from further analysis. Separate mean scores were calculated for each of the three categories in the educational climate questionnaire, and separated into mean score from specialists and from trainees. Similarly, separate mean scores were calculated for the two overall categories "leadership skills" and "management skills" in the MSF procedure.
The overall educational climate score and scores for the three categories in the educational climate questionnaire were correlated to the overall MSF scores and the scores for leadership skills and management skills, respectively using Pearson's correlation coefficient. Scores from trainees and specialists were compared using Oneway ANOVA. A p-value < 0.05 was considered significant.
Descriptive statistics were used to examine whether characteristics of study sample were comparable to the background population. For this purpose the departments were categorised according to specialty type into 1) cognitive specialties (internal medicine and subspecialties, paediatrics, dermatology, oncology, psychiatry and neurology), 2) surgical specialties (surgery, orthopaedic surgery, urology, gynaecology, ophthalmology, otology, thoracic surgery, vascular surgery, brain surgery) and 3) technical specialties (anaesthesiology, radiology, all laboratory specialties).
Figure 2 show how the 154 eligible departments in the Northern Educational Region in Denmark resulted in a study population of 56 departments. Participants represented 21 of the 36 specialties in Denmark. The distribution between cognitive, surgical and technical departments in the study population did not reflect the background population mainly because many of the technical specialty departments (especially the laboratory departments) were too small to participate in the MSF procedure (Table 1). The proportion of cognitive and surgical departments was the same in the study group compared to the background population.
An average of ten doctors answered the questionnaire on the educational climate in each department. The response-rate was 52% (420/811). In the MSF-process each CRE had ten respondents on average (a total of 420 respondents answered the MSF). The response rate was high 84.3% (420/498). Mean educational climate and MSF scores are shown in Table 2. There was no significant correlation between total mean scores of the educational climate and the MSF scores, r = 0.17, p = 0.29. Similarly, there were no significant correlations between the three categories of the educational climate and leadership skills or administrative skills of the leader of PGME in the clinical department for neither the trainees' nor the specialists' scores on educational climate. Pearson's correlation coefficients varied between 0.29 (p = 0.07) and 0.03 (p = 0.87).
Surprisingly, this study did not show a significant correlation between the educational climate in clinical departments and the leadership performance of the CRE. There could be various explanations for this finding including both internal and external validity threats.
Firstly, the scores on the educational climate were quite high with a mean score of 3.9 (SD 0.3) on a five-point Likert scale. Moreover, MSF scores on leadership performance were quite high with a mean value of 5.4 (SD 0.6) on a seven-point Likert scale. Both results might indicate an instrumentation bias. However, the scores on the educational climate in our study varied from 3.0 and 4.5. An educational climate score of 3.0 should be considered a low score, since there is a tendency to get positive scores in measurements of educational climate [25, 26]. Likewise, respondents in MSF procedures are known to give high scores on MSF [20, 22]. The scores on leadership performance ranged between 4.2 and 6.3. A score of 4.2 indicates a rather low leadership performance. The instruments therefore are both able to separate high performers from lower performing CREs and good from a less positive educational climate.
The response rate was moderate 52% (420/811) on the questionnaire on the educational climate and might pose a threat to the validity of these results. In average we got response from ten doctors from each department, which is enough to get a reliable measurement of the educational climate .
Finally, we chose to calculate a total MSF score for all respondents. When measuring leadership performance through an MSF procedure you usually separate the respondents into subgroups according to their position in the organisation. Many studies have shown that you perceive the leaders' performance differently according to your position in the organisation (head of department, peer consultant and trainee) . However, in a previous study we have shown that there were only minor differences between the scores of various respondent groups on a MSF process in CREs in clinical departments .
The rather high average score in educational climate and MSF scorings might indicate a positive selection bias. However, we excluded the departments where CREs had previously voluntarily signed up for a leadership course and most probably represented the most enthusiastic CREs in the region. Moreover, our study sample included 56 departments covering CREs and departments from a whole region in the country and comprising both university and non-university hospitals in addition to representing many specialties. Even with the lower representation of technical specialties in our study population compared to the background population we feel confident that the results reflect the population in general. Especially since the ratio between the cognitive and surgical specialty departments was the same in the study and the background population. The MSF instrument was developed in a way that only doctors could be invited as respondents. Extending the respondent groups to other staff groups in the department might be considered in future studies in order to achieve a more equal representation of specialties.
In summary, although we acknowledge limitations to our study these do not fully account for our findings. Therefore other explanations to the lack of relation between educational climate and CREs' leadership skills might be speculated, including organisational issues of PGME.
In one way PGME relates to a parallel organisation outside the organisation of hospitals and other health care organisations where PGME takes place [6, 28]. PGME is governed by outside bodies like a national boards of health (United Kingdom, Denmark)[15, 29], the Accreditations Councils (USA) , specialist societies (Canada)  or the union (Norway) . Contrary to this the CRE refers to the head of department in administrative matters. This unclear line of reference combined with an unclear task description may contribute to the CRE's low impact . The hospitals are run administratively through the heads of departments. This places the CRE in a position as middle manager primarily concerned with interpreting and implementing policies and programs from the educational bodies or places him as a low-level manager supposedly engaged in structuring, coordinating and facilitating work activities . In a previous study we have shown that the CRE is expected to manage a whole range of administrative duties , among them ensuring that trainees are exposed to the clinical situations they are supposed to learn from. Structuring the learning opportunities and processes is very important in clerkship education [31, 32], and there is reason to believe that it might be even more important in PGME, where involvement, participation and interpersonal relations are so fundamental . Since stakeholders in PGME found that the CRE masters the administrative duties well , we would have expected a relation between the organisation of work and management skills of the CRE. Finding no relation might reflect that the CRE have limited influence on the planning of daily work schedules. Both CREs and stakeholders have suggested the development of a specific leadership course for CREs to strengthen their position . However, if the CRE is in a weak position to influence the working organisation and educational climate these initiatives may be in vain .
CREs might be fairly good leaders but acting in a system that makes it difficult to be perceived as a leader of education and creator of the educational culture. Additionally, the educational climate might be so mixed up in the work environment that maybe focus should be on the working culture instead of isolating the educational climate. This would involve asking other staff groups about their perception of the working culture in the clinical departments.
To further explore factors that influence the educational climate it might be relevant to focus on the leadership performance of the administrative heads of the clinical departments. In particular how the head of the department prioritises PGME and attends to the educational mission in the department. This might have significant influence on the CRE's possibility to excert leadership of education and fulfil expectations .
Our results indicate that there is no relationship between the educational climate in clinical departments and the leadership performance of educational leaders of PGME in the department. The separated administrative and educational lines of reference in PGME might explain this lack of relation. Future studies should focus on exploring how administrative leaders of clinical departments and perhaps other factors related to the clinical organisation influence the educational climate.
BM: MD, MHPE and associate professor in postgraduate medical education is director for specialist training at Aarhus University Hospital, Skejby, Denmark.
LSM: MD, PhD and associate professor in postgraduate medical education is consultant at the Department of Internal Medicine and director for specialist training at the Regional Hospital, Viborg, Denmark.
AJJS: MD, PhD is professor of medical education and scientific director of the Institute for Education, Faculty Health, Medicine and Life Sciences, Maastricht University, Netherlands.
CR: MD, PhD, MHPE is professor of medical education and director of Centre for Clinical Education, Copenhagen University and Capital Region, Rigshospitalet, Denmark.
Postgraduate medical education
Consultant responsible for education in clinical department
Danish National board of Health
Postgraduate hospital educational environment measure
Hoff TJ, Pohl H, Bartfield J: Creating a learning environment to produce competent residents: the roles of culture and context. Acad Med. 2004, 79: 532-539. 10.1097/00001888-200406000-00007.
Beauchamp RD: The changing roles of a surgical department chair: adapting to a changing environment. Arch Surg. 2005, 140: 258-263. 10.1001/archsurg.140.3.258.
Naylor CD: Leadership in academic medicine: reflections from administrative exile. Clin Med. 2006, 6: 488-492.
Bleakley A: Pre-registration house officers and ward-based learning: a 'new apprenticeship' model. Med Educ. 2002, 36: 9-15. 10.1046/j.1365-2923.2002.01128.x.
Kendall ML, Hesketh EA, Macpherson SG: The learning environment for junior doctor training--what hinders, what helps. Med Teach. 2005, 27: 619-624. 10.1080/01421590500069710.
Schwartzstein RM, Huang GC, Coughlin CM: Development and implementation of a comprehensive strategic plan for medical education at an academic medical center. Acad Med. 2008, 83: 550-559. 10.1097/ACM.0b013e3181722c7c.
Cross V, Hicks C, Parle J, Field S: Perceptions of the learning environment in higher specialist training of doctors: implications for recruitment and retention. Med Educ. 2006, 40: 121-128. 10.1111/j.1365-2929.2005.02382.x.
Genn JM: AMEE Medical Education Guide No. 23 (Part 2): Curriculum, environment, climate, quality and change in medical education - a unifying perspective. Med Teach. 2001, 23: 445-454.
Roff S, McAleer S, Skinner A: Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach. 2005, 27: 326-331. 10.1080/01421590500150874.
Parsell G, Bligh J: Encouraging educational leadership. Med Educ. 2000, 34: 199-200. 10.1046/j.1365-2923.2000.0652a.x.
Schein EH: Organizational culture and leadership. 2004, San Francisco: Jossey-Bass
Schwartz RW, Pogge C: Physician leadership is essential to the survival of teaching hospitals. Am J Surg. 2000, 179: 462-468. 10.1016/S0002-9610(00)00395-0.
Souba WW: The new leader: new demands in a changing, turbulent environment. J Am Coll Surg. 2003, 197: 79-87. 10.1016/S1072-7515(03)00335-1.
Wong JG, Fagan M, Pinsker J: Expectations of and for the medical director of the resident's ambulatory clinic. Am J Med. 2001, 111: 84-87. 10.1016/S0002-9343(01)00764-1.
Danish Ministry of Health: Fremtidens Speciallæge (The future specialist). Danish. 1. 2000, Danish Ministry of Health, Ref Type: Report
Danish National Board of Health: Vejledning og evaluering i den lægelige videreuddannelse (Guidance and evaluation in the training of specialists in Denmark). Danish. 1998, DNBH Publications, 1
Accreditation council for medical education (ACGME). Program director guide. 2009, 30-9-2009. Ref Type: Internet Communication, [http://www.acgme.org]
The Royal College of Physicians and Surgeons in Canada (RCPSC). General standards applicable to the university and affiliated sites. 2007, 30-9-2009. Ref Type: Internet Communication, [http://www.rcpsc.medical.org]
Malling B, Scherpbier AJ, Ringsted C: What is the role of the consultant responsible for postgraduate education in the clinical department?. Med Teach. 2007, 29: 471-477. 10.1080/01421590701510561.
Malling B, Bonderup T, Mortensen L, Ringsted C, Scherpbier A: Effects of multi-source feedback on developmental plans for leaders of postgraduate medical education. Med Educ. 2009, 43: 159-167. 10.1111/j.1365-2923.2008.03259.x.
Aspegren K, Bastholt L, Bested KM, Bonnesen T, Ejlersen E, Fog I, et al: Validation of the PHEEM instrument in a Danish hospital setting. Med Teach. 2007, 29: 498-500. 10.1080/01421590701477357.
Bracken DW, Timmreck CW, Church AH: The handbook of multi-source feedback. The comprehensive resource for designing and implementing MSF processes. 2001, San Francisco: Jossey-Bass, 1
Leadership Qualities Framework, National Health Service (NHS), UK. 2009, Ref Type: Internet Communication, [http://www.nhsleadershipqualities.nhs.uk]
Yukl GA: Leadership in organizations. 2006, Upper Saddle River, N.J.: Pearson Prentice Hall, 6
Cassar K: Development of an instrument to measure the surgical operating theatre learning environment as perceived by basic surgical trainees. Med Teach. 2004, 26: 260-264. 10.1080/0142159042000191975.
Clapham M, Wall D, Batchelor A: Educational environment in intensive care medicine - use of Postgraduate Hospital Educational Environment Measure (PHEEM). Med Teach. 2007, 1-8.
Boor K, Scheele F, van d, Scherpbier AJ, Teunissen PW, Sijtsma K: Psychometric properties of an instrument to measure the clinical learning environment. Med Educ. 2007, 41: 92-99. 10.1111/j.1365-2929.2006.02651.x.
Weiner BJ, Culbertson R, Jones RF, Dickler R: Organizational models for medical school-clinical enterprise relationships. Acad Med. 2001, 76: 113-124. 10.1097/00001888-200102000-00007.
Postgraduate medical education and training board. Strategy document: 2006-2010. 2009, 30-9-2009. Ref Type: Internet Communication, [http://www.gmc-uk.org/]
Den Norske Legeforening. Specialist training for physicians in Norway. 2009, 30-9-2009. Ref Type: Internet Communication, [http://www.legeforeningen.no]
Durak HI, Vatansever K, van DJ, van d V: Factors determining students' global satisfaction with clerkships: an analysis of a two year students' ratings database. Adv Health Sci Educ Theory Pract. 2008, 13: 495-502. 10.1007/s10459-007-9061-7.
van der Hem-Stokroos HH, Daelmans HE, van d, Haarman HJ, Scherpbier AJ: A qualitative study of constructive clinical learning experiences. Med Teach. 2003, 25: 120-126. 10.1080/0142159031000092481.
Teunissen PW, Scheele F, Scherpbier AJ, van d, Boor K, van Luijk SJ, et al: How residents learn: qualitative evidence for the pivotal role of clinical activities. Med Educ. 2007, 41: 763-770. 10.1111/j.1365-2923.2007.02778.x.
Malling B, Mortensen L, Bonderup T, Scherpbier AJJA, Ringsted C: Combining a leadership course and multi-source feedback has no effect on leadership skills of leaders in postgraduate medical education. An intervention study with a control group. BMC Medical Education. 2009, 9: 72-10.1186/1472-6920-9-72.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/10/62/prepub
The study was funded by the KUL-fund for quality improvement in PGME, Aarhus County and by the Scientific Funds of Viborg and Aalborg County respectively. The funds did not interfere with the study, and did not take part in decisions regarding study design, collection and analysis of data, interpretation of data or submission of the results.
The authors declare that they have no competing interests.
BM and LM made substantial contributions to the conception, design and the acquisition of data. BM analyzed the data. BM, LSM, AJJS and CR made substantial contribution to the interpretation of data and drafting of the manuscript. BM, LSM, AJJS and CR all made substantial contributions in critically revising the manuscript and content. All authors have given final approval of the version published.