Educational climate seems unrelated to leadership skills of clinical consultants responsible of postgraduate medical education in clinical departments
© Malling et al; licensee BioMed Central Ltd. 2010
Received: 7 April 2010
Accepted: 21 September 2010
Published: 21 September 2010
The educational climate is crucial in postgraduate medical education. Although leaders are in the position to influence the educational climate, the relationship between leadership skills and educational climate is unknown. This study investigates the relationship between the educational climate in clinical departments and the leadership skills of clinical consultants responsible for education.
The study was a trans-sectional correlation study. The educational climate was investigated by a survey among all doctors (specialists and trainees) in the departments. Leadership skills of the consultants responsible for education were measured by multi-source feedback scores from heads of departments, peer consultants, and trainees.
Doctors from 42 clinical departments representing 21 specialties participated. The response rate of the educational climate investigation was moderate 52% (420/811), Response rate was high in the multisource-feedback process 84.3% (420/498). The educational climate was scored quite high mean 3.9 (SD 0.3) on a five-point Likert scale. Likewise the leadership skills of the clinical consultants responsible for education were considered good, mean 5.4 (SD 0.6) on a seven-point Likert scale. There was no significant correlation between the scores concerning the educational climate and the scores on leadership skills, r = 0.17 (p = 0.29).
This study found no relation between the educational climate and the leadership skills of the clinical consultants responsible for postgraduate medical education in clinical departments with the instruments used. Our results indicate that consultants responsible for education are in a weak position to influence the educational climate in the clinical department. Further studies are needed to explore, how heads of departments and other factors related to the clinical organisation could influence the educational climate.
Postgraduate medical education (PGME) is a work-based education where learning and teaching takes place in a clinical context. On the one hand the young doctor (trainee) is under education and on the other hand he is a member of the staff in the clinical department. The clinical departments face the challenge of creating an educational environment that is supportive and learning-oriented  and at the same time meeting the demands from society to deliver efficient clinical services and research [2, 3]. A common feature in PGME is the need for supervision and feedback among trainees and for being engaged in the responsibility for patients [4, 5]. Furthermore, trainees need to be appreciated and valued as team members . The challenge is to find a proper balance between involving trainees in patient treatment, and at the same time ensure patient safety and meet demands for a high production . As the educational climate is perceived to have major influence on how trainees learn and perform, the quality of the educational climate has received increasing attention in the literature on PGME [1, 5, 7].
However, the concept educational climate has been used indiscriminately with culture, environment or learning context . Nevertheless, all of these concepts include the same elements: atmosphere [7–9], the connection and personal relation with colleagues [5, 9, 10], openness to questions and appropriateness of supervision and feedback [1, 5, 9], and eventually the working conditions and organisation of the work [5, 7, 9]. The educational climate may differ considerably across clinical departments. Hence, in order to be able to change the educational climate, it is relevant to find out who contributes to this climate and to the specific ways of behavior and "living" in the clinical departments.
The organisational culture, according to Schein , is closely connected to leadership. Although all employees take part in developing the culture in a department, the leaders are in a position to deliberately influence the culture . In PGME leaders can influence the educational climate by prioritising and attending a variety of activities in the department and by role-modelling. Various authors have described the "good physician leader" as a necessity to the survival of teaching hospitals [10, 12–14]. And since the responsibilities and obligations in managing PGME in the clinical departments are many, and has become a growing business , the nomination of leaders of PGME in clinical departments has been introduced in many countries [15–18].
However we do not know much of these educational leaders in PGME. In a previous study on the clinical consultant responsible for education (CRE) in clinical departments, we found that stakeholders expected the CRE to develop and improve the educational climate , which is in accordance with the description of desired competencies for a leader of PGME provided by Wong et al. . They also indicated that structure of the educational program in the department was a necessary but not sufficient prerequisite for the educational climate. The CREs were expected to take the lead regarding educational matters and stakeholders perceived the CREs to have fairly good administrative and leadership skills . Hence, a positive relation between leadership skills of the CRE and the quality of the educational climate might be expected. Likewise, a positive relation between the CRE's administrative skills and the organisation of the daily clinical work could be expected. However, this inference cannot be drawn from current literature on PGME.
The purpose of this study was to explore the relationship between the educational climate in clinical departments and leadership skills in clinical consultants responsible for PGME.
The study was a trans-sectional correlation study on the relationship between the educational climate in clinical departments and leadership performance of CREs. The unit of analysis is the clinical department.
Context of the study
This investigation took place in the Northern Educational Region in Denmark. This region covers one third of the country and includes both university and non-university hospitals. CREs from clinical departments with more than three consultants in addition to the head of the department and more than three trainees were eligible for inclusion.
CREs who had previously participated in a leadership course for CREs were excluded as they were included in another study also collecting multi-source feedback (MSF) data. Thus, a total of 79 CREs and their departments were eligible for inclusion. Participants were contacted by phone and informed about the study, the questionnaire on the educational climate and the MSF procedure. All gave informed consent. Confidentiality was guaranteed and participants were assured that it would be impossible to trace findings to individual participants, clinical departments or hospitals. The study was presented to the ethical committee for Viborg and Aalborg County. In our jurisdiction studies of this kind do not need approval.
Questionnaire on the educational climate
Various instruments have been introduced to measure educational climate in PGME [9, 21]. We chose a previously validated Danish instrument, derived from the Postgraduate Hospital Educational Environment Measure (PHEEM) instrument [9, 21]. In this study we adjusted the instrument to comprise answers from all doctors in the department, thus including both specialists and trainees. The assumption was that all doctors in a department have an opinion about the educational climate . The questionnaire comprised 36 statements divided into three main areas concerning the educational climate: 1) Learning opportunities, 2) Supervision and feedback, and 3) Organisation of work. A five-point Likert scale was used (1 = "totally disagree" and 5 = "totally agree" or best score). An e-mail based electronic system (Enalyzer®) was used to collect data.
MSF on leadership skills
Multi-source feedback is a widely accepted tool for measuring leadership skills [22–24]. In this study we used an MSF instrument previously developed to specifically evaluate leadership skills of CREs . Multi-source feedback scores were collected from heads of department, consultants and trainees . The MSF instrument comprised 69 statements divided into four categories: 1)Technical skills, referring to the proficiency in specific leadership methods and processes; 2) Human skills, including the ability to work with and through people to meet goals; 3) Citizenship behaviour referring to professionalism regarding interpersonal, organisational and job/task performance; 4) Administrative skills, involving knowledge of the planning, organizing and coordinating of tasks . Leaders' skills are normally perceived as two-dimensional with a leadership dimension and a management dimension. Accordingly, the statements in technical skills, human skills and citizenship behavior comprise leadership skills while administrative skills refer to management. Each statement was scored on a seven-point Likert scale (1 = "not at all" and 7 = "always" or best score). The option "not able to answer" was provided. The CRE chose at least three consultants and three trainees in the department to secure anonymity in addition to the head of department. Enalyzer® was used to collect data
Mean scores of educational climate and MSF were calculated. If an item score was missing it was replaced by a mean of all other scores in the same category from the same respondent, provided that more than half of the items in the category were scored. If scores on more than half of the items were missing the respondent was excluded from further analysis. Separate mean scores were calculated for each of the three categories in the educational climate questionnaire, and separated into mean score from specialists and from trainees. Similarly, separate mean scores were calculated for the two overall categories "leadership skills" and "management skills" in the MSF procedure.
The overall educational climate score and scores for the three categories in the educational climate questionnaire were correlated to the overall MSF scores and the scores for leadership skills and management skills, respectively using Pearson's correlation coefficient. Scores from trainees and specialists were compared using Oneway ANOVA. A p-value < 0.05 was considered significant.
Descriptive statistics were used to examine whether characteristics of study sample were comparable to the background population. For this purpose the departments were categorised according to specialty type into 1) cognitive specialties (internal medicine and subspecialties, paediatrics, dermatology, oncology, psychiatry and neurology), 2) surgical specialties (surgery, orthopaedic surgery, urology, gynaecology, ophthalmology, otology, thoracic surgery, vascular surgery, brain surgery) and 3) technical specialties (anaesthesiology, radiology, all laboratory specialties).
Distribution of specialties in the background and the study population.
Eligible clinical departments
Excluded - previous leadership course
Excluded - department too small
Included in study
Scores on multi-source feedback and educational climate.
Multi-source feedback (max score = 7)
4.2 - 6.3
4.2 - 6.3
4.2 - 6.3
Educational climate (max score = 5)
All doctors in department
3.0 - 4,5
3.2 - 4.6
2.6 - 4.5
All doctors in department
3.3 - 4.4
2,6 - 4,2
2.9 - 4.5
Supervision and feedback*
All doctors in department
2.6 - 4.7
2.8 - 4,9
2.3 - 4,5
Organisation of work
All doctors in department
3.2 - 4.7
3.6 - 4.7
2,7 - 4,8
Surprisingly, this study did not show a significant correlation between the educational climate in clinical departments and the leadership performance of the CRE. There could be various explanations for this finding including both internal and external validity threats.
Firstly, the scores on the educational climate were quite high with a mean score of 3.9 (SD 0.3) on a five-point Likert scale. Moreover, MSF scores on leadership performance were quite high with a mean value of 5.4 (SD 0.6) on a seven-point Likert scale. Both results might indicate an instrumentation bias. However, the scores on the educational climate in our study varied from 3.0 and 4.5. An educational climate score of 3.0 should be considered a low score, since there is a tendency to get positive scores in measurements of educational climate [25, 26]. Likewise, respondents in MSF procedures are known to give high scores on MSF [20, 22]. The scores on leadership performance ranged between 4.2 and 6.3. A score of 4.2 indicates a rather low leadership performance. The instruments therefore are both able to separate high performers from lower performing CREs and good from a less positive educational climate.
The response rate was moderate 52% (420/811) on the questionnaire on the educational climate and might pose a threat to the validity of these results. In average we got response from ten doctors from each department, which is enough to get a reliable measurement of the educational climate .
Finally, we chose to calculate a total MSF score for all respondents. When measuring leadership performance through an MSF procedure you usually separate the respondents into subgroups according to their position in the organisation. Many studies have shown that you perceive the leaders' performance differently according to your position in the organisation (head of department, peer consultant and trainee) . However, in a previous study we have shown that there were only minor differences between the scores of various respondent groups on a MSF process in CREs in clinical departments .
The rather high average score in educational climate and MSF scorings might indicate a positive selection bias. However, we excluded the departments where CREs had previously voluntarily signed up for a leadership course and most probably represented the most enthusiastic CREs in the region. Moreover, our study sample included 56 departments covering CREs and departments from a whole region in the country and comprising both university and non-university hospitals in addition to representing many specialties. Even with the lower representation of technical specialties in our study population compared to the background population we feel confident that the results reflect the population in general. Especially since the ratio between the cognitive and surgical specialty departments was the same in the study and the background population. The MSF instrument was developed in a way that only doctors could be invited as respondents. Extending the respondent groups to other staff groups in the department might be considered in future studies in order to achieve a more equal representation of specialties.
In summary, although we acknowledge limitations to our study these do not fully account for our findings. Therefore other explanations to the lack of relation between educational climate and CREs' leadership skills might be speculated, including organisational issues of PGME.
In one way PGME relates to a parallel organisation outside the organisation of hospitals and other health care organisations where PGME takes place [6, 28]. PGME is governed by outside bodies like a national boards of health (United Kingdom, Denmark)[15, 29], the Accreditations Councils (USA) , specialist societies (Canada)  or the union (Norway) . Contrary to this the CRE refers to the head of department in administrative matters. This unclear line of reference combined with an unclear task description may contribute to the CRE's low impact . The hospitals are run administratively through the heads of departments. This places the CRE in a position as middle manager primarily concerned with interpreting and implementing policies and programs from the educational bodies or places him as a low-level manager supposedly engaged in structuring, coordinating and facilitating work activities . In a previous study we have shown that the CRE is expected to manage a whole range of administrative duties , among them ensuring that trainees are exposed to the clinical situations they are supposed to learn from. Structuring the learning opportunities and processes is very important in clerkship education [31, 32], and there is reason to believe that it might be even more important in PGME, where involvement, participation and interpersonal relations are so fundamental . Since stakeholders in PGME found that the CRE masters the administrative duties well , we would have expected a relation between the organisation of work and management skills of the CRE. Finding no relation might reflect that the CRE have limited influence on the planning of daily work schedules. Both CREs and stakeholders have suggested the development of a specific leadership course for CREs to strengthen their position . However, if the CRE is in a weak position to influence the working organisation and educational climate these initiatives may be in vain .
CREs might be fairly good leaders but acting in a system that makes it difficult to be perceived as a leader of education and creator of the educational culture. Additionally, the educational climate might be so mixed up in the work environment that maybe focus should be on the working culture instead of isolating the educational climate. This would involve asking other staff groups about their perception of the working culture in the clinical departments.
To further explore factors that influence the educational climate it might be relevant to focus on the leadership performance of the administrative heads of the clinical departments. In particular how the head of the department prioritises PGME and attends to the educational mission in the department. This might have significant influence on the CRE's possibility to excert leadership of education and fulfil expectations .
Our results indicate that there is no relationship between the educational climate in clinical departments and the leadership performance of educational leaders of PGME in the department. The separated administrative and educational lines of reference in PGME might explain this lack of relation. Future studies should focus on exploring how administrative leaders of clinical departments and perhaps other factors related to the clinical organisation influence the educational climate.
BM: MD, MHPE and associate professor in postgraduate medical education is director for specialist training at Aarhus University Hospital, Skejby, Denmark.
LSM: MD, PhD and associate professor in postgraduate medical education is consultant at the Department of Internal Medicine and director for specialist training at the Regional Hospital, Viborg, Denmark.
AJJS: MD, PhD is professor of medical education and scientific director of the Institute for Education, Faculty Health, Medicine and Life Sciences, Maastricht University, Netherlands.
CR: MD, PhD, MHPE is professor of medical education and director of Centre for Clinical Education, Copenhagen University and Capital Region, Rigshospitalet, Denmark.
List of abbreviations
Postgraduate medical education
Consultant responsible for education in clinical department
Danish National board of Health
Postgraduate hospital educational environment measure
The study was funded by the KUL-fund for quality improvement in PGME, Aarhus County and by the Scientific Funds of Viborg and Aalborg County respectively. The funds did not interfere with the study, and did not take part in decisions regarding study design, collection and analysis of data, interpretation of data or submission of the results.
- Hoff TJ, Pohl H, Bartfield J: Creating a learning environment to produce competent residents: the roles of culture and context. Acad Med. 2004, 79: 532-539. 10.1097/00001888-200406000-00007.View ArticleGoogle Scholar
- Beauchamp RD: The changing roles of a surgical department chair: adapting to a changing environment. Arch Surg. 2005, 140: 258-263. 10.1001/archsurg.140.3.258.View ArticleGoogle Scholar
- Naylor CD: Leadership in academic medicine: reflections from administrative exile. Clin Med. 2006, 6: 488-492.View ArticleGoogle Scholar
- Bleakley A: Pre-registration house officers and ward-based learning: a 'new apprenticeship' model. Med Educ. 2002, 36: 9-15. 10.1046/j.1365-2923.2002.01128.x.View ArticleGoogle Scholar
- Kendall ML, Hesketh EA, Macpherson SG: The learning environment for junior doctor training--what hinders, what helps. Med Teach. 2005, 27: 619-624. 10.1080/01421590500069710.View ArticleGoogle Scholar
- Schwartzstein RM, Huang GC, Coughlin CM: Development and implementation of a comprehensive strategic plan for medical education at an academic medical center. Acad Med. 2008, 83: 550-559. 10.1097/ACM.0b013e3181722c7c.View ArticleGoogle Scholar
- Cross V, Hicks C, Parle J, Field S: Perceptions of the learning environment in higher specialist training of doctors: implications for recruitment and retention. Med Educ. 2006, 40: 121-128. 10.1111/j.1365-2929.2005.02382.x.View ArticleGoogle Scholar
- Genn JM: AMEE Medical Education Guide No. 23 (Part 2): Curriculum, environment, climate, quality and change in medical education - a unifying perspective. Med Teach. 2001, 23: 445-454.Google Scholar
- Roff S, McAleer S, Skinner A: Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach. 2005, 27: 326-331. 10.1080/01421590500150874.View ArticleGoogle Scholar
- Parsell G, Bligh J: Encouraging educational leadership. Med Educ. 2000, 34: 199-200. 10.1046/j.1365-2923.2000.0652a.x.View ArticleGoogle Scholar
- Schein EH: Organizational culture and leadership. 2004, San Francisco: Jossey-BassGoogle Scholar
- Schwartz RW, Pogge C: Physician leadership is essential to the survival of teaching hospitals. Am J Surg. 2000, 179: 462-468. 10.1016/S0002-9610(00)00395-0.View ArticleGoogle Scholar
- Souba WW: The new leader: new demands in a changing, turbulent environment. J Am Coll Surg. 2003, 197: 79-87. 10.1016/S1072-7515(03)00335-1.View ArticleGoogle Scholar
- Wong JG, Fagan M, Pinsker J: Expectations of and for the medical director of the resident's ambulatory clinic. Am J Med. 2001, 111: 84-87. 10.1016/S0002-9343(01)00764-1.View ArticleGoogle Scholar
- Danish Ministry of Health: Fremtidens Speciallæge (The future specialist). Danish. 1. 2000, Danish Ministry of Health, Ref Type: ReportGoogle Scholar
- Danish National Board of Health: Vejledning og evaluering i den lægelige videreuddannelse (Guidance and evaluation in the training of specialists in Denmark). Danish. 1998, DNBH Publications, 1Google Scholar
- Accreditation council for medical education (ACGME). Program director guide. 2009, 30-9-2009. Ref Type: Internet Communication, [http://www.acgme.org]
- The Royal College of Physicians and Surgeons in Canada (RCPSC). General standards applicable to the university and affiliated sites. 2007, 30-9-2009. Ref Type: Internet Communication, [http://www.rcpsc.medical.org]
- Malling B, Scherpbier AJ, Ringsted C: What is the role of the consultant responsible for postgraduate education in the clinical department?. Med Teach. 2007, 29: 471-477. 10.1080/01421590701510561.View ArticleGoogle Scholar
- Malling B, Bonderup T, Mortensen L, Ringsted C, Scherpbier A: Effects of multi-source feedback on developmental plans for leaders of postgraduate medical education. Med Educ. 2009, 43: 159-167. 10.1111/j.1365-2923.2008.03259.x.View ArticleGoogle Scholar
- Aspegren K, Bastholt L, Bested KM, Bonnesen T, Ejlersen E, Fog I, et al: Validation of the PHEEM instrument in a Danish hospital setting. Med Teach. 2007, 29: 498-500. 10.1080/01421590701477357.View ArticleGoogle Scholar
- Bracken DW, Timmreck CW, Church AH: The handbook of multi-source feedback. The comprehensive resource for designing and implementing MSF processes. 2001, San Francisco: Jossey-Bass, 1Google Scholar
- Leadership Qualities Framework, National Health Service (NHS), UK. 2009, Ref Type: Internet Communication, [http://www.nhsleadershipqualities.nhs.uk]
- Yukl GA: Leadership in organizations. 2006, Upper Saddle River, N.J.: Pearson Prentice Hall, 6Google Scholar
- Cassar K: Development of an instrument to measure the surgical operating theatre learning environment as perceived by basic surgical trainees. Med Teach. 2004, 26: 260-264. 10.1080/0142159042000191975.View ArticleGoogle Scholar
- Clapham M, Wall D, Batchelor A: Educational environment in intensive care medicine - use of Postgraduate Hospital Educational Environment Measure (PHEEM). Med Teach. 2007, 1-8.Google Scholar
- Boor K, Scheele F, van d, Scherpbier AJ, Teunissen PW, Sijtsma K: Psychometric properties of an instrument to measure the clinical learning environment. Med Educ. 2007, 41: 92-99. 10.1111/j.1365-2929.2006.02651.x.View ArticleGoogle Scholar
- Weiner BJ, Culbertson R, Jones RF, Dickler R: Organizational models for medical school-clinical enterprise relationships. Acad Med. 2001, 76: 113-124. 10.1097/00001888-200102000-00007.View ArticleGoogle Scholar
- Postgraduate medical education and training board. Strategy document: 2006-2010. 2009, 30-9-2009. Ref Type: Internet Communication, [http://www.gmc-uk.org/]
- Den Norske Legeforening. Specialist training for physicians in Norway. 2009, 30-9-2009. Ref Type: Internet Communication, [http://www.legeforeningen.no]
- Durak HI, Vatansever K, van DJ, van d V: Factors determining students' global satisfaction with clerkships: an analysis of a two year students' ratings database. Adv Health Sci Educ Theory Pract. 2008, 13: 495-502. 10.1007/s10459-007-9061-7.View ArticleGoogle Scholar
- van der Hem-Stokroos HH, Daelmans HE, van d, Haarman HJ, Scherpbier AJ: A qualitative study of constructive clinical learning experiences. Med Teach. 2003, 25: 120-126. 10.1080/0142159031000092481.View ArticleGoogle Scholar
- Teunissen PW, Scheele F, Scherpbier AJ, van d, Boor K, van Luijk SJ, et al: How residents learn: qualitative evidence for the pivotal role of clinical activities. Med Educ. 2007, 41: 763-770. 10.1111/j.1365-2923.2007.02778.x.View ArticleGoogle Scholar
- Malling B, Mortensen L, Bonderup T, Scherpbier AJJA, Ringsted C: Combining a leadership course and multi-source feedback has no effect on leadership skills of leaders in postgraduate medical education. An intervention study with a control group. BMC Medical Education. 2009, 9: 72-10.1186/1472-6920-9-72.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/10/62/prepub