Factors related to the role of programme directors in association with quality in postgraduate medical education – a cross-sectional study
BMC Medical Education volume 19, Article number: 462 (2019)
Educational leaders have been pointed out as being important for quality of medical education. However, their actual influence on the education can be limited. At the postgraduate level, educational leadership and its connection with quality is underexplored and knowledge about how to increase its impact is lacking. An increased understanding could be used in order to prioritize actions for strengthening the role. The aim of this study was to investigate factors related to the role of programme director associated with quality in postgraduate medical education.
A cross-sectional study was carried out. A questionnaire was sent to programme directors in Sweden (n = 519) comprising questions about background factors, work characteristics, work tasks, hindering and enabling factors, and the Utrecht Work Engagement Scale. A logistic regression and classification tree were used to identify factors associated with high qualitative education, defined as compliance with national regulations.
The response rate was 54% (n = 279). In total, 62% of the programme directors reported high quality and factors associated with high quality included experiences of communication with residents, superiors and supervisors, and support from the supervisors. Other factors were consensus regarding postgraduate medical education at the workplace, adequate financial resources, the programme directors’ competence, and their perceived impact on education. Factors of particular importance seemed to differ depending on whether the programme directors were responsible for one or for multiple units. Most high-quality education was found in cases where programme directors were responsible for a single unit and perceived sufficient impact on education.
These results indicated that there was an association between factors related to programme director and quality in postgraduate medical education. The findings pointed out the importance of combining activities at both individual, group and organizational levels. Relational aspects should not be underestimated; faculty development and involvement are crucial.
A goal for all those involved in postgraduate medical education (PGME) is to contribute to good patient care by providing an education with high quality. Quality in medical education is difficult to define, and thus, there is no consensus on the actual meaning [1, 2]. Quality generally has been defined as, for example, conformance to requirements , fitness for use , and meeting customers’ expectations . In the context of PGME, quality is often assessed by the fulfilment of different process components or standards , in line with the “conformance to requirements” above. The transition from a time- and process-based system to a competency-based framework also influences the view of educational quality  and another suitable definition may be “fitness for use” in the sense of producing competent physicians, which can be measured by different assessment methods . In this study, we have defined high quality as the fulfilment of national external regulations. This definition was chosen since it is included in the standards of The World Federation for Medical Education (WFME)  and the definition is also used in the Swedish system of quality assurance .
The standards of WFME point out the importance of a clearly stated role of professional leadership in PGME . In accordance with this, faculty for PGME usually includes a formal position responsible for specialist education. The conditions, responsibilities, and mandate for this role, here called the programme director (PD), varies [11,12,13]. In common is the need to handle educational issues in an environment where clinical activities sometimes set conflicting demands and education must be prioritized and negotiated in close collaboration with clinical activities. The role of the PD is often organized as a lower leadership role, a level that is characterized by focus upon co-operative and inspirational leadership and a possible lack of position power [14,15,16]. The relatively limited impact of leaders in medical education on the educational quality may be a problematic aspect of the role [17,18,19]. Some educational leaders in PGME are in a managerial position, which implies a greater legitimacy due to their formal leadership role.
Despite of the fact that PDs are identified as being important components of high qualitative PGME and the fact that their impact on education may be limited, knowledge about how to increase their impact is lacking. The aim of this study is to investigate which factors related to how PDs perceive their role are associated with high quality in PGME.
Context of the study
In Sweden, responsibility for providing health care is devolved to the county councils and, in some cases, municipal governments. Each county defines the needs per specialty and finances suitable caregivers to organize and implement physician’s specialist training, which can be offered by most hospital departments and health care centres for primary care. The admission and selection of residents is done locally, which means that there is no national admission. The National Board of Health and Welfare regulates the process of specialist training in Sweden, formulates national educational outcomes for each specialty and approves applications for specialist competence after completion of training . The regulations include a mandatory PD function. The PD’s area of responsibility can relate to one or several training units and the role is most often managed as a part-time duty alongside work as a consultant. Some departments have chosen to organize the role of the PD to include a managerial position vis-à-vis the residents. The total number of PDs in Sweden is unknown, since there is no central registration.
This study was carried out in Sweden and is part of a larger project being conducted, with a focus on the role of PD in PGME. The sample consisted of all persons registered in a voluntary list server of PDs provided by the Swedish Medical Association. Inclusion criteria were PDs with more than six months’ experience, a time limit we considered to be appropriate for having experience enough to be able to answer the questions, while not excluding persons having the assignment on an annual basis. PDs not working with specialist training, i.e. PDs at an internship level, were excluded.
The Swedish Medical Association contacted individuals who fulfilled the inclusion and exclusion criteria by e-mail, asking those not wishing to participate, as well as those who were no longer PDs, to contact them. The final sample consisted of 519 individuals. Participants were invited by e-mail during December 2016. In addition to a link to the questionnaire, information was given on the study, outlining the voluntary nature and the extent to which the data would be used and maintained confidentially. Non-responding participants received three reminder e-mails about the survey before the survey was concluded in February 2017.
The questionnaire was based on the results of two studies earlier in the project [11, 20] and on the Utrecht Work Engagement Scale (UWES) . The survey contained 30 questions divided into five distinct sections. First, the survey gathered information about (1) demographic and role-specific factors and high quality. The second section contained (2) questions about the quality of the PGME at the participants’ workplace. Subsequently, the survey gathered information on (3) work tasks. The participants were asked to indicate the extent to which they performed different tasks on a six-point scale. They were also asked to indicate how important they considered the task to be. The next section of the survey (4) sought to measure the experience of enabling and hindering factors influencing the PD’s performance of work tasks. Respondents were asked to indicate their level of agreement with each statement on a 5-point scale. To further validate the responses in this section, the respondents were asked to mention the three most important factors hindering their performance of work tasks. The answers were consistent with the outcome of the qualitative study on which the questionnaires were based (i.e. the obstacles mentioned in the free answers were the same as the alternatives in the questionnaire). Finally, the last section of the instrument was (5) work engagement measured using the UWES  in its Swedish version . The questionnaire was pilot tested on 13 persons, leading to minor changes in the final version.
The outcome was the quality of PGME, defined as fulfilling the national external regulations. Five paragraphs with mandatory regulations were chosen. These criteria are in accordance with the Accreditation Council for graduate medical education programme requirements .
All residents have individual training programmes that comply with the requirements in the description of objectives.
The individual training programme is regularly followed up and revised if needed, in consultation with the PD, supervisor, and resident.
The resident has access to supervisors during each training period.
The resident’s development is continuously assessed based on the description of objectives and their individual training programme throughout the whole specialist training.
The continuous assessment is made using known and agreed assessment methods.
The participants should indicate to what extent they considered the statements to be consistent with the PGME at their workplace on a four-point scale (1 = totally disagree, 2 = disagree, 3 = agree, 4 = totally agree). The outcome was dichotomized into high or low quality. High quality was defined as those people who assessed their own workplace as three or four on all five questions. Thus, low quality was one or two on at least one or more of the questions.
To study which factors were associated with high quality, the following overarching dimensions were chosen: demographic background, the prerequisites of the role (e.g. type of role, time), work tasks, hindering and enabling factors influencing the performance of work task and engagement (according to UWES). These factors were primarily related to contextual and role-specific aspects of the role which have been described in earlier studies [11, 20, 24]. However, none of the previous studies has examined the relationship between these factors and the quality of education. In a similar way as for the outcome, the work tasks and hindering and enabling factors were dichotomised. For the questions on work tasks, “yes” was defined as those people who answered 5–6 on the six-point scale (5 = a fairly large part, 6 = a large part), whereas “no” was defined as those who answered 1–4 (1 = never, 2 = a small part, 3 = a fairly small part, 4 = neither or). For the questions on enabling and hindering factors, “yes” was defined as those who answered 4–5 on the five-point scale (4 = agree to a high degree, 5 = totally agree), whereas “no” was those who answered 1–3 (1 = totally disagree, 2 = agrees to a low degree, 3 = partly agree).
All analysis was done in the Statistical Package for Social Sciences (SPSS version 23; SPSS Inc., Chicago, IL, USA). Study participant characteristics were analysed using descriptive statistics, i.e. frequencies, percentages, and mean values. We used logistic regression to study the relationship between high quality and the factors chosen.
Our model strategy was as follows: first, we studied the unadjusted association between high quality and each separate factor in univariable models. Second, we used a multivariable model to adjust for the six factors that we considered to be associated with high quality but that were not possible to influence: gender, years in practice, years as PD, medical specialty, the organizational level of the role and number of residents. We then added the other factors, one a time. Thus, the adjusted results could be interpreted as associations with high quality after adjustment for fixed factors. A p value of < 0.05 was regarded as statistically significant.
We also used classification tree to identify PDs that confirmed high quality and to identify factors associated with high quality. We used the CHAID algorithm to build the tree  including all variables in Tables 1, 2, 3 and 4. A CHAID analysis starts with all the data in one group. Each possible split on each independent variable is considered, in order to find the split that leads to the strongest association with high quality. Tree depth was limited to five levels, minimum parent node was set to 30 and minimum child node was 20. No split with Bonferroni adjustment of less than 0.05 was executed.
The participants were informed in an e-mail about the study’s aims and design, where it was also stated that the results would only be used for research purposes and that participation was voluntary. Ethical approval for this study was applied for from the regional Ethical Review Board (dnr 2012/1662–31/5).
Demographic and role-specific factors and high quality
Completed questionnaires were received from 279 PDs representing an overall response rate of 54% (279/519). All Swedish geographic regions were represented as well as all specialty groups, and 66% (183/279) of PDs were female and 56% male (96/279). Their experience as a PD varied between 6 months and 20 years, with an average of 7 years.
In all, 62% (174/279) of the PDs reported high quality on PGME and the percentage varied between different PD roles (Table 1).
The demographic and role-specific factors associated with high quality in the adjusted analysis were medical speciality and number of residents (Table 1).
The most commonly reported work tasks were to organize and plan the process of PGME at the workplace (75%, 209/279) followed by making PGME visible and highly valued (59%, 166/279). The tasks least reported were handling conflicts and disagreements (12%, 33/279), discussions and negotiation between different residents in the department (15%, 42/279), and to support and supervise the supervisors (22%, 61/279). The work task associated with high quality after adjustment was to discuss and negotiate between different residents (Table 2).
Enabling and hindering factors
The respondents were asked to mark their level of agreement with different statements connected to enabling and hindering factors. The factors most respondents agreed on were sufficient own skills (reported by 68%, 186/275), and the presence of adequate rules and guidelines (64%, 177/275). The statements that few respondents agreed on were sufficient communication with supervisors (27%, 74/272), sufficient support from supervisors (37%, 98/266), and having sufficient possibility to influence the education (38%, 104/277).
Factors associated with high quality that were felt to hinder or enable the PD in their role were: sufficient communication with residents, sufficient communication with superiors, sufficient communication with supervisors, sufficient support from the supervisors, sufficient opportunities to influence the education, sufficient own skills, consensus about PGME at the workplace, and adequate financial resources. Associations remained after adjustment for factors linked to the PD (Table 3).
According to UWES, 55% (154/279) reported an average work engagement vis-à-vis their role as a PD, 30% (84/279) high, and 15% (41/279) low work engagement. Quality of PGME was not associated with the PD’s work engagement (Table 4).
The classification according to the classification tree analysis showed that the rate of high qualitative PGME ranged between 33 and 90% within different groups (Fig. 1), compared with an overall proportion of high quality of 62%. The PDs with the highest proportion of high quality were PDs in a single unit with sufficient possibilities to influence the PGME (90%, 47/52 high quality). In contrast, the group with the lowest occurrence of high quality (33%, 17/52) consisted of PDs responsible for several units who felt there was no consensus around PGME in the organization. The classification tree analysis also indicated that the most important factor associated with high quality was the type of PD role. For PDs for several units, consensus around PGME was the factor strongest associated with high quality. That is, in workplaces with consensus on what characterized high educational quality, the quality of the PGME was higher. For PDs at a single unit (also including PDs with a managerial position) on the other hand, the experience of being able to influence the postgraduate medical education at the workplace was the most important factor associated to high quality.
The result showed that factors associated with educational high quality included relational aspects (the perceived quality of the communication with residents, superiors and supervisors and support from the supervisors), structural aspects (adequate financial resources, medical speciality and number of residents), individual aspects (the PD’s own skills), and attitudinal aspects (consensus about PGME at the workplace). Also, the PD’s perceptions of their possibility of influencing the education, for example to plan and implement improvements, was associated with high quality, a factor that likely is influenced by the other factors. According to the multi-frame leadership theory of Bolman and Deal’s [26, 27] four essential perspective should be utilised for effective leadership: structural, human resource, political and symbolic. Since PGME is controlled by various formal requirements and policies, and creating formal structures for the training is a responsibility that is often mentioned [11, 12, 23] there is a possible consequence that PDs may focus on the structural perspective. Our findings point to the importance of the human resource perspective of the PD role which therefore focuses on people emphasising support, staff development and communication. Communication is also a tool according to the political perspective, an approach that emphasises negotiation and building power bases when there are competing interests within an organization. The importance of these personal power sources of the PD role is in line with earlier research on power for leaders without formal authority , and it has also earlier been argued that informal aspects have greater importance than formal aspects in workplace learning .
Clinical supervision is one of the key components of the residents’ learning, which, however, sometimes is lacking in quality [29, 30]. Still, only a minority of the PDs in our study reported that communication with the supervisors was sufficient, nor was support from the supervisors sufficient. Collaboration within the faculty is an important part of the educational leadership in medical education [31, 32]. Based on our results, communication and collaboration between PDs and other key groups within PGME should be given priority in order to create a community of practice  around the educational domain. This could be strengthened by educational activities for PDs in combination with creating practical communication opportunities by prioritizing time and creating routines for how and when communication and collaboration will take place.
Many of the factors affecting clinical learning is culturally embedded . A high qualitative PGME requires a framework in which education is not seen as a burden but as having a production value. Our results suggest that a workplace culture which supports educational activities by prioritizing financial and time resources, and has a clear consensus on how that education should be delivered, will strengthen the role of the director and their ability to contribute to high quality education.
This study has some limitations. Firstly, the sample was based on PDs in a voluntary database and the response rate was moderate. As there is no central registration of PDs in the Swedish context, the use of the voluntary database was the only opportunity of reaching the target group. As a result of being voluntary, the database used was not updated; many people only have a PD assignment for a shorter period, and then remain on the list, which may be an explanation. If this dropout resulted in a selection bias that might have affected the result is unknown. However, by controlling the spread of respondents in terms of geographical location, medical specialties, and other background variables, it was found that the respondents had a distribution likely to be representative of the whole population of PDs.
Secondly, this study was limited to PDs’ subjective perceptions. It could be argued that alternative sources of information might have offered a more objective view. However, in this study the use of programme PDs’ perceptions was inevitable because we wanted to explore the association between how the role is experienced by PDs and the educational quality, which involves internal processes and, as a consequence, cannot be assessed objectively. Concerning the validity of the measure of educational quality, the ideal measurement should be outcome of patient care, which obviously is associated with many challenges. As described in the introduction, we used the “conformance to requirements” definition, which means that quality has been operationalized to fulfil a number of formal requirements. This was considered to be most appropriate to be able to replicate the study in other countries. One weakness in the quality measure is the interpretable formulation of the regulations on which the questions about quality rely on (e.g “The resident’s development should be continuously assessed ( …)” ) which could make the inter-rater reliability low. The variation for an individual is probably lower, indicating a high test-retest reliability.
Finally, educational leadership in PGME differs around the world, both in terms of educational culture and formal structures. The generalisability of the study outside Sweden depends on how the PD role is organized, and the result should be read with the contextual background in consideration. Thus, the external validity outside Sweden is unknown and should be investigated further.
These results indicate that there was an association between factors related to the PD role and quality in postgraduate medical education. The findings pointed out the importance of combining activities at both individual, group and organizational level in order to strengthen the PDs’s positive impact on the educational quality. Relational aspects should not be underestimated and the role and faculty development and involvement are crucial.
We make three recommendations to enhance the quality of education:
Create forums that enable communication, collaboration, and peer-to-peer learning within the faculty.
PDs should participate in educational activities designed to develop leadership and communication skills.
Development of the PD role must be done in parallel with the organization prioritizing educational activities at a more general level.
Availability of data and materials
The datasets used during the current study are available from the corresponding author on reasonable request.
Postgraduate medical education
Utrecht Work Engagement Scale
World Federation of Medical Education
Bleakley A, Browne J, Ellis K. Quality in medical education. In Understanding medical education – evidence, theory and practice. 2. Oxford: Wiley-Blackwell; 2013. p. 47–60.
Laura S, Sarah P-E, Heather W, Linda C. Definitions of quality in higher education: A synthesis of the literature. High Learn Res Commun. 2015;5(3):3–13.
Crosby PB. Quality is free : the art of making quality certain. New York: McGraw-Hill; 1979.
Juran JM, Godfrey AB. Juran's quality handbook. New York: McGraw Hill; 2000.
Deming WE. Out of the crisis : quality, productivity and competitive position. Cambridge: Cambridge Univ. Press; 1988.
Corrigan O, Ellis K, Bleakley A, Brice J. Quality in medical education. In: Swanwick T, editor. Understanding Medical Education - Evidence, Theory and Practice Wiley-Blackwell; 2010. p. 379–90.
Ten Cate O. Competency-Based Postgraduate Medical Education: Past, Present and Future. GMS J Med Educ. 2017;34(5):Doc69.
Bailey D. Ensuring quality in postgraduate medical education: competency testing is the key. Virchows Arch. 2016;468(1):115–9.
World federation for medical education. WFME Global standards for Quality Improvement. Copenhagen, 2003.
Socialstyrelsens föreskrifter och allmänna råd om läkares specialiseringstjänstgöring, SOSFS 2015:8 (2015).
Fryden H, Ponzer S, Heikkila K, Kihlstrom L, Nordquist J. Roles, tasks and educational functions of postgraduate programme directors: a qualitative study. Postgrad Med J. 2015;91(1080):588–93.
Malling B, Scherpbier AJ, Ringsted C. What is the role of the consultant responsible for postgraduate education in the clinical department? Medical teacher. 2007;29(5):471–7.
Lieff S, Albert M. What do we do? Practices and learning strategies of medical education leaders. Med Teach. 2012;34(4):312–9.
DeChurch LA, Hiller NJ, Murase T, Doty D, Salas E. Leadership across levels: levels of leaders and their levels of impact. Leadersh Q. 2010;21(6):1069–85.
Bass BM, Bass R. The Bass handbook of leadership : theory, research and managerial applications. New York: Free Press; 2008.
Yukl G, Falbe CM. Importance of different power sources in downward and lateral relations. J Appl Psychol. 1991;76(3):416–23.
Sundberg K, Josephson A, Reeves S, Nordquist J. Power and resistance: leading change in medical education. Stud High Educ. 2017;42(3):445–62.
Malling B, Mortensen LS, Scherpbier AJ, Ringsted C. Educational climate seems unrelated to leadership skills of clinical consultants responsible of postgraduate medical education in clinical departments. BMC Med Educ. 2010;10:62.
Sundberg K, Josephson A, Reeves S, Nordquist J. May I see your ID, please? An explorative study of the professional identity of undergraduate medical education leaders. BMC Med Educ. 2017;17(1):29.
Wijk H, Ponzer S, Heikkila K, Kihlstrom L, Nordquist J. Factors influencing effectiveness in postgraduate medical education – a qualitative study of experiences of the responsible clinical consultants. BMC Med Educ. 2019;19:3. https://doi.org/10.1186/s12909-018-1433-6.
Schaufeli W, Salanova M, González-romá V, Bakker A. The measurement of engagement and burnout: a two sample confirmatory factor analytic approach. J Happiness Stud. 2002;3(1):71–92.
Hallberg UE, Schaufeli WB. “Same same” but different? Can work engagement be discriminated from job involvement and organizational commitment? Eur Psychol. 2006;11:119–27.
Common Program Requirements; effective July 1 , 2017. Accreditation Council for Graduate Medical Education Web site. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf. Accessed 22 Nov 2018.
Long TR, Brown MJ, Elliott BA, Rose SH. Characteristics of anesthesiology residency program directors. J Clin Anesth. 2010;22(8):583–6.
Kass G. An exploratory technique for investigating large quantities of categorical data. Appl Stat. 1980;29(2):119–27.
Bolman LG, Deal TE. Leadership and management effectiveness - a mulitframe, multisector analysis. Hum Resour Manag. 1991;30(4):509–34.
Bolman LG, Deal TE. Reframing organizations : artistry, choice, and leadership. San Francisco: Jossey-Bass; 2013.
Kilty C, Wiese A, Bergin C, Flood P, Fu N, Horgan M, et al. A national stakeholder consensus study of challenges and priorities for clinical learning environments in postgraduate medical education. BMC Med Educ. 2017;17(1):226.
Kilminster S, Cottrell D, Grant J, Jolly B. AMEE guide no. 27: effective educational and clinical supervision. Med Teach. 2007;29(1):2–19.
Kilminster SM, Jolly BC. Effective supervision in clinical practice settings: a literature review. Med Educ. 2000;34(10):827–40.
Lieff SJ, Zaretsky A, Bandiera G, Imrie K, Spadafora S, Glover TS. What do I do? Developing a competency inventory for postgraduate (residency) program directors. Med Teach. 2016;38(10):1011–6.
Bing-You RG, Holmboe E, Varaklis K, Linder J. Is it time for Entrustable professional activities for residency program directors? Acad Med. 2017;92(6):739–42.
Wenger E. Communities of practice : learning, meaning, and identity. Cambridge: Cambridge University Press; 1998.
Open access funding provided by Karolinska Institute.
The study is an integrated part of a larger PhD project which has been financed by the Department of Research and Education, Karolinska University Hospital, Stockholm, Sweden. The funding body played no role in the design of the study and collection, analysis, interpretation of data or in writing the manuscript.
Ethics approval and consent to participate
Ethical approval for this study was applied for at the regional Ethical Review Board in Stockholm with the decision that it was not applicable (dnr 2012/1662–31/5). Informed written consent was obtained from all participants before taking part in survey.
Consent for publication
We have read and understood Bio Med Centrals policy on declaration of interests and declare that we have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Wijk, H., Ponzer, S., Järnbert-Pettersson, H. et al. Factors related to the role of programme directors in association with quality in postgraduate medical education – a cross-sectional study. BMC Med Educ 19, 462 (2019). https://doi.org/10.1186/s12909-019-1885-3
- Postgraduate medical education
- Professional development
- Educational leadership
- Medical education