- Research article
- Open Access
Identification of capacity development indicators for faculty development programs: A nominal group technique study
BMC Medical Education volume 20, Article number: 163 (2020)
Although there have been many research studies of the effectiveness of faculty development in health profession education, the contribution of these programs to organizational development through capacity development has not been studied. Further understanding of capacity development requires appropriate indicators and no previous indicators for faculty development of health profession educators were identified. The aim of the study was to identify indicators of capacity development in the context of faculty development programs at Tehran University of medical sciences (TUMS).
A nominal group technique session was conducted with key informants from faculty development program providers to generate and prioritize a list of capacity development indicators.
A list of 26 indicators was generated and five categories were identified: Development and innovation in teaching and learning process, Development and innovation in communication and collaboration at different levels, Development and sustaining faculty development programs, Development of educational leadership and management, Development in scholarship.
Capacity development for faculty development interventions of health profession educators is a process of engagement within a wider system, including individual and collective action, and involves the socialization of the teachers into suitable roles through professional identity development and participation within the wider system.
Changing concepts of education and the increasing complexity of healthcare have led many medical universities to design and implement a variety of faculty development programs in order to help health profession educators to effectively perform their multiple roles which include teaching, leadership and production of resources . Faculty development involves all of the activities that are designed to help health profession educators to improve theknowledge, skills and behaviors that are required for their different roles, both as an individual and within organizational settings .
Faculty development is essential for any medical university, with the development of a basic collection of competencies in faculty members that enable them to cope with their workload and environment changes so that they can adequately perform their critical functions . Faculty development can also be a useful approach for the advancement of organizational development by producing a sense of support, inspiration and energy, carrying out change and creativity, improving organizational capabilities, and creating a future workforce of skilled educators [4, 5].
In spite of the potential contribution of faculty development programs to organizational development, studies tend to focus on only measuring the outcomes at an individual level. Most studies have focused on the evaluation of participant satisfaction immediately after they attended a program [6,7,8]. Some studies have explored learning, which included attitudinal changes and increases in knowledge or skills [9,10,11]. A few studies have assessed the changes in educational behavior of the participants in their multiple roles [12, 13]. Most of these evaluations have also only had a focus on short term outcomes, especially at an individual level.
Policy makers and funding agencies are increasing their demands for evaluating change at a much broader level, seeking evidence of impact for potential sustainability of the faculty development programs by wider involvement of the participants in their sphere of work and at the organizational level . However, faculty development programs occur within rapidly changing organizations in which people work and the impact of these programs is not linear . The simple linear program evaluation models consider that the acquisition of new knowledge and skills will produce organizational change but these models usually ignore the context, with its complexity of factors that influence the outcomes [16, 17].
The concept of capacity development has been expanded to recognize the importance of development as a system-wide process and has increasingly been used to inform an understanding of the complexity of how the organizational impact of human development programs can be achieved [18, 19]. Capacity development has a focus on how changes in human behavior, such as the growth of new attitudes, values, knowledge, skills, and relations hips with others, are generated over time within a complex organizational system . This generative process leads to the enhanced capability of organizations to perform within the complex environment. An essential feature of this process is that the development of individuals become embedded in a collective manner through their relationships with others . A recent study of a faculty development program in South Africa appears to be the first to demonstrate the important features of the capacity development process in health profession education, with collaborative knowledge sharing and support across to produce individual personal and professional development but also change in the organizations in which they worked . Capacity development is essential for the sustainability of faculty development programs, with the evolution in the professional identity and empowerment of faculty members to become change agents who can shape a wider impact in their workplace . The professional identity development occurs at both individual and collective levels through the relationships with others in the workplace and the broader context of the workplace [22,23,24]. This collective ability through wider sharing is a key characteristic of capacity development and enables the organization to effectively cope with the complexity of environmental changes [2, 25]. Evaluation of the broader capacity development impact of faculty development programs for health profession education requires indicators that can be used to inform evaluations. No previous published research about the indicators of capacity development of faculty development programs for health profession educators were identified.
The aim of this study is to identify the indicators of capacity development in the context of faculty development programs at Tehran University of medical sciences (TUMS). The TUMS Education Development Centre introduced a comprehensive faculty development program in 2015 with the aim of improving the quality of teaching and learning, to engage individuals in collective education development actions, and to promote student learning. A variety of faculty development activities were designed and implemented to address the needs of health profession educators throughout the university. These activities were in different formats, including workshops and seminars, short courses and fellowships, and covered a variety of topics in medical education. One of these programs, the “Basic Teaching Skills Course”, covers essential subjects for teaching effectiveness such as instructional design, teaching methods and, student assessment. This is a longitudinal course designed for new or less experienced faculty members. Participating in this course is essential for all assistant professors for promotion to associate professor. It is delivered in an interactive format with lectures, group work, and assignments and feedbacks. Faculty members from all eleven schools of TUMS are the participants in this program. We chose to study the “Basic Teaching Skills Course” since it has a large number of participants and covers the basic elements of teaching and learning. Hence, we expected that networking and both individual and collective development would occur.
We used nominal group technique (NGT) to elicit the indicators of capacity development for health profession education in the context of the “Basic Teaching Skills Course” at TUMS in December, 2017. The NGT method was chosen because it promotes the active participation of all group members to produce new ideas through a brain storming format. We followed a classic NGT technique including 1) silent generation of responses to a particular question, 2) round-robin sharing and recording of ideas, 3) group discussion for clarification, and 4) voting on items, with respect to importance, or relatedness .
Nineteen key informants were invited via email and, then invitations were confirmed in person. The informants were recruited based on their long-standing association with the TUMS Education Development Center and all had previous experience in the management and administration of faculty development programs, some in the “Basic Teaching Skills Course”. The TUMS’s institutional review board approved the study, participants did not receive any incentives, and participation was voluntary. Verbal consent for participation was obtained based on the proposal approved by the ethics committee.
The NGT meeting was facilitated and lasted 3 hours. In the first step of the NGT, we gave participants a brief description of the NGT process, details of the capacity development concept as a process and the need to have capacity development indicators. We clarified that the indicators could identify the growth and development of attitudes, values, behaviors, abilities, motivations, and identity, and could also help to identify changes in persons, and organizations, as they try to develop their performance. We also presented some examples which would not be appropriate as capacity development indicators, such as an increase in the number of trained faculty members, increase in the number of publications, or increase in adopting educational leadership roles. We then provided participants with some examples of capacity development indicators that have been used to evaluate personnel development in various settings such as energy and environmental programs, welfare programs and community management. The aim of providing examples was to clarify an abstract concept such as capacity development. These examples were only provided as a short initial Power Point presentation, with the intention to raise awareness and to avoid participants simply replicating the examples.
Participants were asked the following questions: “what are the indicators of capacity development in the context of the Basic Teaching Skills Course?” and “how may participants in Basic Teaching Skills Course contribute to organizational development (in education)?” and we asked each participant to independently and privately write down individual notes what he or she considered as the indicators of capacity development of the “Basic Teaching Skills Course”. During this phase of the NGT process, the participants worked alone and were not permitted to talk to others in the room or to ask for clarification of the questions. The indicators were then shared with the group in a round-robin format, with each member sharing one item from their list, without naming any previously mentioned items, until all items had been exhausted. All the named indictors were typed by a facilitator in full view of all the others. During this phase of the process, no questions were allowed.
In the next phase, with the assistance of the facilitator, participants discussed the list of indicators, resolved questions of clarity and combined related indicators into a single item and clarification was provided. In cases of dispute, the person who raised the indicator could decide whether or not to combine it with another item. Similar indicators were grouped through group discussion and a label was selected for each category. During the next phase, we followed the nominal group technique and asked participants to vote privately on the indicators and rank the indicators on a Likert scale, where 1 was “completely not related” to capacity development of faculty development and 5 was “completely related”.
Finally, voting results were summed across participants and mean score of votes for each indicator was calculated to determine the most related indicators of capacity development of faculty development programs for health profession educators. For this study, we only determined the related indicators with a mean score greater than or equal to 3.50.
Nine key informants agreed to participate in the NGT. The majority of them (66.7%) were women. Almost half (44.4%) were assistant professors, 33.3% were associate professors, 11.1% was professor and the rest were instructors.
The initial list of capacity development indicators of the “Basic Teaching Skills Course” consisted of 88 items organized in six categories. After the voting step, the list of indicators was reduced to 26 indicators in five categories. These five categories were: Development and innovation in teaching and learning process, Development and innovation in communication and collaborations at different levels, Development and sustaining faculty development programs, Development of educational leadership and management, Development in scholarship.
The development and innovation in teaching and learning process category has a focus on developing competencies in the teaching and learning process, including various teaching and student assessment methods. Development and innovation in communication and collaboration at different levels includes networking and the relationships within the community of medical colleagues. The category of development and sustaining faculty development programs represents the interest of teachers in medical education and their support and collaboration with colleagues, which is essential to sustain and develop the programs. Development of educational leadership and management category refers to involvement in the development, implementation and evaluation of the medical education institution. Development in scholarship represents educational research and scholarship, including involvement in activities that share new knowledge. For descriptive purposes, the capacity development indicators of faculty development programs are presented within their assigned category in Table 1.
The most important capacity development indicators that were identified were within the development and innovation in teaching and learning process category, including enhanced competencies to transfer concepts and skills to learners, improved competencies to manage the classroom and competencies to apply novel methods for assessing learners (mean = 4.83). In the development and innovation in communication and collaboration at different levels category, the most important indicator of capacity development was competencies to communicate with learners, colleagues and patients appropriately (mean = 4.50). The highest score for a capacity development indicator in the development and sustaining faculty development programs category was refer to specialized evidence or consult with experts when answering a question or to inform decision making in the field of medical education (mean = 4.33). In the development of educational leadership and management category, the main capacity development indicator was to cooperate in the implementation of educational development processes at university/school (mean = 4.50). The most important capacity development indicator for development in scholarship category included motivation to identify educational problems, and to design and implement appropriate interventions (mean = 4.5).
Our study is the first, to our knowledge, to identify the indicators of capacity development for faculty development programs in health profession education. The study identified five categories of capacity development indicators that were considered by key faculty members to be important outcomes for the “Basic Teaching Skills Course” at TUMS.
We found that the “Development and innovation in teaching and learning process” was one of the main categories of capacity development indicators for faculty development programs. This is consistent with the aims and expected learning outcomes of most faculty development programs. An important outcome of these programs is that participants begin to thoughtfully apply their new knowledge and skills to modify their teaching by increasing their use of a variety of educational methods to influence curriculum development, including course design and organization . Evans et al (2013) described and reflected on their experience of running a 2 years nursing faculty development program and noted that developing innovations in teaching and learning was one of the main outcomes . In our study, enhanced competencies to transfer concepts and skills to learners were prioritized highly, which is supported by the importance of the ‘information provider’ role of medical teachers . Because of the importance of this role, most faculty development programs aim to help teachers to become more effective in this role [8, 30,31,32,33]. The importance of improved competencies to manage the classroom has been previously noted, . One of the aims of faculty development programs, especially new instructors, is to help them to find solutions to their classroom problems and experiment management processes consistent with their own education values and those of their employing universities [35, 36].
The “development and innovation in communication and collaboration at different levels” was identified as another important category representative of capacity development of our faculty development programs. This finding has been noted in two previous studies that had a focus on capacity development. Frantz et al (2015) investigated the perceptions of the personal and professional impact of the participants of a faculty development program in sub-Saharan Africa . Participants reported that the program had given them the opportunity and skills to communicate and engage with other colleagues and education interest groups. Another study by Frantz et al (2019) investigated the contribution of a faculty development program to individual and collective capacity development in Africa. One of their emergent themes was networking which was identified as providing both critical personal support and engagement within a wider professional community of practice . Despite the importance of this category, two of the identified indicators (“help new colleague for career progression" and "competences to do teamwork") had low scores. This may be because of cultural barriers, lack of appropriate opportunities for communication, and conflict of interests . Previous studies have noted that teamwork and networking is less valued in some developing countries . To fulfill TUMS comprehensive program, faculty development programs that foster the creation of mutual and common values, beliefs and norms among the participants within a safe and supportive environment encourage cooperation and networking .
In the “Development and sustaining faculty development programs” category most of the capacity development indicators that were identified are consistent with the impact of faculty development programs which have been reported by prior studies. However, we found some new and important indicators in relation to the increased motivation and effort of participants of these programs to become familiar with medical education. This finding highlights the high commitment that is demonstrated by many participants on faculty development programs for health profession education.
The “cooperate in the implementation of educational development processes at university/school” category, is identified as being an important indicator in the “Development of educational leadership and management” category. This finding emphasizes the importance of preparing health profession educators for educational leadership roles and responsibilities . Our findings are consistent with Schreurs et al (2016) work that reported that faculty development programs increased understanding of educational organizations and their policies and also enhanced confidence to implement educational development process, which generated improvement in the educational culture of the organization . Participants also considered that “Development in scholarship” was an important category, with motivation to identify educational problems, and to design and implement appropriate interventions. Frantz et al (2019) identified educational research and scholarship among their themes about the contribution of a faculty development program to individual and collective capacity development in Africa. They reported that the development of the participants’ individual knowledge and skills in research and scholarship led to the individual and collective advancement of scholarly educational activities .
An interesting finding of our study is the high agreement about the importance of collective agency of the teachers in the organization in which they work, with the faculty development program enhancing the motivation and confidence of participants to take collective action for organizational change. This capacity development outcome requires active engagement in the wider sharing of acquired competencies and the co-development of capabilities at a system level. Our findings are consistent with the main aspects of capacity development, which highlights the collective process of interactions in a wider system to address problems and bring about transformational change within a specific context . The development of agency is closely aligned with the evolution of professional identity as a health profession educator and their empowerment to produce organizational change in complex environment .
The findings of our study are important, they highlight that the identified indicators of capacity development in the context of faculty development programs for health profession education at TUMS consider not only the impact on individuals but also within the wider TUMS organization. The need for a broader approach to program evaluation that is beyond individual perceptions, acquisition of new learning and changes in behavior is required for future evaluation of faculty development programs. Also, a systems level perspective is essential to expand future faculty development curricula from only teaching competencies to demonstrating communication, leadership and management skills with the purpose of strengthening teachers’ capacity to become organizational change agents .
Producing a list of indicators can be a vital part of the capacity development process itself. The initial identification and further refinement of the indicators, such as by the use of NGT, is a collective process and both the process and outcome of producing the indicators can lead to organizational change . An important aspect of the collective process is that a shared consensus about the future direction of a faculty development program can be achieved, which is essential for defining clear objectives of the program, determines the way of monitoring and evaluation and providing resources to support a program .
Some limitations of this research should be considered. The study was performed in one organization with a limited number of key informants and represents the indicators of capacity development of only one type of faculty development program; therefore, findings may not be generalizable to other context. However, recruiting participants at a variety stages of profession created a rational list of indicators related to capacity development of faculty development programs for our institution. Further research done at other universities is recommended to determine if there are other significant indicators missed in this pilot study and if these findings generalize over one institution. Collaborative research across universities could lead to the refinement of a list of common indicators that could be used to evaluate the capacity development of faculty development programs for health profession education. The participants were also provided with examples of capacity development indicators before they started to generate indicators, which may have influenced the participants’ answering options. Finally, within the existing relatively limited literature on capacity development in the context of faculty development, there is a noticeable paucity of this type of research in the health profession education context and future research is recommended to understand the process of capacity development.
Identification of the indicators of capacity development in the context of faculty development programs for health profession education provides insight into the impact of these programs on organizational development. The findings highlight the importance of understanding, implementing and evaluating faculty development programs from a capacity development perspective that considers both the development of individual and collective abilities to positively contribute to change in the wider health profession education system.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Tehran University of Medical Sciences
Nominal Group Technique
Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M, Prideaux D. A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8. Med Teacher. 2006;28(6):497–526.
Steinert Y, Mann K, Anderson B, Barnett BM, Centeno A, Naismith L, Prideaux D, Spencer J, Tullo E, Viggiano T, Ward H. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: A 10-year update: BEME Guide No. 40. Med Teacher. 2016;38(8):769–86.
Morgan P. The concept of capacity. European Centre Development Policy Management. 2006:1–9.
Bligh J. Faculty development. Med Educ. 2005;39(2):120–1.
Swanwick T. See one, do one, then what? Faculty development in postgraduate medical education. Postgrad Med J. 2008;84(993):339–43.
Sarikaya O, Kalaca S, Yeğen BÇ, Cali S. The impact of a faculty development program: evaluation based on the self-assessment of medical educators from preclinical and clinical disciplines. Advances Physiol Educ. 2010;34(2):35–40.
Moore P, Montero L, Triviño X, Sirhan M, Leiva L. Impact beyond the objectives: a qualitative study of a faculty development program in medical education. Revista medica de Chile. 2014;142(3):336–43.
Abu-Rish Blakeney E, Pfeifle A, Jones M, Hall LW, K. Zierler B. Findings from a mixed-methods study of an interprofessional faculty development program. J Interprofessional Care. 2016;30(1):83–9.
Hewson MG, Copeland HL, Fishleder AJ. What's the use of faculty development? Program evaluation using retrospective self-assessments and independent performance ratings. Teaching Learning Med. 2001;13(3):153–60.
Tax CL, Doucette H, Neish NR, Maillet JP. A model for cultivating dental hygiene faculty development within a community of practice. J Dental Educ. 2012;76(3):311–21.
Saiki T, Imafuku R, Pickering J, Suzuki Y, Steinert Y. On-site observational learning in faculty development: impact of an international program on clinical teaching in medicine. J Continuing Educ Health Professions. 2019;39(2):144–51.
Behar-Horenstein LS, Childs GS, Graff RA. Observation and assessment of faculty development learning outcomes. J Dental Educ. 2010;74(11):1245–54.
Lee SS, Dong C, Yeo SP, Gwee MC, Samarasekera DD. Impact of faculty development programs for positive behavioural changes among teachers: a case study. Korean J Med Educ. 2018;30(1):11.
Jolly B. Faculty development for organizational change. In: Steinert Y, editor. Faculty development in the health professions: a focus on research and practice. Dordrecht: Springer; 2014. p. 119–37.
Tan AL, Chang CH, Teng P. Tensions and dilemmas in teacher professional development. Procedia Social Behav Sci. 2015;174:1583–91.
Sandars J, Brown J, Walsh K. Producing useful evaluations in medical education. Educ Primary Care. 2017;28(3):137–40.
Fernandez N, Audétat MC. Faculty development program evaluation: a need to embrace complexity. Advances Med Educ Pract. 2019;10:191.
Morgan P The idea and practice of systems thinking and their relevance for capacity development. Maastricht: European Centre for Development Policy Management. 2005.
Brinkerhoff DW, Morgan PJ. Capacity and capacity development: Coping with complexity. Public Administration Development. 2010;30(1):2–10.
Morgan P The design and use of capacity development indicators. Political and social policies division, policy branch, CIDA. Hull: Canadian International Development Agency (CIDA). 1997.
Frantz J, Rhoda A, Sandars J, Murdoch-Eaton DB, Marshall M, Burch VC. Understanding faculty development as capacity development: A case study from South Africa. African J Health Professions Educ. 2019;11(2):53–6.
Starr S, Ferguson WJ, Haley HL, Quirk M. Community preceptors' views of their identities as teachers. Academic Med. 2003;78(8):820–5.
Starr S, Haley HL, Mazor KM, Ferguson W, Philbin M, Quirk M. Initial testing of an instrument to measure teacher identity in physicians. Teaching Learning Medicine. 2006;18(2):117–25.
Monrouxe LV. Identity, identification and medical education: why should we care? Medical Education. 2010;44(1):40–9.
Steinert Y, Cruess S, Cruess R, Snell L. Faculty development for teaching and evaluating professionalism: from programme design to curriculum change. Medical Education. 2005;39(2):127–36.
Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Medical Teacher. 2017;39(1):14–9.
McLean M, Cilliers F, Van Wyk JM. Faculty development: yesterday, today and tomorrow. Medical Teacher. 2008;30(6):555–84.
Evans C, Razia R, Cook E. Building nurse education capacity in India: insights from a faculty development programme in Andhra Pradesh. BMC Nursing. 2013;12(1):8.
Harden RM, Crosby JO. AMEE Guide No 20: The good teacher is more than a lecturerÐthe twelve roles of the teacher. Medical Teacher. 2000;22(4):334–47.
Hitchcock MA, Mylona ZH. Teaching faculty to conduct problem-based learning. Teaching Learning Medicine. 2000;12(1):52–7.
Cole KA, Barker LR, Kolodner K, Williamson P, Wright SM, Kern DE. Faculty development in teaching skills: an intensive longitudinal model. Academic Medicine. 2004;79(5):469–80.
Pololi LH, Frankel RM. Humanising medical education through faculty development: linking self-awareness and teaching skills. Medical Education. 2005;39(2):154–62.
Lown BA, Newman LR, Hatem CJ. The personal and professional impact of a fellowship in medical education. Academic Medicine. 2009;84(8):1089–97.
Al-Shammari ZN. Enhancing higher education student attendance through classroom management. Cogent Education. 2016;3(1):1210488.
Wubbels T. An international perspective on classroom management: What should prospective teachers learn? Teaching Education. 2011;22(2):113–31.
Ragnemalm EL, Samuelsson M. Simulating variation in order to learn classroom management. Educational Media International. 2016;53(4):274–84.
Frantz JM, Bezuidenhout J, Burch VC, Mthembu S, Rowe M, Tan C, Van Wyk J, Van Heerden B. The impact of a faculty development programme for health professions educators in sub-Saharan Africa: an archival study. BMC Medical Education. 2015;15(1):28.
Amorim Neto RD, Picanço Rodrigues V, Campbell K, Polega M, Ochsankehl T. Teamwork and Entrepreneurial Behavior among K-12 Teachers in the United States. InThe Educational Forum 2020 Jan 25 (pp. 1-15). Routledge.
Collins N, Chou YM, Warner M, Rowley C. Human factors in East Asian virtual teamwork: a comparative study of Indonesia, Taiwan and Vietnam. International J Human Resource Management. 2017;28(10):1475–98.
Lewis LD, Steinert Y. How Culture Is Understood in Faculty Development in the Health Professions: A Scoping Review. Academic Medicine. 2020;95(2):310–9.
Steinert Y, Naismith L, Mann K. Faculty development initiatives designed to promote leadership in medical education. A BEME systematic review: BEME Guide No. 19. Medical Teacher. 2012;34(6):483–503.
Schreurs ML, Huveneers W, Dolmans D. Communities of teaching practice in the workplace: evaluation of a faculty development programme. Medical Teacher. 2016;38(8):808–14.
We thank the faculty members who participated in the nominal group for their support and involvement in the study. We also thank Tehran University of Medical Sciences for supporting this paper as a pieces of a PhD thesis.
This study was funded by the National Agency for Strategic Research in Medical Education. Tehran. Iran. Grant No.970080. The role of the funding body was in providing support for data collection and analysis.
Ethics approval and consent to participate
The Institutional Review Board of Tehran University of medical sciences approved the study (No. IR.TUMS.IKHC.REC.1396.4122). Verbal informed consent was sufficient since the study had no intervention and its subject was not sensitive. Information on the study’s aim and how expert’s responses would be used was provided at the start of the nominal group technique to the participants. They were informed that their participation was voluntary, and anonymity was assured. When they agreed to participate, nominal group technique started. For documentation, whole process was audio recorded.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Salajegheh, M., Gandomkar, R., Mirzazadeh, A. et al. Identification of capacity development indicators for faculty development programs: A nominal group technique study. BMC Med Educ 20, 163 (2020). https://doi.org/10.1186/s12909-020-02068-7
- faculty development
- capacity development
- nominal group technique