We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
Evaluation of a consulting training course for international development assistance for health
BMC Medical Educationvolume 18, Article number: 234 (2018)
Development assistance for health (DAH) is an important component of foreign assistance. International health consultants usually play a key role in the international DAH field. However, there is still a shortage of consulting training in China. To address this issue and develop new backup force of DAH for China, the Global Health Institute of Wuhan University (GHIWHU) launched a training program called the “Consulting Training Course for International Development Assistance for Health”. The purpose of this article is to evaluate the impact of the training on participants.
We conducted the analysis using Kirkpatrick’s model. An evaluation survey examining participants’ reaction (level 1) and learning (level 2) was carried out among trainees following the training, and a follow-up telephone interview of application (level 3) was made in three months after the training.
A total of 25 participants from Chinese Consortium of Universities for Global Health (CCUGH) attended the training program. Results of satisfaction evaluation indicated that the training program was well received, with more than 85% of participants felt satisfied or relatively satisfied with the training. Trainees’ self-ratings of the consulting knowledge and skills showed a significant increase (p < 0.001) from pre- to post-training. The follow-up interview revealed that the majority of participants applied the acquired knowledge and skills under various circumstances such as consulting program, teaching processes, writing reports, and et al. Meanwhile, participants considered that the lack of opportunities was one of the major application barriers. In addition, they expressed the willingness to participate in more relevant training and the need for more practice opportunities.
This is the first study evaluating a consulting training program in China. The results show that the training course has been successfully implemented and participants have been given consulting knowledge and skills. Future research should use better-designed training methods based on demand surveys and consider providing participants with practice or practicum opportunities. Also, it is necessary to conduct both primary and advanced training courses and evaluate participants’ long-term behavior changes resulting from the training.
The 2030 Agenda for Sustainable Development increases focus on global health . Nowadays, China is becoming increasingly influential in the field of global health and has been under-recognized by the international health community . Emerging powers, including China, have gradually become donors of international health assistance [3,4,5]. According to the reported total amount of foreign aid, the actual amount invested by the Chinese government on Development assistance for health (DAH) was the highest among the BRICS countries during 2005–2010, reaching a peak of about $550 million in 2010 . Additionally in December 2015, the Chinese government agreed to spend $60 billion in three consecutive years to boost its cooperation with Africa that would benefit 10 major projects, including China-Africa public health cooperation program . These commitments and initiatives reflected the willingness of China to provide more international assistance and make greater contributions to the international community. Meanwhile, the developed countries in Europe are also concerned about the construction of global health value system . To handle the global health challenges, more actions are needed for workforce planning and capacity building of health professionals . As one of the emerging powers, Brazil is also actively involved with global health issues to enhance its international influence [10, 11]. However, China’s efforts in global health have not kept up with the pace of other countries, especially for the training of global health consultants.
Having qualified global health professionals is a prerequisite for any country’s involvement in global health. Faced with many challenges in public health, complex methods and strategies need to be designed by qualified public health workers to make effective responses . With the strengthening of international responsibilities, China has established solid relations with the African continent over the past three decades by providing all kinds of assistance to Africa, including those in the field of health . Ministerial Forum on China-Africa Health Development, one of various China-Africa health-related cooperation programs, has been established to strengthen health cooperation between China and Africa . In addition, the Africa Centres for Disease Control and Prevention (Africa CDC), supported by the Chinese Centers for Disease Control and Prevention (CCDC) and the United States Centers for Disease Control and Prevention (US CDC), has been launched to address Africa’s health problems . However, there is a huge gap between the capabilities of Chinese government officials and experts in international health programs and the needs of global health projects. Compared with the developed countries, China’s professionals engaging in global health mainly come from government, universities, and relevant research institutes. This makes it difficult to apply Chinese experiences to the African context . Chinese personnel are unable to provide high quality health advisory services and most of them lack practical experience. Few people have the skills to provide consultation. To a certain extent, this has limited China’s further participation in the field of global health.
To address this challenge, a training program, “Consulting Training Course for International Development Assistance for Health”, has been implemented by the Global Health Institute of Wuhan University (GHIWHU) under the support of Chinese Consortium of Universities for Global Health (CCUGH). The theme of this training is “how to provide technical advisory services to the international health development assistance program”. The training objects are experts, scholars, government personnel, and a few college students from the CCUGH member units. This training program is designed to: (1) enhance trainees’ understanding of global health, international DAH and the role of consultants; (2) help them recognize the whole process of technical consultation; (3) help participants acquire fundamental consulting knowledge and skills. At present, the training program of DAH for consultants has just emerged in China. Additionally, evaluation is an essential step in the building of a curriculum or program [17,18,19,20]. As one of the earliest training courses in China, it is necessary to conduct an evaluation of the training program . This paper describes the process and evaluation of the training, including satisfaction assessment, self-ratings of knowledge and skills, and a follow-up interview of application after 3 months of the course. To our knowledge, this is the first study to present the impact of a consulting training program on trainees in China.
The research was conducted by a mixed-methods approach based on Kirkpatrick’s model, using both quantitative and qualitative data. Participants received an evaluation questionnaire regarding their satisfaction of the training program and retrospective self-assessments on knowledge and skills by email after the training. A semi-structured telephone interview was adopted in the follow-up study to investigate trainees’ application on the acquired knowledge and skills after three months of the training.
We released the information regarding this training on CCUGH website on August 8, 2016. Participants registered the training program through the Mymova network system. As a result, a total of 25 applicants from universities, national health development research departments, and CCDC were recruited into the program. The study was approved by the Ethics Committee of Wuhan University School of Medicine. We obtained trainees’ oral informed consent before implementing both the training evaluation questionnaire and follow-up interview. In addition, the written informed consent was included in the cover letter of the questionnaire and sent to each participant.
The training course
Training content and teaching methods were designed carefully based on necessary competences in the international DAH consultation program. Through several rounds of discussion and debates, a consensus on the final arrangements and detailed contents of the training course was reached. The two and a half-day training program was implemented at School of Health Sciences, Wuhan University, between October 9 and October 11, 2016.
The training courses comprised 11 sessions, divided into 3 modules including theory, methods, and practice. It is noteworthy that the distance learning was used during the 6th session and the rest were all on-site training courses (see Table 1). Methods of interactive learning between trainers and participants were applied throughout the training. Combined with traditional didactic approach, various training methods were used including group discussion, case analysis, simulation training method, and role-play. The duration for each session was one and a half hours. Under normal circumstances, half an hour would be set aside in each session for trainees to express their thoughts and ask questions. Trainers would then respond to these questions and make appropriate adjustments to the course schedule.
We applied Kirkpatrick’s model to evaluate the outcomes of the training program . As a standard assessment tool, this model was widely used for training program evaluation [17, 23, 24]. Additionally, it can provide a systematic evaluation framework as both quantitative and qualitative data were collected during the assessment . The model focused on four levels of outcomes: (1) reaction of participants (the satisfaction of participants during the training); (2) participants’ learning (attitude changes of participants, their knowledge and skills obtained from the training); (3) behaviors of participants (applying knowledge and skills during daily practice); (4) the overall results (the impact of the training on the organization or institution). As the four levels are not ordered hierarchically, this model provided a flexible structure that can be tailored according to the needs of a specific evaluation . The aim of this study was to evaluate the effect of a training program on participants’ individual development, and we mainly focused on the first three levels.
Assessment tools and measures
After the training, participants were asked to complete a two-section training assessment questionnaire. The first section was a satisfaction evaluation test and was used to assess participants’ reaction on the training. Evaluation indicators included the satisfaction of course content, teaching methods, trainers, and program organization. Participants were asked to rate these indicators on a 5-point Likert scale, ranging from “unsatisfied” (1) to “satisfied” (5). The usefulness of the training and their willingness to participate in consulting services were also surveyed. Additionally, participants could write suggestions on the training in a free-text comment box. The second section was a retrospective knowledge and skills test, which was comprised of 25 items in eight modules (see Table 3). We asked participants to assess their learning of the training using a self-efficacy survey which was based on a 5-point Likert scale ranging from “know nothing about it” (1) to “can apply knowledge and skills effectively” (5). Combined with the advice from training experts, the evaluation model and the course content, the questionnaire was designed by GHIWHU researchers. Prior to the actual evaluation, the questionnaire was distributed to the volunteers from the CCUGH as a pilot study. The final version of the questionnaire was determined after several rounds of pilot investigations.
After searching literature and adopting ideas from instructors, we determined the interview guide before the follow-up interview. To evaluate the application of knowledge and skills, we interviewed participants who consented to participate in the follow-up assessment approximately three months after the training. The telephone interview consisted of five open-ended questions in three parts, including the daily application of the learning, application barriers and further learning. For instance, we asked participants to describe their daily practice (e.g., Have you applied the acquired knowledge in your daily work and study? If so, please give examples). In addition, we encouraged participants to report barriers in the application of the acquired knowledge and skills. The interview also explored participants’ willingness to attend more training and their expectations on further training. In order to assess the interview guideline and help interviewers master the interview skills, face to face interviews based on the impact of the training were conducted within the Global Health Institute.
Data collection and analysis
We extracted the baseline data (gender, age, title, position, and e-mail address) from the registration system and managed them in Microsoft Excel 2010. We also downloaded resumes of participants. Within one week after the training, all participants were surveyed anonymously. Finally, we collected 23 questionnaires, with 21 of them being valid and the other 2 uncompleted. Meanwhile, each participant was informed that a telephone interview would be conducted to access the long-term effect of the training. We consented that the information collected from interviews would be kept confidential. As a result, 22 trainees completed their interviews. All interviews were audiotaped, and were transcribed verbatim into the Microsoft word 2010 text format.
EpiData 3.0 was used to establish databases, and SPSS 19.0 was used for statistical analysis. Quantitative data (baseline data and training satisfaction ratings) were descriptively analyzed. Because of the relatively small sample, the results of self-assessments in pre- and post-training were not normally distributed. Therefore, median, upper quartile, lower quartile, and nonparametric statistics were used in the analysis. The Wilcoxon signed ranks test was used to compare the self-assessments of knowledge and skills before and after the training. A p value less than 0.05 was considered statistically significant. Missing data (uncompleted questionnaires) were excluded from the analysis.
Qualitative data (text comments and telephone interview contents) were analyzed using content analysis . The first author initially developed a code list according to the interview guide. Then, two authors read each transcript carefully and categorized the text data individually. The first author subsequently compared the themes by checking differences and similarities of the results. During this process, codes were combined or split in order to ensure a more accurate categorization. When a disagreement on the code emerged, the two authors would discuss the difference to reach a consensus. All responses would be reviewed by both authors in the end.
Twenty-five participants took part in the training course. These responders have different occupations: academics (9), civil servants (3), researchers (4) coming from health departments of government, and communication and advocacy officer (1) coming from the Joint United Nations Programme on HIV/AIDS (UNAIDS) China focal point. The rest of participants (n = 8) were medical students. The majority of participants were females (71.4%), with an average age of 30 years. The participants have good education backgrounds. More than half of them (n = 15, 60%) earned a master’s or doctoral degree. In addition, the participants expressed strong interests in international DAH. A total of 21 participants completed the training evaluation survey, and 22 completed the follow-up interview. The response rates were 84% and 88%, respectively.
The results of satisfaction evaluations showed the training was well received. More than 95% of participants responded that they were relatively satisfied or satisfied with the training scheme, and the satisfaction rate was 100% on training environment. More than 85% of participants were satisfied with course content, teaching methods, and trainers (see Table 2). In addition, 8 participants (38.1%) responded that the acquired knowledge of the training would be probably useful in their future work, while 12 (57.1%) thought it would be certainly helpful to their work. Moreover, when trainees were asked “If there is an opportunity, would you like to take part in the international consulting work”, 14 (66.7%) responded that they would certainly engage in the work. Content analysis of the free-text comment responses revealed further information regarding training assessment. Participants indicated that the group homework helped apply the acquired knowledge and skills in practice effectively, as “enthusiasm for learning will be enhanced under stresses between groups”. Some participants advised that enrollment publicity of the training should be strengthened, because “I hope more young scholars would have the opportunity to participate in such good training”. Meanwhile, more than one participant expressed that the effect of distance learning was not good. Also, participants expected to add interactive activities in small groups to help with the mutual acquaintance and communication in the future.
Knowledge and skills assessment
Table 3 shows results of pre- and post-training self-assessment of trainees on knowledge and skills. The Wilcoxon’s signed ranks test indicated a significant improvement in the post-training assessment for all 25 items in eight modules, as compared with the pre-training evaluation (p < 0.001). Prior to the training, the self-rated knowledge and skills scores regarding the eight modules were relatively low, ranging from 1 to 4. After the training, the improvement was observed that all items’ scores reached 3 and higher. More than 75% of participants obtained some knowledge and over half of them grasped relevant knowledge and skills on the consulting service after the training. A higher gain in self-assessed knowledge and skills could be found in the modules: Terms of Reference (ToR) & Curriculum Vitae (CV); organization and implementation of consulting activities; and writing consulting report. The items for the concept of global health, Global health governance, six elements of the health system, epidemiological analysis methods, and quantitative research methods have been marginally increased. It is noteworthy that in the 25 self-ratings, 7 items’ scores reached 4.5 or above, including definition of international DAH, quality and ability of experts, project management, six elements of the health system, epidemiological analysis methods, China’s contribution to global health, and quantitative research methods.
A total of 22 participants (88%) received the telephone interview. Table 4 shows the results on applying the acquired knowledge and skills in the follow-up interviews. 19 participants (86.4%) had applied the acquired knowledge and skills during their daily work or study (including teaching processes, writing reports, collecting and analyzing data, and updating CV), while only one participant applied them in the consulting program. Besides, two participants have not applied the acquired knowledge yet. Some of the participants indicated that:
“I used it in my work. In the process of course teaching, I introduced the theory about global health to my students.”
“The theoretical part was useful when I wrote research reports, including the role of international development assistance for health, problems China is facing, and the function of World Health Organization (WHO).”
When participants were asked about the impact of training on their day-to-day practices, they responded that it was difficult to articulate the concrete aspects of the most and least influential parts in the training and more than half of them considered all the acquired knowledge to be useful. Specifically, trainees applied knowledge and skills, namely, theoretical knowledge of international DAH consultant, methods of collecting and analyzing data, and writing ToR & CV most frequently. Except for 3 participants, all of the trainees have shared the acquired knowledge and experience with their colleagues or classmates. The following are some responses from the participants:
“In the global health center of Zhejiang University, I shared the training information with teachers, doctors, and post-graduates at the regular meeting.”
“That evening, I came back from training, and I immediately talked to students about what I have learned from the training.”
In addition, participants also encountered obstacles when applying these knowledge and skills. Most of them said they did not have the opportunity to participate in the international consultation program. Some said that there is no suitable consulting program for them before they could get improved. In terms of further training, 95.5% of the participants (n = 21) expressed that they were glad to attend more training. Only one trainee responded: “I wouldn’t attend such training in the near future. I attended another training in Beijing after this training. Now I am waiting for the practice opportunities to use knowledge I learned from these two training courses.” About suggestions, more consulting cases discussions and consulting practices were expected. Some of the trainees indicated:
“More cases that the trainer has completed should be added. I want to learn every step of the case and more detailed operations”.
“Different forms of training or activities tailored to different needs should be designed. If our practicum regarding consulting activities cannot be conducted outside the country, it may be more effective to have an interview or a seminar in a practicum such as half-day or two hours in the domestic sector. I hope to practice.”
In this research, we conducted an evaluation of an international DAH consulting training that aimed to enhance participants’ technical consulting competence. The evaluation results revealed that participants were satisfied with the training program and their self-assessment showed a significant improvement of their consulting knowledge and skills upon completion of the training. According to the follow-up interview, participants have applied the knowledge and skills they learned in their daily work and study. To our knowledge, this is the first study to evaluate an international DAH consulting training in China.
Previous studies have demonstrated that characteristics of participants may affect the training outcomes. Frantz et al. found  that the diversity of experiences, nationalities, culture and professions, and present in participants’ recruitments strongly influenced the personal development of trainees and group identity in training sessions. Harris et al. found the training program could be strengthened through interprofessional dialogue between multiple professionals during the training . In this study, although the participants came from different backgrounds, the participants’ engaging fields were relatively similar. As international DAH consulting work are multidisciplinary, cross-cultural and cross-regional, WHO attaches great importance to cross-professional education and cooperative practices around the world . Therefore, it is essential to train substantial numbers of experts with different professions, diverse educational backgrounds, and rich experience to bolster the global health workforce.
In the satisfaction assessment, participants gave a high rating on all aspects of the training program. Almost all the participants were satisfied, which could be interpreted as trainers with extensive consulting experience in the international DAH field and trainees were interested in the training content. Furthermore, a previous study showed that a high level of participants’ satisfaction was important for projects that helped to prevent a decline in participation . In addition, participants thought the training was useful and some of them were willing to attend DAH consulting program in future, which showed the training was successful and meaningful. Prior studies also found that changes of participants’ attitudes and behavioral intentions could influence their knowledge and performance during and after the training [32, 33].
The issue of distance learning is worth noting. With the progress of computer technology, a large number of training courses adopted the way of network learning and achieved good results [34, 35]. In this research, results of qualitative responses showed that distance learning was less useful than other methods in imparting consulting knowledge. Several factors may explain this phenomenon. First, the course taught by distance learning was in English and the trainer spoke at a relatively fast speed. For non-native English speakers, language is one of the major barriers for learning . To address this problem, Brownson et al. suggested  that it was essential to have clear slides, to keep presentations at a reasonably slow pace, and to allow plenty of time for questions in the course. Second, Internet connection and modern information technology had an impact on the quality of distance learning [35, 38]. Third, the lack of interaction between trainers and distance learners may be another reason for this phenomenon. Existing research shows that trainers and trainees prefer interactive teaching, which could usually receive good results during the training and is useful for participants to obtain knowledge and skills [17, 39,40,41].
According to the consulting knowledge and skills assessment results, there was a significant increase in participants’ self-ratings. After the training, the scores for all items were increased to greater than 3. This may be because that trainees are highly educated (more than half of them hold masters’ or doctoral degrees) and possess a background in public health. Similarly, Maxwell et al. found  that trainees with high levels of education tend to enhance self-efficacy following the training. A previous study also pointed out that personal efficacy is influenced by the experiential sources and the self-efficacy can be enhanced by the mastery experiences of the related and non-related areas . Furthermore, most of participants are young scholars that are open to new knowledge and re-education. In addition, few participants’ self-scores reached close to 5 (can apply knowledge and skills effectively) in a few sessions of the training. This may be explained by the fact that the investigation was conducted within a week after the training and it was difficult for participants to immediately apply the acquired knowledge and skills to their work. This is similar to results found by Silva et al. . Besides, most of participants were new to the field of DAH making them difficult to apply knowledge and skills proficiently.
The study found that most trainees applied the acquired knowledge and skills in various aspects, while only one applied them in the international consulting program. Lack of opportunity for consulting practice was the biggest barrier leading to this outcome. Several key factors may account for this situation. First, DAH consulting training is not yet mature in China and lacks the platform for clients and consultants to share information between each other. Second, some experienced trainers indicated that clients, such as WHO project officers, have their own think tanks. Third, consistent with findings of Ridde et al. , it is relatively easier to acquire knowledge than to apply them.
The participants expressed their training needs for further learning. Almost all of them responded that they were glad to attend more training courses. They also hope that future training can add more consulting cases, and training courses can be divided into ones with introductory- and advanced-levels. Regarding the form of training, some participants suggested that the training sponsors could organize practicum in the domestic sector. It is also worth noting that the participants considered that the training program successfully introduced the panoramic view of international DAH consulting knowledge and broadened their horizons.
Several limitations exist in the research. First, this training recruited members restricted to CCUGH leading to a small sample size and the reduced statistical power. Future training should recruit more trainees who are interested in engaging in international DAH programs. Second, we did not conduct the demand survey for training content and methods. Even though, the satisfaction evaluation results indicated that participants gave a high rating on the training. We will make improvements regarding this issue and conduct a needs assessment in our next training course. Third, Due to the lack of appropriate testing tools, we had to use self-assessment questionnaires rather than objective assessments to evaluate the results. Finally, we did not complete the fourth level of the Kirkpatrick model because it is very difficult to assess the long-term impact of the training. And various factors can influence the training effectiveness, such as organizational environment [38, 45].
The consulting training of international DAH in China is still in its infancy. This is the first study to present an evaluation of a consulting training program carried out in China. The results of this study show that the training has been well received and we have successfully introduced the participants to the initial procedures for the consulting activities. Also, the research demonstrated that a short-period training of international DAH could be used as a tool to improve the consulting knowledge and skills of health professionals. Compared to distance learning, some trainees prefer on-site training that is more effective. Furthermore, most participants commented that they need similar training with more consulting cases, and the lack of access to consulting practice was the biggest barrier. This study provides a useful consulting training model for international DAH in China. Additionally, it is necessary to provide more primary and advanced training courses to evaluate participants’ long-term behavior changes resulting from the training.
- Africa CDC:
Africa Centres for Disease Control and Prevention
Brazil, Russia, India, China, and South Africa
the Chinese Centers for Disease Control and Prevention
Chinese Consortium of Universities for Global Health
Development assistance for health
Global Health Institute of Wuhan University
Terms of Reference
the Joint United Nations Programme on HIV/AIDS
- US CDC:
the United States Centers for Disease Control and Prevention
World Health Organization
Nunes AR, Lee K, O’Riordan T. The importance of an integrating framework for achieving the sustainable development goals: the example of health and well-being. BMJ global health. 2016;1(3):e000068.
Han Q, Chen L, Evans T, Horton R. China and global health. Lancet (London, England). 2008;372(9648):1439–41.
Harmer A, Xiao Y, Missoni E, Tediosi F. BRICS without straw'? A systematic literature review of newly emerging economies' influence in global health. Glob Health. 2013;9:15.
Tytel B, Callahan K. Shifting paradigm: how the BRICS are reshaping Global Health and development. New York: Global Health Strategies Initiative; 2012.
Shajalal M, Xu J, Jing J, King M, Zhang J, Wang P, Bouey J, Cheng F. China's engagement with development assistance for health in Africa. Global health research and policy. 2017;2:24.
Cao G, Wang Y-P, Fu Z, Fan X-D, Jin N, Liang W-J. Development assistance for health in BRICS countries. Chinese Journal of Health Policy. 2015;8(05):44–7.
Xi announces 10 major China-Africa cooperation plans for coming 3 years. Forum on China-Africa Cooperation. 2015. https://www.focac.org/eng/ljhy_1/dwjbzjjhys_1/hyqk/t1322068.htm. Accessed 3 Oct 2018.
European Commission. Communication from the commission to the council, the European Parliament, the European Economic and Social Committee and the Committee of the Regions. The EU Role in Global Health. 2010. https://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1538571970206&uri=CELEX:52010DC0128. Accessed 3 Oct 2018.
Bhutta ZA, Chen L, Cohen J, Crisp N, Evans T, Fineberg H, Frenk J, Garcia P, Horton R, Ke Y, et al. Education of health professionals for the 21st century: a global independent commission. Lancet. 2010;375(9721):1137–8.
Costa Vaz A, Inoue CYA: Emerging donors in international development assistance: the Brazil case. Rowlands, D(2008): Emerging Donors in International Development Assistance, Partnership&Business Development Division IDRC 2007.
Almeida C, RPd C, Buss P, Ferreira JR, Fonseca LE. Brazil’s conception of south-south “structural cooperation” in health. Reciis. 2010;4(1).
Demers AL, Mamary E, Ebin VJ. Creating opportunities for training California's public health workforce. J Contin Educ Heal Prof. 2011;31(1):64–9.
Florini A, Nachiappan K, Pang T, Pilcavage C. Global Health governance: Analysing China, India and Japan as Global Health aid donors. Global Policy. 2012;3(3):336–47.
Lin S, Gao L, Reyes M, Cheng F, Kaufman J, El-Sadr WM. China's health assistance to Africa: opportunism or altruism? Glob Health. 2016;12(1):83.
Nkengasong JN, Maiyegun O, Moeti M. Establishing the Africa Centres for disease control and prevention: responding to Africa's health threats. Lancet Glob Health. 2017;5(3):e246–7.
Peilong L, Berhane Y, Fawzi W. China, Africa, and US academia join hands to advance global health. Lancet (London, England). 2017;390(10096):733–4.
Ridde V, Fournier P, Banza B, Tourigny C, Ouedraogo D. Programme evaluation training for health professionals in francophone Africa: process, competence acquisition and use. Hum Resour Health. 2009;7:3.
Nestel D, Regan M, Vijayakumar P, Sunderji I, Haigh C, Smith C, Wright A. Implementation of a multi-level evaluation strategy: a case study on a program for international medical graduates. J Educ Eval Health Prof. 2011;8:13.
Quigley RJ, Taylor LC. Evaluating health impact assessment. Public Health. 2004;118(8):544–52.
Haque H, Hill PC, Gauld R. Aid effectiveness and programmatic effectiveness: a proposed framework for comparative evaluation of different aid interventions in a particular health system. Glob Health Res Policy. 2017;2:7.
Ghaffari F, Dehghan-Nayeri N, Navabi N, Seylani K. Evaluation of the Master's curriculum for elderly nursing: a qualitative study. Clin Interv Aging. 2016;11:1333–42.
Kirkpatrick D. Evaluation, training programs: the four levels, San Francisco: Berrett-Koehler publishers. Learning transfer. Int J Train Dev. 1994;6(1):36–48.
Banerjee SC, Manna R, Coyle N, Penn S, Gallegos TE, Zaider T, Krueger CA, Bialer PA, Bylund CL, Parker PA. The implementation and evaluation of a communication skills training program for oncology nurses. Transl Behav Med. 2017.
Bylund CL, Brown RF, Bialer PA, Levin TT, Lubrano di Ciccone B, Kissane DW. Developing and implementing an advanced communication training program in oncology at a comprehensive cancer center. J Cancer Educ. 2011;26(4):604–11.
Dorri S, Akbari M, Dorri Sedeh M. Kirkpatrick evaluation model for in-service training on cardiopulmonary resuscitation. Iran J Nurs Midwifery Res. 2016;21(5):493–7.
Aleksic B, Hamilton J, Stevens G, Girdler S. Becoming a Mentor: the impact of training and the experience of Mentoring University students on the autism Spectrum. PLoS One. 2016;11(4):e0153204.
Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.
Frantz JM, Bezuidenhout J, Burch VC, Mthembu S, Rowe M, Tan C, Van Wyk J, Van Heerden B. The impact of a faculty development programme for health professions educators in sub-Saharan Africa: an archival study. BMC Med Educ. 2015;15(1).
Harris WA, Spencer P, Winthrop K, Kravitz J. Training mid- to late-career health professionals for clinical work in low-income regions abroad. J Contin Educ Heal Prof. 2014;34(3):179–85.
Gilbert JH, Yan J, Hoffman SJ. A WHO report: framework for action on interprofessional education and collaborative practice. J Allied Health. 2010;39(Suppl 1):196–7.
Kim DH, Yoon HB, Sung M, Yoo DM, Hwang J, Kim EJ, Lee S, Shin JS. Evaluation of an international faculty development program for developing countries in Asia: the Seoul intensive course for medical educators. BMC Med Educ. 2015;15:224.
Qureshi KA, Gershon RR, Merrill JA, Calero-Breckheimer A, Murrman M, Gebbie KM, Moskin LC, May L, Morse SS, Sherman M. Effectiveness of an emergency preparedness training program for public health nurses in new York City. Fam community health. 2004;27(3):242–9.
Wang C, Xiang H, Xu Y, Hu D, Zhang W, Lu J, Sun L, Nie S. Improving emergency preparedness capability of rural public health personnel in China. Public Health. 2010;124(6):339–44.
Alfieri J, Portelance L, Souhami L, Steinert Y, McLeod P, Gallant F, Artho G. Development and impact evaluation of an e-learning radiation oncology module. Int J Radiat Oncol Biol Phys. 2012;82(3):e573–80.
Chastonay P, Zesiger V, Moretti R, Cremaschini M, Bailey R, Wheeler E, Mattig T, Avocksouma DA, Mpinga EK. A public health e-learning master's programme with a focus on health workforce development targeting francophone Africa: the University of Geneva experience. Hum Resour Health. 2015;13:68.
Ferry LH, Job J, Knutsen S, Montgomery S, Petersen F, Rudatsikira E, Singh P. Mentoring Cambodian and Lao health professionals in tobacco control leadership and research skills. Tob Control. 2006;15(Suppl 1):i42–7.
Brownson RC, Diem G, Grabauskas V, Legetic B, Potemkina R, Shatchkute A, Baker EA, Campbell CR, Leet TL, Nissinen A, et al. Training practitioners in evidence-based chronic disease prevention for global health. Promot Educ. 2007;14(3):159–63.
Omar M, Gerein N, Tarin E, Butcher C, Pearson S, Heidari G. Training evaluation: a case study of training Iranian health managers. Hum Resour Health. 2009;7:20.
Yoon HB, Shin JS, Bouphavanh K, Kang YM. Evaluation of a continuing professional development training program for physicians and physician assistants in hospitals in Laos based on the Kirkpatrick model. J Educ Eval Health Prof. 2016;13:21.
Davis MV. Teaching practical public health evaluation methods. Am J Eval. 2006;27(2):247–56.
Burke MJ, Sarpy SA, Smith-Crowe K, Chan-Serafin S, Salvador RO, Islam G. Relative effectiveness of worker safety and health training methods. Am J Public Health. 2006;96(2):315–24.
Maxwell AE, Danao LL, Cayetano RT, Crespi CM, Bastani R. Evaluating the training of Filipino American community health advisors to disseminate colorectal cancer screening. J Community Health. 2012;37(6):1218–25.
Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev. 1977;84(2):191–215.
Silva LC, Teixeira M, Ribeiro EL, Paula CS. Impact of a provider training program on the treatment of children with autism spectrum disorder at psychosocial care units in Brazil. Rev Bras Psiquiatr. 2018;40(3):296–305.
Tracey JB, Tannenbaum SI, Kavanagh MJ. Applying trained skills on the job: the importance of the work environment. J Appl Psychol. 1995;80(2):239–52.
We would like to thank all the participants in this study for their willingness to accomplish our investigation. We would also like to thank Dr. Shenglan Tang, Professor of Medicine and Global Health at Duke Global Health Institute at Duke University and Director of the Global Health Research Center at Duke Kunshan University; Dr. Tuohong Zhang, national technical Official of WHO Representative Office in China; Dr. Yan Guo, Professor of School of public health at Peking University and Dr. Zheng Xie, Associate Professor of School of public health at Peking University for their support to this program. We are grateful to the trainers and training counselors for their utmost efforts in the training process. Finally, we would like to acknowledge Dr. Jay Maddock for his critical reading of the manuscript.
The study was funded by the global health research program (No.13–134) from China Medical Board (CMB). The sponsors were not involved with research design, data collection, data analysis, and data interpretation, nor did they participate in manuscript writing.
Availability of data and materials
All data generated or analyzed during this study are included in this published article.
Ethics approval and consent to participate
The study was approved by Ethics Committee of Wuhan University School of Medicine. Researchers obtained informed consent of each trainee prior to the evaluation.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.