Identifying research priorities for health professions education research in sub-Saharan Africa using a modified Delphi method

Background Recent increases in health professions education (HPE) research in sub-Saharan Africa (SSA), though substantial, have predominantly originated from single institutions and remained uncoordinated. A shared research agenda can guide the implementation of HPE practices to ultimately influence the recruitment and retention of the health workforce. Thus, the authors aimed to generate and prioritise a list of research topics for HPE research (HPER) in SSA. Methods A modified Delphi process was designed to prioritise a shared agenda. Members of the African Forum for Research and Education in Health (AFREhealth) technical working group (TWG) were asked to first list potential research topics. Then, members of the same TWG and attendees at the annual AFREhealth academic symposium held in Lagos, Nigeria in August 2019 rated the importance of including each topic on a 3-point Likert scale, through two rounds of consensus seeking. Consensus for inclusion was predefined as ≥70% of respondents rating the topic as “must be included.” Results Health professions educators representing a variety of professions and 13 countries responded to the survey rounds. Twenty-three TWG members suggested 26 initial HPER topics; subsequently 90 respondents completed round one, and 51 completed round 2 of the modified Delphi. The final list of 12 research topics which met predetermined consensus criteria were grouped into three categories: (1) creating an enabling environment with sufficient resources and relevant training; (2) enhancing student learning; and (3) identifying and evaluating strategies to improve pedagogical practice. Conclusions Establishing research priorities for HPE is important to ensure efficient and appropriate allocation of resources. This study serves as a reminder of how the prevailing context within which HPE, and by implication research in the field, is undertaken will inevitably influence choices about research foci. It further points to a potential advocacy role for research that generates regionally relevant evidence. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-020-02367-z.


Introduction
There has been a rapid increase in the number of health professions training institutions in Sub-Saharan Africa (SSA) in order to train more health professionals for the region. Despite this, SSA remains challenged in meeting the health needs of its populations, exacerbated by existing and emerging epidemiologic challenges. [1] There has, however, been global interest in strengthening human resources for health (HRH) in the region. Multiple initiatives to improve both the number of graduate outputs and the quality and relevance of their training have been launched. [2] One example is the Medical Education Partnership Initiative (MEPI) and Nursing Education Partnership Initiative (NEPI), a $130 million competitively awarded grant by the President's Emergency Plan for AIDS Relief (PEPFAR) to medical and nursing schools in 12 Sub-Saharan African countries from September, 2010 to August, 2015. The goals of MEPI and NEPI were to increase the capacity of the awardees to produce more and better doctors and nurses, strengthen locally relevant research, promote retention of graduates within their countries, and ensure sustainability. [3] The establishment of the African Forum for Research and Education in Health (AFREhealth), in 2017, has now sustainably consolidated these initiatives. AFREhealth aims to collaborate with stakeholders to improve health outcomes, work towards an AIDS-free generation, establish a research agenda for health priorities in Africa, and mobilise vital resources. [4] Much of the published material regarding research in Africa has cited the need to establish a coordinated research agenda to inform priorities. [5] This is similar for health professions education research (HPER), which is a tenet of successful health professions education (HPE) centers and departments. [6,7] Relatively little has been published on HPER in SSA, despite the increase in the number of training institutions. Although there has been an increase in research outputs in recent years, assisted by the establishment of the African Journal for Health Professions Education (AJHPE), this increase has not matched the rapid growth in the number of institutions. During MEPI, over 376 peer-reviewed publications, including a special supplement in Academic Medicine (2014), were published, however most of these articles tended to be descriptive in nature. [3] Van Schalkwyk [8] highlighted the need for strengthening research capacity to generate a wider evidence base in HPE, moving beyond description to contributing to theory building in the eld. [9] As a rst step towards responding to this call, we sought to establish what role players in the region would regard as priorities for HPER initiatives. An assumption was that having a shared agenda can strengthen collaborative work in the eld and can contribute to appropriate allocation of resources. In addition, and given the importance of context in improving health and the delivery of HPE, local research can also identify potential challenges, set priorities, devise original solutions, and make the best use of scarce resources. [10,11] Therefore, through leveraging the input of regional experts and researchers, we sought to identify a set of HPER priorities to facilitate the implementation of regionally relevant initiatives and concomitantly guide resource allocation.

Methods
We used the Delphi method (with modi cations) to establish priorities for HPER in SSA. The Delphi method is a consensus-building approach which seeks expert opinion on a pre-determined topic in a structured and iterative manner. [12,13] The Delphi method can be useful in areas where evidence-based literature is limited, as it can unearth collective knowledge from those in the eld. [13][14][15] A series of rounds are used to clarify, re ne, and ultimately achieve consensus on the area under discussion. A key feature of the method is that participants or respondents provide input independently and anonymously during each round, resulting in a process that is not unduly in uenced by any one individual or subset of respondents. [12,16] The Delphi method usually involves six steps, namely the identi cation of a research question/problem; conducting a literature search; developing a set of statements around the topic of choice; performing anonymous iterative rounds; providing feedback to the respondents between rounds; and summarising the ndings. [14] We report our process accordingly.
Step 1 -Identifying the research problem/question Our process involved one round of item generation and two rounds of consensus seeking carried out between April 2019 and October 2019. A group of researchers involved in the AFREhealth network led the process. The authors formed a study group that re ned the research question over several project meetings and discussions. Three of the authors (SvS, EKM, MdV) are practicing health professions education experts based in Africa. The other two team members (MR and JB) are infectious diseases experts with a special interest in health professions education, based in the USA. The consensus development process is summarised in Fig. 1.
Step 2 -Literature search To inform the development of the protocol for our study, we conducted a review of relevant literature, including work describing the Delphi methodology, and focused on studies, both national and international, that have sought to establish research priorities for their speci c contexts.
Step 3 -Topic generation A core group of 62 AFREhealth educators and investigators, members of the AFREhealth HPER Technical Working Group (TWG) comprising colleagues from the region with an expressed interest in HPER, were invited to generate topics for consideration. In April 2019, we sent via email a survey designed in Qualtrics ♥ (Provo, UT) [17] asking, 'what are possible research questions, topics, or areas of focus that should be a priority in a health professions education research agenda for sub-Saharan Africa?' We also requested descriptions and rationales for each research question proposed. Two email reminders were sent. Twenty-three people responded proposing an initial list of 34 research priorities. This list was synthesised by two of the researchers (SvS and MdV) to remove unnecessary duplication, resulting in a list of 26 items. At this point, the research team made a decision to rephrase all of the items as priority topics to ensure consistency across the items.
Step 4 -Conducting two iterative anonymous rounds The list of 26 research priorities generated during stage 3 were distributed in the form of a survey for prioritisation to members of the working group, as well as to a broader group of researchers and stakeholders who attended the AFREhealth Symposium in Lagos, Nigeria in August 2019. This step (Round One) represents a modi cation in our Delphi in that we introduced additional respondents after the topic generation stage. This meeting involved researchers and educators in the health professions, as well as service providers in the eld, mainly from across Anglophone sub-Saharan Africa. Those attending the symposium were asked to complete the prioritisation survey either online (designed in Qualtrics ♥ ) or using a paper-based version. Those members of the TWG not present in Lagos were invited via e-mail to complete the same online survey. Two email reminders were sent. Ninety individuals participated in this rst round to begin prioritising the topics generated in step 3. Basic demographic information was requested from respondents. Responses were anonymous and treated con dentially. All respondents were asked to rate how important each of the 26 research topics would be to include in a HPER research agenda for SSA using a 3-point Likert scale that ranged from 'do not include' to 'could be included' to 'must be included.' Respondents could also pick 'no vote,' if they felt they did not have enough knowledge on a topic to make an informed decision of its importance. Those votes were coded as null. Topics were itemised in random order. Each respondent was limited to one survey response each. Consensus criteria, that only topics rated as 'must be included' by ≥ 70% of respondents would be included, were set a priori. In addition, an open-ended question prompted submission of additional research topics or agenda items in this round. New topics were reviewed and consolidated by the research team into existing themes or added to the list.
For Round Two, in September 2019, all respondents who provided their e-mail addresses in Round One were sent a link via email to the nal online survey in Qualtrics ♥ . The nal survey included 10 topics which had met consensus criteria in Round One, and 4 additional suggested topics, generating a total of 14 topics. Respondents were again invited to rate each research priority on a 3-point Likert scale in terms of importance to include with consensus set at ≥ 70% of respondents who rated the topics as 'must be included.' Each participant was limited to one survey response, and topics were itemised in random order. After Round Two, su cient consensus was reached on a smaller number of topics [10] for the process to be terminated.
Step 5 -Providing feedback to respondents Round One respondents who provided their email addresses received feedback. This feedback was included in the email request to participate in Round Two and outlined the areas that achieved consensus in the rst round, in addition to including the percentage of respondents rating each top as 'must be included,' as well as the additional items that were generated in Round One.
Step 6 -Summarise the ndings The research team, rst individually and then collectively, thematically grouped the items that achieved consensus in three areas, thus applying a next order round of analysis. These are reported in the Results section below.
In sum, Table 1 reports the quality criteria for Delphi studies as proposed by Humphrey-Murto et al. [14], with three additional criteria from Diamond et al, [12] as applied in our study. of respondents in both rounds were Nigerian, an additional sensitivity analysis was performed to determine if research priorities were different among respondents from Nigeria compared to elsewhere in SSA. The nal list of priority research issues was the same among Nigerian respondents as compared to respondents from other countries. Respondents reported a median of 14 years' experience working in HPE, with a range from one to 42 years. Although the majority of the respondents were medically and nursing quali ed, the individuals represented a wider range of health professions as well as a few others (Table 3).  Less than 5 30 9 More than 5 25 25 No answer 2 --

HPER = Health professions education research, HPE = Health professions education
Thirty-four topics and topic descriptions were generated by the TWG members, which were reduced to 26 topics by the research team (see previously). In Round One, ten topics met consensus criteria. Twentythree new topics were suggested, and four of these were incorporated into the next round; those that were not included were deemed su ciently similar to ones already included in the existing list. In the second topic prioritisation round, ten topics met consensus criteria. (Table 5)

Synthesis of Topics
The process of synthesis that occurred as a result of establishing consensus within the Delphi saw interesting shifts in perspectives from the initial set of 26 topics, to the nal list of ten priorities. The early list, for example, included more generic issues such as 'assessment practices', 'student retention', 'graduate competencies', 'post-graduate training' and 'self-regulation skills' -topics that would likely resonate with health professions educators all over the world. The nal list, however, represents what could be regarded as higher-order topics which, with one or two exceptions, speak to national and regional issues. They re ect the context within which health professionals in SSA are being trained and the challenges that characterise this context. During the topic generating phase of the study, the rationales that respondents provided in support of their selections emphasised the need for understanding 'our own' challenges in order to 'decolonise our way of teaching and assessment.' It was also argued that existing frameworks had been generated in 'developed countries' acknowledging the need for locally, relevant and responsive research. Ultimately, second-tier analysis by the research team resulted in three over-arching, but inter-connected themes: (1) creating an enabling environment with su cient resources and relevant training; (2) enhancing student learning; and (3) identifying and evaluating strategies to improve pedagogical practice. These could be regarded as a triumvirate of educational endeavors -the teaching (pedagogy), the learning, and then the environment (context) within which it should occur. The themes are discussed below, supported in some instances with direct quotations from some of the rationales provided during the topic generation phase, and with several priorities having relevance across more than one.

An enabling environment
A key message from this Delphi is that a rst step to de ning and addressing the HPER priorities in SSA is to create the environment that will foster researchers, with su cient training and resources to address the most pressing questions. This thesis is underscored by the fact that the top 10 priority topics include three that are focused on understanding the factors that currently make HPER challenging in SSA:

Priority # 1 -Addressing the human resources for health challenges in rural and remote settings
This rst ranked topic focuses on addressing human resources for health challenges in rural and remote settings. This may be an unexpected result for colleagues around the world, but needs to be seen in the context of HPE in SSA as described in the introduction and points to the lived experience of many educators in the eld who practice their teaching in resource-constrained contexts. The importance of the environment in in uencing teaching and learning has been well-documented. [18] It can be inferred that for our respondents, addressing the need for practitioners who can respond to the increasing burden of disease, particularly in rural areas, is a non-negotiable imperative whether in terms of the taught curriculum (as many medical schools and health sciences faculties embrace distributed clinical learning [19]), or in terms of ensuring an environment in which teaching can meaningfully occur. Linked to this was the underlying premise of providing quality health care for all.

Priority # 5 -Faculty development for clinical teaching
Supporting those responsible for teaching, speci cally clinical teaching, ranked fth. Recommendations spoke speci cally to looking to discern the status of faculty development in the region, describing it as a particular area requiring further investigation. Given the current drive towards the professionalisation of the educational role, and the growing need for faculty who can teach the growing numbers of HPE students, this focus was expected as there has been limited work in this area. An exception is research into the role of emerging clinical teachers which has been conducted in the region in recent years, [20,21] with studies emphasising the need for further work and ongoing support for those responsible for HPE students in clinical training, particularly those who are placed in rural or distributed sites. Priority # 7 -Resources, political commitment and funding for HPE in SSA This priority picks up on the higher order focus established in Priority # 1, foregrounding the economic and political instability that characterises much of the SSA region, the impact that this has on health care and the subsequent effect on HPE. During topic generation, respondents felt that education, research and service delivery activities should all be directed, "towards addressing priority health concerns of the community, region and/or nation that they have a mandate to serve." Enhanced student learning Among the priorities identi ed, four foregrounded approaches could potentially enhance student learning. An important proviso for these priorities was that the ultimate aim of enhanced student learning was the delivery of graduate professionals who could respond to local and regional health care imperatives and provide quality health care.

Priority # 2 -Interprofessional collaboration in clinical practice in SSA
Priority #2 highlights an essential approach to healthcare training and practice, namely interprofessional education and collaborative practice (IPECP), that should inform the student learning experience. IPECP is currently foregrounded in HPE research, visible in a plethora of publications. [2] In the rationales provided for interprofessional collaboration during the topic generating phase of this study, respondents spoke directly to the under resourced context within which many in SSA work, and the extent to which collaboration across all healthcare practitioners will be crucial to address the workforce challenges identi ed under Priority # 1 including that it would enhance patient care. It should be noted that the establishment of a SSA organisation dedicated to fostering IPECP (The African Interprofessional Education Network (AfrIPEN): https://afripen.org/) in 2017 may account for why this particular topic was ranked so high. AfrIPEN has an a liate relationship with AFREhealth, and it is plausible that some of our respondents are members of both organisations. In their rationales, however, respondents also made reference to the 'African patriarchal social system' that needs to be problematised as it is 'contrary to the philosophical underpinnings of shared leadership, shared decision-making etc.' that are so needed in the region. Priority #9 -Potential of rural communities as platforms for training health care professionals Linked strongly to the need to address human resources for health challenges was a focus on enhancing student learning through distributed clinical training, speci cally in rural areas. This priority emphasised the growing awareness of how large academic hospitals are not necessarily the best environment for training students to meet community needs.
Priority # 3 -Teaching a holistic and person-centered approach and Priority # 10 -Relevance of communication skills training in culturally diverse contexts Teaching person-centered and holistic care underscores the value of a comprehensive approach to the person in the African context which also takes into account the person's values and needs, as well as their Identifying and evaluating strategies to improve pedagogical practice A nal group of priorities, which includes some already mentioned above, relates to identifying and evaluating strategies to improve pedagogical effectiveness or assess new modes of curricula and pedagogical innovation. Priority # 4 -The role of information communications technology in HPE It could be argued that a focus on the role of information communications technology (ICT) in HPE would have been expected given the ubiquitous nature of blended and e-learning approaches in modern-day HPE. [24] Interestingly, however, this topic did not feature in the topic generating process, but was introduced in Round One and validated in Round Two. There can be no doubt as to the importance of this research focus, particularly investigation that can explore options for drawing out the affordances of ICT amid resource constraints. For example, while many rural areas in SSA may be isolated in terms of connectivity, the region is known for its high rate of cellular telephone coverage providing a lifeline for health care workers in these remote regions. Investigating the potential for HPE using mobile technology could, therefore, have particular relevance. [25] Priority # 6 -Quality assurance processes and procedures in health professions education This priority demonstrates the intent of respondents to ensure that educational practices and teaching innovations are carefully monitored and evaluated. Responses provided during the topic generation phase suggested that this focus has to do with being accountable given expectations by stakeholders for the provision of quality education. Priority # 8 -Determining how to develop and implement curricula that are responsive to the health needs of SSA In 2010, the Lancet commissioned article exploring Health professional in the 21st Century, [26] argued that curricula had not kept pace with community healthcare needs, catalysing introspection among health professions educators and curriculum developers across the world. Given SSA speci c resource constraints, and its unique burden of disease, exploring what a responsive curriculum for the region might look like would appear to be of signi cant value for the HPE community. It could be argued that this priority possibly provides a research focus that could encapsulate most if not all of the others that made the nal list.

Discussion
Establishing research priorities for HPE at a national or higher level is important to ensure maximum impact of efforts. Related work has previously been conducted in countries such as Canada [27], New Zealand [28], and Scotland [11], catalysed by a desire to foster collaborative and coordinated approaches to research [28] and to ensure targeted allocation of increasingly scarce resources [11]. Comparison between our results and those obtained in these studies highlights both similarities and differences. For family and community. [23] A further indicator of the complexity of context can be seen in priority # 10 which recognises the need for students to be trained to communicate effectively with patients across multiple contexts and different cultures. This links with previously mentioned perspectives of respondents who felt it was important to support research initiatives that would provide responses for local and regional contexts.
example, issues of student access and selection, the role of assessment and feedback, resilience and well-being, phases of transition across curricula, amongst others, are strongly foregrounded in these earlier studies, but do not make it to our nal list even though some featured in earlier iterations. The importance of faculty development is, on the other hand, one area of congruence, as is the importance of interprofessionalism. The key difference, however, resides in the extent to which the described need for political and economic stability, and the imperative to ensure suitably trained health professionals who can respond to national and regional challenges, is relatively silent in this earlier work, although there is reference to change management and the importance of leadership to facilitate such change.
What does this study mean for HPER in SSA going forward? What our Delphi highlights is that it is precisely the factors that currently undermine HPER activities in the region that have emerged as research priorities. Thus we are faced with a conundrum on a number of levels. Firstly, the identi ed set of priorities does not re ect speci c research gaps, but rather points to respondents' perception of challenges within HPE in general, emphasising the complexity of the context in which the training of future health professionals much occur. We acknowledge that this is probably linked to the fact that many of the respondents were clinicians and researchers with an interest in HPER, but not necessarily experts in the eld itself. It could be argued that research activities per se are unlikely to impact the national political and economic structures that currently determine the different health systems represented in the study, but perhaps that advocacy work, strengthened by locally generated evidence, is what will be needed.
Secondly, that while we do believe that the consultative process followed in generating this list of priorities can assist in establishing a more coordinated strategy for research in the future, it is also clear that capacity for conducting such research will need to be grown. Close to 40% of our respondents, for example, indicated that they had never published in the eld, although they may have done so in other disciplines. As mentioned in the introduction to this article, HPER outputs in the region are low. Indeed, one of the stated aims of the AFREhealth HPER TWG is to grow the community of active scholars in SSA and intentional steps towards such growth will be a necessary condition for implementing a research

Strengths and Limitations
We premised our choice of the Delphi approach on its relevance for HPE educators and ability to establish consensus on priorities. [14] The primary idea of the Delphi technique, though, is to de ne, select, and engage a relatively homogenous group of experts throughout the process. [13] We modi ed this by engaging a larger and more diverse group, whose status as 'expert' in the eld was unknown at that stage, for the two consensus building rounds. We attempted to ascertain their experience in the discipline of HPE in their own view as well as through their number of publications in the eld. This seems to have resulted in both advantages and limitations. Involving respondents with divergent opinions increased the number of perspectives to be considered, and relative 'newcomers' could have contributed more novel opinions than those established in the discipline. This may be the reason why our number one priority is of a wider contextual nature. On the other hand, this could also have limited the research priorities that emerged from the structured sub-elds of HPE.
Sustaining participation across rounds in a Delphi study is a known challenge.
[30] Nonetheless, the outcomes of the process are strongly shaped by those most engaged, which is underlined by the fact that the number of individuals with more than ve publications remained the same during both the consensus seeking round, meaning that those with more 'expertise' remained engaged through the two rounds.
Another possible limitation is of course non-respondent bias. Consequently, there may be important research priorities for some settings that are not re ected here.

Conclusion
To the best of our knowledge, this is the rst study in SSA to explore HPER priorities for the region. We did so through leveraging the input of regional experts and researchers. The research was premised on the assumption that having a shared agenda could build evidence that is regionally relevant while facilitating the e cient and appropriate allocation of resources. Our challenge going forward will be to see the effective translation of this priority setting activity into education research policy and practice.
Notwithstanding the challenges identi ed in terms of policy and funding, individual institutions in the region can re ect on these priorities as they seek to establish research strategies for themselves. Such strategies can intentionally look to engage regional partners as a rst step to growing a larger network of Flow diagram documenting the Delphi process