The impact of rural outreach programs on medical students’ future rural intentions and working locations: a systematic review

Background Significant investment has been undertaken by many countries into ‘Rural Clinical Training Placement Schemes’ for medical students in order to deal with shortages of trained health care professionals in rural and remote locations. This systematic review examines the evidence base of rural educational programs within medical education and focusses on workforce intentions and employment outcomes. The study provides a detailed description of the methodological characteristics of the literature, thematic workforce outcomes and key related factors are identified, study quality is assessed, and the findings are compared within an international context. Methods A systematic review looking at international literature of rural placement programs within medical education between January 2005 to January 2017 from databases including; Medline, Embase, NursingOVID, PubMed and Cochrane. The study adopted the PRISMA protocol. A quality assessment of the literature was conducted based on the Health Gains Notation Framework. Results Sixty two papers met the inclusion criteria. The review identified three program classifications; Rural Clinical Placement Programs, Rural Clinical Placement Programs combined with a rural health educational curriculum component and Rural Clinical School Programs. The studies included were from Australia, United States, Canada, New Zealand, Thailand and Africa. Questionnaires and tracking or medical registry databases were the most commonly reported research tools and the majority were volunteer programs. Most studies identified potential rural predictors/confounders, however a number did not apply control groups and most programs were based on a single site. There was a clear discrepancy in the ideal rural clinical placement length. Outcomes themes were identified related to rural workforce outcomes. Most studies reported that an organised, well-funded, rural placement or rural clinical school program produced positive associations with increased rural intentions and actual graduate rural employment. Conclusions Future research should focus on large scale methodologically rigorous multi-site rural program studies, with longitudinal follow up of graduates working locations. Studies should apply pre-and post-intervention surveys to measure change in attitudes and control for predictive confounders, control groups should be applied; and in-depth qualitative research should be considered to explore the specific factors of programs that are associated with encouraging rural employment. Electronic supplementary material The online version of this article (10.1186/s12909-018-1287-y) contains supplementary material, which is available to authorized users.


Background
Rural populations generally have poorer health outcomes, higher mortality rates, lower life expectancy, increased hospitalisation rates, greater chronic disease and higher cancer rates [1,2]. Factors attributed to these health outcomes include; lifestyle, behavioral factors and lower socioeconomic status. However, another key factor is the lack of effective health services in rural locations and the difficulty in attracting a skilled health workforce to work rurally [1,[3][4][5].
In the provision of medical services, approximately one half of the world's population lives in rural areas but these areas are served by only 25% of the total physician workforce [6]. In the United States (US), 20% of the population lives in rural locations; however only 10% of physicians practice in these same locations [7]. In Australia, it is reported that there are 58 practitioners per 100,000 people in remote Australia compared to 196 per 100,000 in metropolitan areas [8]. These workforce issues are of international concern.
Commonly reported barriers in encouraging health professionals to work rurally are: reduced access to continuing medical education, limited professional interaction with peers, heavy responsibilities and workload, substandard medical equipment and facilities, inadequate financial remuneration, social isolation, poor social services, a lack of job opportunities for partners, and inadequate educational opportunities for children [9][10][11]. Reported factors that encourage interest in rural employment are: a welcoming community, partner employment, family located in a rural area, and the outdoor lifestyle [12]. Furthermore, it is generally accepted that health professionals who grew up in a rural area are more likely to practice in rural locations [13][14][15][16][17].
Attempts to increase the rural health workforce have included; recruitment of qualified doctors from overseas [18,19], workplace reform [18], improved rural working conditions [18], loan repayment schemes [5,20], and financial incentives [18,21]. However, these strategies have achieved mixed results, and over the last two decades there has been a focus on clinical training and education in rural areas, to encourage health professionals to work permanently in rural locations [22][23][24]. For example, in 1997 the Australian government funded an educational initiative, called the 'Rural Undergraduate Support and Co-ordination Program (RUSC)' [25]; designed to improve curriculum design, rural placements and rural teachers. Furthermore, in 2008, the Australian Federal government committed a $1.1 billion investment in the Health Workforce, which included $500 million in Commonwealth funding for undergraduate clinical training and the establishment of Rural Clinical Schools (RCS) [26].
Ranmuthugala et al. [27] reviewed the evidence of rural exposure on rural medical practice and found that the evidence is inconclusive, as the aspects of rural exposure that are driving positive attitudes toward rural practice are not being identified. They stated that longitudinal analysis is required of these government initiatives aimed at driving workforce re-distribution, and there should be a focus on the structure of these programs in terms of the aspects and factors that impact on rural workforce. A 2017 systematic review focussed on rural training programs in the US and supported the need to further explore the factors of the programs that are contributing to rural practice, with the information being critical for informing strategists and planners. The study stated that currently programs are showing promise but require further refinement and the specific aspects of the training experience that are leading to the program's success, are poorly understood and need further research [28].
This systematic review considers the current international evidence within medical education and focusses on the how these programs are being conducted and evaluated, and the value of rural clinical placement schemes; specifically, if these programs encourage graduates' intentions to work rurally and/or lead to actual rural employment. It is hoped by reviewing the methodological characteristics of the programs, the workforce outcomes and related factors, that this will explore the program designs and aspects that are producing positive workforce outcomes.

Methods
This study adopted the quality appraisal tool and guideline for systematic reviews; 'Preferred Reporting Items for Systematic reviews and Meta-Analysis' (PRISMA) protocol instructions, as a guide in the development of the reviews methodology [29,30].

Search strategy
The databases searched were Medline, Embase, NursingO-VID, PubMed and Cochrane. As PubMed and Cochrane are not OVID databases, the same terms were used as outlined in the example of the search strategy shown in Fig. 1. The Search Strategy provided was applied, with slight variations to fit within the structure of the OVID databases (Embase, Medline, NursingOVID).
The following key words and combinations were used in the search strategy: rural placements, clinical placements, rural health, rural school, rural hospitals, rural health services, rural, outreach, rural health program, rural initiative, education, medical, graduate, students, internship and residency, dental health, dental program, rural doctor, medical education, rural students, rural dentists, rural intentions, rural internships, rural clinical internships, workforce outcomes, working locations, student intentions, nursing, continuing education, longitudinal, nursing graduate/ rural, outplacement, employment, dental/medical graduates/ employment, clinical clerkship, placement effectiveness, intervention effectiveness, rural strategies and rural employment. This paper presents the findings related to 'Medical Education Studies'.
All five databases were searched for the period January 2005 to January 2017 (inclusive). Articles available in the databases up to the search date, were included according to the inclusion criteria as stated in Table 1. The search strategy criteria was developed by author one (GJ) who then presented and discussed with the other authors and made modifications as developed through a consultation process (CFW & KF). Figure 2 shows the systematic flowchart of the paper selection process. Stage one involved one author (GJ) independently assessing the titles and abstracts of all the identified articles, to see if they met the basic inclusion and exclusion criteria. Initially all papers related to rural placements or rural clinical initiatives were considered. This resulted in 395 mixed discipline papers for consideration. All articles selected in this review were in English.
At stage two, a reviewer (GJ) applied the exclusion criteria and removed duplicates and irrelevant articles. The two key inclusion criteria were that papers had to consider rural intentions, and rural interest and/or employment outcomes to be considered for further review. In addition, review articles, editorials, or commentaries were removed. We removed reviews as we wanted to assess individual studies/programs and primary evidence within the selected literature, and we planned to compare relevant review evidence within the discussion. At stage three, all non-medical education papers were removed, giving 101 medical education papers.
From this process, 46 papers were selected. The reference lists of all publications were searched and appropriate papers were included in the final review. Any papers that were added from the reference list review also had their reference list searched and again appropriate papers were included (GJ). This resulted in 62 papers included in this systematic review. Additional file 1: Appendix S1 provides a detailed summary of the 62 selected studies.

Data extraction and analysis
The review adopted Pharweller et al. [31] extraction process and analysed information related to the studies methodological characteristics, intervention information and reported outcomes from the articles. Two data extraction sheets were developed (GJ) and pilot tested on three articles. The first sheet collected information on the methodological characteristics and adopted the characteristics applied by Pharweller et al. [31] with modification to be specific for this research area. The second sheet collected extracted outcome themes related to workforce outcomes and followed a thematic content analysis approach of categorising trends in the papers findings.
One author (GJ) applied these extraction data sheets and reviewed all the full articles. A second author (FCW) completed an independent review of six random studies to ensure reliability of the data extraction spreadsheets. To ensure quality control the first reviewer (GJ) completed multiple checks of the quantitative summaries against the literature. Authors two (FCW) and three (KF) also reviewed the completed extraction analysis data sheets and discussions occurred during a final consultative review of the extracted information.

Quality screening process
The review adopted the Quality Assessment process applied by Yevlahova and Satur [32], as it assesses quality for public health and health promotion studies [32]. The model applies a combined schema, including the; 'Type of Evidence Schema' , [33] 'The Health Gains Notation framework' , [30,31] and the 'Cochrane Health Promotion and Public Health Field (CHPPHF) quality assessment screening questions for quantitative and qualitative studies [30,32]. The CHPPHF tool assessed the quantitative studies for internal and external validity and rated the following criteria; allocation bias, selection bias, blinding, data collection methods, consent rate, statistical analysis and intervention integrity [34].
To be specific for the research area, the quality appraisal tools were modified to assess bias by reviewing the additional study characteristics; study type, multischool sites, volunteer program, control/comparison groups, intervention parameters, survey methods and potential for confounders. In addition, the authors (GJ, FCW & KF) considered the outcomes, study length and follow-up period (longitudinal or short-term study) and how this might impact on the significance and generalisability of any findings.
The quality screening process led to 40 studies being classified as 'moderate' and therefore were well reported Studies not focused on strategies/interventions measuring either rural intentions or actual rural employment. (Example, studies that are focused on clinical competence of rural clinical placements were not included unless they related the findings to outcomes related to workforce intentions or actual rural employment). Reviews, commentaries, editorials, news and policy briefs were excluded from the results section, however these papers were considered and discussed in the introduction and discussion of this paper. We did not include programs that were not involved/related to medical education programs i.e. rural initiatives aimed at qualified working professionals unrelated to assessing or considering a current or previous educational initiative. Adapted from D Yevlahova,* J Satur* Models for individual oral health promotion and their effectiveness: a systematic review but the study designs could have been better by applying; larger sample sizes, achieving higher consent/participant rates, reducing risk of selection and recall bias, multi-site programs, applying control groups, or being multi-strategy comparison programs [12,. However, due to the nature of the studies, with the majority being descriptive or observational studies and none being randomised control studies (RCT); 'moderate' was considered in this review as an acceptable standard. Almost all the studies suffered from selection bias as most rural programs were voluntary and many of the retrospective surveys would have encountered recall bias. No studies were classified as 'very strong' , due to a lack of randomisation; however, six were assigned into the 'strong category' [74][75][76][77][78][79] due to their large sample sizes, or multi-site population, high consent or participation rates, clear reporting, control of potential rural confounders through statistical analysis, longitudinal reporting and use of control or applicable comparison groups.
Additional file 2: Appendix S2 provides a detailed analysis summary of the quality review process for the 62 studies. This was a working table to summarise and evaluate each manuscript through a systematic process.

Results
This section presents the trends in the methodological characteristics of the studies, and the outcome themes identified from the data extraction process. Additional file 1: Appendix S1 provides the reader  with more specific summary information of the individual studies characteristics and outcomes. Table 2 provides a quantitative summary of the selected studies methodological characteristics and key aspects of the table are discussed below.

Classification type
The studies were separated into three classifications. The first was a Rural Clinical Placement (RCP) program (twenty seven studies) [35-38, 44, 46, 56, 58, 59, 64-66, 68, 71, 73-75, 81-84, 86, 89-91, 93, 95]. These included any rural clinical placement programs where participants were from a metropolitan based school (the school or study is not based in a rural location and is not classed as part of a rural clinical school).
The second classification were Rural Clinical Placement Programs (same criteria as a RCP) with an additional rural education component (RCP + Education) (12 studies) [12, 50-55, 61, 62, 76, 80, 94]. The additional rural educational component was often rural cultural training, Aboriginal culture and/or extended general rural employment and community information.
Rural Clinical School Programs (RCSP) were the final classification (twenty-six studies) and this includes involving schools located within a geographic rural location, and/or studies/schools self-identified as part of a rural clinical school (irrespective of their location.) (For example, a RCS may be located in a metropolitan location but may be termed an RCSP, as they have rural campuses and rural initiative programs) [39-45, 47-49, 57, 58, 60, 63, 64, 67, 69, 70, 72, 77-79, 85, 87, 88, 92]. One study directly compared a RCSP to a RCP [44] and two studies looked at large populations that related to both RCP and RCSP programs, hence these programs have been placed in both classifications [58,64].

Data analysis
The quantitative analysis methods were classified; either by adjusting for rural confounders/predictors within the statistical analysis model, or not adjusting for them. Over three quarters of the studies were noted as adjusting for one or more potential rural confounders within the statistical analysis methods [ studies were between four weeks and up to 12 weeks [12, 38, 44, 46, 50-52, 54, 58, 66, 91].
Four of the strongest studies identified rural background as an important predictor of rural employment [74,76,77,79] and one of the group reported that rural background is a more predictive factor of rural intentions than an educational placement experience [77].
Other potential predictors of rural employment or associated factors were identified as; family medicine [50-55, 58-62, 75] and primary care [50-53, 62, 67, 72, 75-77], and a recent 2016 paper reported to be the first study to demonstrate that being in a graduate entry program or having dependent children, is negatively associated with working rurally [71].

Discussion
This review has provided a timely update on the literature related to rural outreach schemes in medical education and workforce outcomes. The review has presented the methodological characteristics of the identified evidence base, identified workforce outcome themes and reviewed the quality of the literature. By drawing out the methods the studies applied, it has allowed program implementation and evaluation recommendations to be made, and the extracted workforce outcome themes provide a detailed presentation of the key workforce factors related to the identified rural initiatives.
This discussion will present the methodological characteristics of the literature, the workforce outcome themes and identified workforce factors; and then compares the findings with other current review evidence, in order to consider implications and make recommendations.

Methodological characteristics
The high number of Australia and US studies supports Dolea, Stormont and Braichet [17] who reported that workforce studies come from high income countries. An explanation for the prominence of studies in Australia may be due to the government focus on addressing rural workforce shortages through the RUSC program and significant government investment in undergraduate clinical training in 2008 [25,26].
Much of the evidence consists of descriptive crosssectional studies, longitudinal tracking projects and a number of large cohort studies. No RCTs are identified, and while this methodological standard would reduce selection bias, it would be difficult logistically; and ethically challenging to make educational interventions mandatory and assign students to specific groups for sampling robustness [96]. Victora, Habicht and Bryce [97] reported that RCTs are often inappropriate for the assessment of the impact of large scale complex public health interventions and evidence-based public health must go beyond the RCT model.
Most studies involved single University/site programs and were often small to moderate sample sizes of the intervention group, limiting the power and generalisability of the findings. There was a lack of pre-questionnaires to measure important attributes such as rural background, rural intentions/interest and career intent, prior to intervention. However, a number of studies had pre-admission criteria to the programs and collected important participant information from alumni records. Pre-and post-survey data and post-graduate workforce follow-up is, in general, a more robust method of determining longitudinal change in student attitudes and provides important participant information to assist in controlling for potential rural confounders. It is noted however in the larger retrospective programs looking at workforce trends across large populations and multiple programs, that pre and post in-depth survey evaluation is not always applicable or possible.
A considerable number of studies did not use control groups, which are considered essential to Cochrane-type evidence-based research analysis to effectively measure change and the magnitude of effect, and to minimise the impact of potential confounding variables except for the independent variable [17]. In addition, the majority of the programs involved voluntary participation and this can lead to self-selection bias, with volunteer respondents more likely to favour rural employment and respond positively to the anticipated rural workforce outcomes. In addition, it was noted that the majority of studies did not provide information on the specific evaluation tools applied, such as questionnaire format/ style and this limited this reviews ability to report on evaluation methods applied, such as the application of Likert Scales, or questionnaire structure. Furthermore, a number of studies did not provide a detailed explanation of the intervention and specific program aspects, which is important to assess, or for successful programs to consider adopting. This finding supports prior evidence which has reported that rural immersion research has been mainly focussed on outcomes, with limited description of aspects such as program design and student selection [27,98,99].
Longitudinal tracking programs were commonly reported in the review, however the length of the follow up varies considerably, with some looking at intern choices, directly after graduation and others tracking longer term workforce outcomes. This finding supports a 2018 scoping review of Australian immersion programs, which stated that the time point at which the follow up of graduates working location occurs is important. The study provided an example, as it stated internships do not reflect 'graduate choice' of employment location, but rather it is a matching allocation process based on student and hospital preference and it occurs within a competitive state-based system [100]. Furthermore this review identifies studies are often only measuring workforce outcomes at one point in time and therefore there is a lack of rural workforce retention data. The need for longitudinal programs is supported by Humphreys et al. [101] who comment that well conducted, purpose built, ongoing studies with longitudinal data are required to produce robust medical workforce planning evidence. Pharwaller et al. [31] stated there is a need in rural workforce programs for multifaceted strategies organised within coherent longitudinal programs.
The most reported program duration (placement or RCSP) was between six months and one year. The number of year-long programs in Australia can be attributed to Australia implementing a policy in 2000, to fund medical schools to select 25% of students from a rural background, and also for 25% of students to participate in at least one year of rural clinical training [102,103]. A number of studies in this review report that increasing the length of the rural exposure increases the association of rural influence in terms of rural intentions/rural employment. Previous studies suggest that the ideal placement length and timing is still unknown and needs further exploration. The mixed lengths in this review support the need for further investigation, but the majority of evidence leans towards longer placements/programs (more than 6-12 months) being associated with positive rural workforce outcomes [27,99,104].
Ranmuthugala et al. [27] assert that studies fail to adjust for critical independent predictors of rural practice, and programs need to systematically investigate specific aspects of the rural experience to identify the factors that create a positive impact on trainee medical practitioners. An Australian review [103] states that it is challenging for research to isolate the independent causal effect of rural programs on rural employment. This review reports that improved data collection methods and statistical analysis in the control for potential predictors, is now occurring in a larger number of studies. However, within the literature there is little consistency in the application of these rural predictors and the statistical modelling applied, and it is proposed that identifying set specific predictors for future programs may produce more consistent and higher quality research.

Key outcome themes and identification of potential rural workforce predictors
The positive rural workforce themes reported in this review supports a 2008 systematic review by Rabinowitz et al. [105] which identified a multi-fold increase in the rural physician supply and reported strong evidence that medical schools have significant potential to address the workforce mal-distribution. The study also stated the importance of identifying any positive workforce outcomes from effective programs, as even a small number of rural clinicians can have a critical impact on a rural location (due to ratio of clinician to rural population numbers), as one rural clinician can have a major impact on access to care for rural communities. They provide an example of the PSAP program which has on average 14 students participate per year but leads to 12% of rural family physicians in the state of Pennsylvania [105,106]. The positive associations between rural programs and rural employment identified in this review, supports a 2018 review which reported that Australia's immersion programs are moderately associated with increased supply of junior doctors into rural locations [100].
Rural background is deemed an important predictor of rural intentions and/or rural employment by more than a third of the studies. Furthermore, the majority of the highest quality studies deemed it a key predictor of rural intentions and or rural employment, and that it should be factored and considered within rural program planning. Two reviews state that being from a rural background is the strongest predictor of choosing a rural practice location [28,107]. In addition, an independent Australian Government review recommended that students with a rural background should be preferentially offered places to improve health workforce outcomes [108]. However, a recent review comments that due to the lack of students from rural backgrounds this means that rural background recruitment should not be the primary policy for addressing rural workforce shortages, and interventions need to target all students [28]. Playford et al. [109] reported that more is needed than selecting students with a rural background and while a rural intent is important, combining with rural exposure during training increases the likelihood of rural employment [109].
It is clear that rural background is an important aspect for program planners to consider, however our review identifies there are other key potential rural predictors including; rural interest/intentions prior to the program, generalist practice intentions, an interest in primary care and family medicine, financial and rural bonded scholarships and importantly the type and quality of a rural immersion experience and its duration. The control for rural predictors is crucial and is supported by a recent (2017) evaluation of a rural clinical school program, which reported that rural background, rural intention and rural experience during medical school, all need to be incorporated within future workforce programs [109].
In addition, a 2017 systematic review identified an association between family medicine focus and primary care interest, with rural practice [28]. It is clear there are a wide range of program parameters to consider in the development of a rural program experience.

Study limitations and strengths
The initial searches in this study used key words, one of which led to 454,000 results. Multiple combinations of key words were therefore applied to increase specificity. It was expected that using educational search terms would lead to a high number of results as was encountered by Crampton, McLachlan and Llling [96], who reported that the balance between sensitivity and specificity is a complex challenge with modern systematic reviews. Also, a large search study base may lead to errors and the chance of missing relevant studies. However, the ease of use of modern databases and their comprehensive search engines means that relatively accurate systematic searches are possible. Multiple databases were searched, and findings were limited to published papers considering education evaluation programs related to rural intentions and workforce outcomes. The reference lists of all manuscripts analysed in the review were studied and applicable papers included to attempt to reduce the likelihood of missing relevant studies.
A strength of this review is that despite a large range of interventions, methods and quality of studies, the findings were generally similar and reported positive rural clinical experiences, increased rural intentions, and positive associations with increased rural employment. In addition, the quality of the interventions were assessed; and a general score was applied over a more specific system [32][33][34]. A further methodological strength of this review is that we used the PRISMA protocol instructions as a guide in the systematic review structure [29,30]. In regards to the quality review, the authors would like to comment on the challenges of categorising such wide ranging studies within three broad categories and ascertaining the weight of certain study characteristics on the quality of a manuscript. The authors mitigated this challenge through a systematic transparent approach (Additional file 2: Appendix S2), discussions of contestable papers and attributed caution and lower categorisation when a manuscript appeared to cross multiple categories.

Implications
The breadth of different programs this review identifies, demonstrates uncertainty about the types of rural educational programs that are most effective in terms of placement length, timing and frequency, and that program planners are continuing to refine and build larger and more ambitious longitudinal research programs [27]. The review highlights some study design limitations being applied in these medical education programs, with a large number of studies not using control groups or pre-questionnaires to identify important participant characteristics and views on rural interest/employment, that could provide important data for statistical modelling. Furthermore, the review notes the lack of clarity on the potential predictors of rural employment that need to be effectively controlled for and managed, and the lack of consistency in the application of these potential predictors across the literature. The majority of the evidence focusses on descriptive, cross sectional and cohort research findings of any significant positive associations, rather than conclusive causal factors in health workforce predictions. Nevertheless, it can be concluded that well designed and financially supported rural clinical placement programs and rural clinical schools do have a positive association with rural practice location, and that these programs warrant continued investment and further longitudinal review, with longer term retention information of graduates' employment history a key area that requires further investigation.
It is only with the availability of more rigorous evidence and sufficient political commitment that we will be able to address the pressing issue of equitable healthcare delivery in rural locations and identify winning strategies to guide future practice and policy. Based on the reflection of the literature a number of key recommendations have been made below.

Key recommendations
Future studies should focus on more methodologically rigorous longitudinal studies with measurements at multiple time points and report on graduate rural employment retention, with an aim to, isolate and measure the impact of the individual predictors that influence the career choice of medical students/graduates. In-depth qualitative research may further assist in exploring the intrinsic and extrinsic predictors driving rural intentions and actual rural workforce outcomes. This additional information would help to contextualise the larger scale quantitative data being reported. Programs should incorporate pre and post surveys to assess the change in participants after the program intervention, and to collect important participant characteristic information (e.g. preprogram rural interest and rural intentions). Future programs should analyse associations and control for potential rural workforce predictors through statistical analysis (e.g. multivariate statistical modelling). Programs need to be more consistent in the rural predictors (confounders) they aim to control and should include; rural background, rural experiences prior to the program and pre-placement/program rural intentions. In addition, programs should consider controlling for and further investigating the impact of potential confounders including; student's interest in primary care, family medicine and generalist practice, bonded scholarships, graduate schemes and graduates with child dependency. All future studies should apply control (preference on specific control groups) or comparison groups to effectively measure the change and impact of an intervention. Rural background is a significant, independent predictor of workforce choices and should be considered within all rural placement research. Considerable evidence supports selecting rural background students for medical schools and multiple studies have identified combining rural background students with a rural educational initiative as a more powerful means of encouraging rural employment. However, given the dearth of rural background students compared to metropolitan background students, rural educational initiatives need to also target both groups, especially as a number of studies have identified that an effective rural education intervention, is an independent predictor of rural employment outcomes. Further investigation is required to ascertain the optimum program period length; however, the strongest evidence in medicine trends toward periods of six months or longer to provide a significant rural experience. Due to the large number of single site developed and managed RCP and RCSP identified in this review, it is advised that medical institutions consider greater collaboration between schools and other partners; in order to develop multi-site programs to increase the impact, power and generalisability of the evidence base. Larger scale multi-site programs should provide more powerful data to help inform national policy.

Conclusions
The review's detailed presentation of the studies methodological characteristics, outcomes and recommendations, makes it useful to inform future research on rural placement programs. Investigators need to record the factors driving the success of programs through applying pre-and post-placement surveys, control groups, and large scale methodologically rigorous longitudinal cohort studies which look at workforce outcomes at multiple time-points. Programs should also consider partnering up to expand beyond single institution programs to enhance the power and generalisability of the evidence. In addition, explorative and in-depth qualitative research should be considered to explore the predictive/causal factors within successful programs.
The goal must be to establish a successful and repeatable design model for rural clinical programs that can be replicated and implemented on an international or national basis. In addition, policy advisors and future placement program developers should consider the evidence presented in this review, as a guide of what is working and where the evidence base is currently positioned.

Additional files
Additional file 1: Appendix S1. A summary of the 62 selected studies. Provides the reader with summary information of the 62 selected studies specific characteristics and outcomes. (DOCX 77 kb) Additional file 2: Appendix S2. A summary analysis of the quality review process for the 62 selected studies. Provides a detailed analysis summary of the quality review process for the 62 studies. This is a working table to summarise and evaluate each manuscript through a systematic process. (XLS 263 kb)