Efficacy of evidence-based medicine training for primary healthcare professionals: a non-randomized controlled trial

Abstract

Background

The impact of evidence-based medicine (EBM) training techniques in primary healthcare professionals remains to be determined.

Methods

A non-randomized controlled trial (NRCT) was performed aiming to assess the two methods of evidence-based medicine training for primary healthcare professionals by assessing evidence based practice (EBP) related knowledge (EBP-K), attitude (EBP-A), personal application (EBP-P), anticipated future use (EBP-F), and community management of hypertension. Participants were recruited and assigned to either an EBM training group that receiving a weekly face-to-face EBM training course, or an EBM self-instruction course for eight weeks. A validated instrument was applied to evaluate the four aspects of EBP. Additionally, community management of hypertension was assessed by comparing the the rate of detection, blood pressure control, standard management, grading management and patient satisfaction between 2015 and 2016 to measure training efficacy. The difference between the impact of these two interventions was assessed statistically.

Results

One hundred fifty-one participants (69 in the face-to-face EBM training group and 82 in the self-instruction group) were included. Compared to self-instruction, the face-to-face EBM training was associated with significantly improved EBP-Knowledge (26.14 ± 4.22 vs. 22.44 ± 4.47, P < 0.05), EBP-Personal application (22.52 ± 6.18 vs. 16.89 ± 5.99, P < 0.05), and EBP-Future use (44.04 ± 8.97 vs. 37.71 ± 8.39, P < 0.05). EBP-Attitude scores (10.89 ± 4.52 vs.14.93 ± 5.92, P < 0.000) were lower in the EBM training group. Stratified analyses showed that the results were consistent regardless of the participants’ gender, professional role (doctors & apothecaries or nurses), rank (junior or senior doctors & apothecaries), or specialty (Traditional Chinese or Western Medicine). Assessment of community hypertension management revealed that the rate of blood pressure control, standardized hypertension management and patient satisfaction was significantly better in group A than group B (1.14% vs.0.69, 2.85% vs.1.68 and 2.41% vs.0.84%).

Conclusions

A face-to-face EBM training course improved primary healthcare professionals’ EBP knowledge, attitudes, personal application, and anticipated future use. Effective EBM training may improve the efficacy of primary health care services.

Trial registration

Non-Randomized Controlled Trial ChiCTR1800017498, August 1, 2018.

Background

Evidence based medicine (EBM) is defined as a conscientious, explicit, and judicious use of the current best evidence to inform decisions about the care of individual patients [1]. The positive impact of EBM on clinical practice has been established in healthcare practice all over the world, including in low- and middle-income countries [2,3,4]. In developed countries, with improved medical education and advanced teaching techniques, great efforts have been made to improve EBM education of medical students and other health care professionals [5,6,7]. The importance in EBM education is increasingly recognized and efforts have been made to improve the quality of education through the introduction of innovative teaching and training models [4, 8, 9]. It is well accepted that EBM education should be integrated with clinical practice [10], and should be evaluated and guided by evidence of its own effectiveness [11]. Previous research into EBM education and training have demonstrated that EBM knowledge and skills can be improved through various medical school-based education models [12], resident training programs [13] and continuing medical education programs [14]. The training methods employed by these studies typically include extended, self-learning, faculty mentored instruction [12] and a mixture of interactive lectures, workshops and case-based studies [13]. For example, a methodologically sound research conducted in a developing country shows that clinically integrated e-learning EBM curriculum compared with a self-directed EBM course resulted in higher knowledge and skill scores and improved educational environment [3]. Although various strategies, including literature searching education and a blended learning education, have been applied in the teaching of EBM [15,16,17,18,19], the precise impact of these strategies on quality of evidence-based practice (EBP) remain to be determined [20,21,22].

It has been suggested that EBM curricula should be developed into a 5-step model including translation of uncertainty to an answerable question, systematic retrieval of the best evidence available, critical appraisal of the clinical relevance and applicability of evidence, application of results in practice, and evaluation of performance [2]. Also, it has been noted that EBM educational programs should be designed at an appropriate level for the trainees to optimize learning efficacy [22]. The efficacy of EBM training in undergraduates and residents is established, however EBM training in physicians is less well studied [23]. In mainland China, EBP skills have not traditionally been covered in continuing medical education for primary healthcare practitioners. The optimal strategy for teaching EBM to primary healthcare professionals such as general practitioners (GPs) and nurses, remains to be determined.

Thus, in this study, we carried out an entry-level EBM training program based on the 5-step model for primary health care professionals. The clinical context and specific objectives of the study were:

• 1. To compare the impact of a face-to-face EBM training strategy and EBM self-instruction on a validated instrument definition of EBP knowledge (EBP-K), attitude (EBP-A), personal application (EBP-P), and future anticipated use (EBP-F), and to develop optimal EBM learning strategies in primary healthcare professionals; and

• 2. To compare the rate of hypertension management (detection, blood pressure control, hypertension grading, standard management and patient satisfaction) achieved by these two community health service centers.

Methods

Trial design

This non-randomized controlled trial was performed with primary healthcare professionals, including doctors, apothecaries and nurses, from the Mi-shi Lane Community Health Service Center and Xiao-he hu-shu Lane Community Health Service Center of Gongshu district in Hangzhou city, and each center has 7 service stations, 106 and 112 medical staffs respectively. Primary healthcare professionals provided basic medical services, prophylactic immunization, women healthcare, and rehabilitation therapy for the community in China. To cater to the professionals working schedule, those who from the same center were allocated to the same group to participate into either a face-to-face EBM training group (group A), or a self-instruction group (group B) by tossing a two-sided digital coin. The participants allocated to group A received a weekly 2-h EBP-structured presentation covering EBP approaches to patient care experiences offered by the EBM faculty team of the second hospital affiliates to Zhejiang Chinese Medical University. The general practice department of the hospital was founded in 2011, and responsible for the tutorship of medical students in the 4th to 5th years of medical school, residents in post-graduate training and continuing medical education for general practice. Participants of group B were assigned to receive a weekly self-instruction course covering the essentials of EBM, which was uploaded by a research assistant through the center’s own network. Blinding and allocation concealment were not possible in the present study because teachers and participants were all aware of the courses they were going to attend. However, study hypothesis had not been disclosed to all participants. This study was based on the provincial fund project, and written informed consent was obtained from each participant. (Additional file 1).

Participants

Primary healthcare professionals were recruited from two Community Health Service Centers, which have intimate association with the second hospital affiliates to Zhejiang Chinese Medical University. To meet the eligibility, participants were required to be:

• 1. doctors and apothecaries (including GPs, therapists, apothecaries, public health professionals);

• 2. nurses;

Participants who were unwilling to participate in the study or did not wish to provide consent were excluded from the study. The following variables were recorded for each participant: age, gender, professional role (doctors & apothecaries or nurses), and rank (junior or senior doctors & apothecaries), or specialty (Traditional Chinese or Western Medicine).

EBM educational interventions and outcomes

The face-to-face EBM course for group A was developed by a physician and an EBM professor at the second hospital affiliated to Zhejiang Chinese Medical University (Yanhua Li and Junwei Li). This course was intended to provide an interactive forum for participants to improve the clinical implementation of EBM. The primary outcome of this study was knowledge, attitudes, personal application, and anticipated future use, which was measured by using the previously validated EBP-KAB tool [24]. Unlike the medical students, the included primary healthcare physicians had excellent clinical skills, but no formal training in EBM. Thus, the educational intervention was designed as a practical and targeted entry-level EBM training course. The physicians were also extremely busy, so the training schedule was adapted, following feedback, to cater to their work, taking only two hours per week. In total, participants in group A received a 16-h EBM course, including 2 lectures, 3 conferences and 3 small group discussions. The training course is outlined in Table 1. Briefly, the introduction session provided an overview of EBM, including the grading and recommendation of evidence. The subsequent four sessions were developed according to the 5-step model of EBM teaching, covering i) construction of a relevant clinical problem, ii) comprehensive literature search of medical databases, iii) critical appraisal and synthesis of evidence, iv) apply the evidence to the practice. The final session described three actual clinical examples of primary care to demonstrate the real-time application of EBM skills. The EBM faculty team included two professors and three novice teachers. All faculty members participated in a team-based teaching model to develop facilitation and interactive teaching skills to promote the practice of EBM. We also paired experienced teachers with three novice teachers. In these sessions, the latter are encouraged to record trainees’ feedback to the leading teachers. Participant feedback was used to adapt teaching in future sessions. EBM faculty members answered participants’ questions during the training period. For participants in group B, a study assistant would help to upload the same curriculum once a week with documents including texts and pictures via the center’s Office Automation platform, which allows users to transfer data, mail and even voice across the network. Then the participants would download the curriculum file and learn by themselves in their spare time every week, in which circumstances participants finished the mission all by self-discipline or their attitude towards to EBM. Meanwhile, the study assistant was able to see the number of downloads and alert the participants who had not downloaded the files to learn in time. Once the participants had any question, they could communicate with each other or send e-mails to the teachers through the platform, and teachers would make a question list and answer it every week through the platform.

Table 1 Evidence-based medicine course curriculum

In China, the government attaches great importance to the management of hypertension, because, if not treated early, hypertension can become a devastating disease with poor prognosis. Hypertension has been highlighted by the Ministry of health, and in order to improve the management of hypertension, patients’ health records must be available to primary health service professionals. As the management of hypertension is dependent largely on early diagnosis and risk stratification, the case of hypertension in the final session was designed to focus on the application of updated evidence-based hypertension screening, diagnosis and control guidelines, (Chinese Guidelines for the Prevention and Control of Hypertension, 2010 edition). Management of hypertension was assessed by comparing the rate of detection, blood pressure control, standard management, grading management and patient satisfaction between 2015 and 2016. All these five items are included in the patients’ health record, and each item was recorded by medical teams at the service stations at each follow-up.

Evaluation instrument

Four principle components of EBM, including EBP-K, EBP-A, EBP-P, and EBP-F [24], were measured before and after EBM training using an instrument previously validated for the assessment of EBM education in the undergraduate learning environment [24]. Assessment questionnaires consisted of 26 questions answered using a six point Likert scale. The EBP-K section included five items (scored from 5 to 30), the EBP-A and EBP-P sections six items each (scored from 6 to 36), and the EBP-F section included nine items (scored from 9 to 54) [24] (Additional file 2). These instruments were applied by assessors blinded to participant group to eliminate performance bias.

Sample size estimation

To estimate the sample size, we assumed that the standard deviation of the two types of intervention was the same when within the two groups, and the probability of alpha = 0.05, beta = 0.10, and the difference between the two increases is 60% of the standard deviation. Known to the delta/sigma = 0.6, the two sides of alpha = 0.05, mu 0.05/2 = 1.282. The input formula is N1 = N2 = 60, so the two groups need 120 people to detect a statistically significant difference. A 10% drop out rate was estimated, so at least 66 participants were required for each group.

Statistical analyses

All statistical analyses were conducted using SPSS version 17.0. Continuous variables were presented as means and standardized deviations (SDs) if normally distributed; otherwise, median and interquartile range (IQRs) was presented. The categorized variables were presented as numbers and proportions. Differences in continuous variables and categorized variables were analyzed by independent t-test and chi-square test, respectively. Percentage change in scores was compared using the Mann-Whitney test due to their non-normal distribution. Percentage change = (post-course score – pre-course score)/pre-course score *100%. Stratified analyses were preformed according to participants’ gender, professional role (doctors & apothecaries or nurses), and rank (junior or senior doctors & apothecaries), or specialty (Traditional Chinese or Western Medicine). All statistical tests were 2-tailed, and P < 0.05 was considered to indicate statistical significance.

Results

The flow diagram of the study participants throughout the trial is outlined in Fig. 1. Finally, excluding the 12 participants who were not meet the inclusive criteria, we recruited 157 physicians and nurses from these two centers and defined participants from the same center as a “group” for feasibility, and 72 physicians and nurses from Mi-shi Lane center allocated to the trial group (group A), Of those, 69 (95.83%) completed the EBM assessment. Reasons for exclusion included personal attributes leading to lack of timeliness for advanced studies. 85 from Xiao-he Hu-shu Lane center attended to the control group (group B), 82(96.47%) completed the EBM self-learning. Reasons for not completing the exercise included technical difficulties with the online program and not filled out all the information in the post assessment.

Fig. 1

Participant inclusion flow chart

Participants’ characteristics such as age, gender, professional role (doctors & apothecaries or nurses), and rank (junior or senior doctors & apothecaries), or specialty (Traditional Chinese or Western Medicine) did not differ significantly between the groups (Table 2).

Table 2 Participant Characteristics (chi-square test)

EBP was assessed before and after the eight-week course, and changes in the scores for each domain are shown in Table 3, and percentage change in four EBP domains scores in Table 4. For group A, significant changes were observed in the before and after comparisons for all four domains of assessment (all P < 0.01) (Table 3). The increase was most pronounced for EBP-P and EBP-K, followed by EBP-A and EBP-F. For group B, consistent results across all four domains of assessment indicated that self-instruction method rarely changed EBP outcomes.

Table 3 Percentage change in four EBP domains scores

Table 4 Assessment of EBM pre- and post- training

Both groups had similar EBP-K, EBP-A, EBP-P and EBP-F scores before the intervention. After the intervention, participants in group A had significantly higher EBP-P (19.52 ± 6.18 vs. 16.89 ± 5.99, P < 0.000), EBP-F (44.04 ± 8.97 vs. 37.71 ± 8.39, P < 0.000), and EBP-K scores (26.14 ± 4.22 vs. 24.44 ± 4.47, P < 0.017) than in with group B. On the other hand, EBP-A scores were significantly lower in group A than B, (10.89 ± 4.52 vs. 14.93 ± 5.92, P < 0.000), indicating an improvement in the attitude to EBM.

Stratified analyses showed consistent results across participant characteristics such as gender, professional role, rank, or specialty (Table 5).

Table 5 Stratified comparisons of EBM scores after intervention

We investigated the management of hypertension as an indirect measure of the clinical effect of EBM training. Our results indicate that the blood pressure control rate, standardized management rate and patient satisfaction was significantly better in group A than group B (1.14% vs.0.69, 2.85% vs.1.68 and 2.41% vs.0.84%) in Table 6. There were no harms or unintended effects in each group.

Table 6 Comparison of standardized management and prevention of community hypertension in each group

Discussion

To our knowledge, this is the first study in mainland China to assess the impact of an EBM training program in primary health service professionals using a validated and reliable tool. Our study results indicate that practice based face-to-face training may be an optimal strategy to educate primary healthcare professionals with the application of EBM.

Currently, EBP is recognized as a core competency that must be acquired by all medical professionals, because it can improve the quality of health care by supporting clinical decision-making [25]. Although recent studies have shown that various initiatives may be effective in improving EBM knowledge, no convincing evidence indicates that teaching EBM also changes professional behavior in practice [26]. This study underlines the need not only to enhance EBM skills, but also to improve the ease of use of EBM resources at the point of care. Thus, an entry-level EBM training program was designed by tailoring evidence-based information retrieval systems to busy clinical schedules.

EBM education should be evaluated and guided by evidence of its own effectiveness [27]. Many tools are available to clinicians. And the Fresno test seems to be a key candidate for assessing the efficacy of EBM training [28]. However, it was inappropriate for our participants due to the high item difficulty in the pilot trial. We finally chose a validated instrument for assessment of EBP-K, EBP-A, EBP-P, and EBP-F. Furthermore, application of EBM in clinical practice must be assessed by measuring participants’ care in acute and chronic clinical situations for which there are clear EBM standards [28]. For this reason, we evaluated the management of hypertension by comparing the rate of detection, blood pressure control, standard management, grading management and patient satisfaction between 2015 and 2016. Moreover, this data was available in patients’ health records.

In this study, we found that an eight-week face-to-face EBM training program improved nurse and physician EBP knowledge, attitudes, personal application and anticipated future, and these measures improved more significantly than a self-instruction training program. Subsequent stratified analyses showed that results were consistent regardless of participants’ gender, professional role, rank, or specialty, which indicates that these factors did not affect the efficacy of EBM training. In addition, as assessed the impact of this training on management of community hypertension and found that measures of community hypertension were improved more significantly following face-to-face EBM training than self-instruction training, which indicate that EBM training can significantly improve primary health service staff EBP competency, and contribute to EBP behavior. Lectures, conferences and small group discussions facilitate more interaction between educators and their audience than self-instruction models, suggesting that these interactions may play an important role in EBM training. It might be difficult to gain sufficient EBM knowledge and skills without appropriate guidance by trainers. Face-to-face interaction can provide a superb opportunity for trainees to discuss the problems with professional educators who may provide solutions. The advantage of these methods was observed in all analyzed groups regardless of participants’ gender, professional role, rank, or specialty.

It must be noted that the results of this study may reflect characteristics of particular participants, or the precise training programs applied, and may be limited to interventions of the precise type and duration applied in the current study. In this study, participants were recruited from two community health service centers in Hangzhou city in China. Both sets of participants had the similar EBM background: they had never received formal EBM education but were exposed to EBM through continuing medical education. Second, the EBM training course was designed by EBM professors at the university and teaching hospital, and focused on practical aspects including constructing a clinically relevant question, developing search strategy, assessing the evidence. Third, our training program was implemented for eight weeks. Lengthening or shortening the training duration may change the effects.

Strengths and limitations

This study has some strength. First, results of our study can minimize the probability of confounding and selection bias to some content, which, nevertheless, were quite common in most of the other relevant studies and challenged their findings [29]. Second, we adopted a validated instrument to assess EBP skills. This instrument was developed based on adult learning models, and covered the four key components of EBP. This tool was more comprehensive and practical than other scales which mainly focused on assessing the effects of curriculum on knowledge and skills [30], including subjective questionnaires and objective tools, such as the Fresno test [31], the Objective Structure Clinical Examination station [32], Berlin questionnaire [33], and the assessing competency of EBM tool [34]. Third, the study was sufficiently powered by including adequate numbers of participants according to the sample size estimation.

Some limitations of our study should also be noted. First, this study is not a RCT which may lead to selection bias and we only included two centers in this study for convenience sampling. This may limit the generalizability of our results to other settings in China [35]. Second, the EBM educational intervention applied was based on the curricular design by our university and teaching hospital. The curriculum tends to be more theoretical than practical. Further studies should pay more attention to more practical components, such as how to use evidence in practice and how to re-evaluate the evidence-based practice. The contents in future training should also be tailored to the audience according to their background and knowledge when the training program is designed and implemented. Third, long-term outcomes should have been assessed to investigate the duration of the interventions’ effects on clinical practice. Finally, the training duration, strategies for interactive response and feedback, as well as other details should be optimized in future implementation of face-to-face EBM training.

Conclusion

In conclusion, a face-to-face EBM training course may improve EBP knowledge, attitudes, personal application and anticipated future use in primary healthcare professionals. Further studies will be required to confirm our results and to optimize the implementation of the face-to-face EBM training.