Skip to main content
Download PDF

Learning results of GP trainers in a blended learning course on EBM: a cohort study

BMC Medical Education volume 15, Article number: 104 (2015) Cite this article

Abstract

Background

General practitioners (GPs) experience barriers to the use of evidence-based medicine (EBM) related to a negative attitude and to insufficient knowledge and skills. We therefore designed a blended learning intervention to develop the competence of GP trainers in EBM. This study investigated the effectiveness of this intervention in increasing the trainers’ EBM competencies (i.e. knowledge, skills, attitude and behaviour).

Methods

In total 129 GP trainers participated in the blended learning course on EBM consisting of four 3-h face-to-face meetings and an intensive preparatory e-course before each meeting over a 12-month period. The primary outcomes were changes in knowledge and skills (Fresno test), changes in attitude (McColl test) and intentions to change behaviour. Secondary outcomes were changes in self-rated knowledge, skills and attitude, and the relation between personal characteristics and changes in knowledge, skills and attitude. Data were collected before the start of the intervention (T0), at the end of the last day of the intervention (T1) and four months after the end of the intervention (T2).

Results

The mean changes in scores on the Fresno test were ∆T1-T0 = 40.8 (SD ±36.7, p < .001) and ∆T2-T0 = 20.8 (±39.9, p < .001). The mean changes in scores on the McColl test were ∆T1-T0 = 2.2 (SD ±12.8, p = .16) and ∆T2-T0 = -.87 (±10.0, p = .49). Of the GP trainers, 16.7 % fulfilled their intentions to change in behaviour, 47.6 % partly fulfilled them and 35.7 % did not fulfil them at all. Female trainers scored significantly higher on the Fresno test after the intervention compared to male trainers. There was a weak positive correlation between self-rated knowledge and the scores on the Fresno test. A moderate correlation was found between the overall score on the McColl test and self-rated attitude.

Conclusion

An intensive blended learning course on EBM for GP trainers induces an increase in knowledge and skills that, although decreased, remains after four months. Attitude and behaviour towards EBM show no differences before and after the intervention, although GPs’ intention to use EBM more often in their practice is present.

Peer Review reports

Background

General practitioners (GPs) experience several barriers to the use of evidence-based medicine (EBM) [1] related to a negative attitude towards the evidence itself and to the use of EBM in their clinical practice, as well as to insufficient knowledge and skills. They also experience barriers related to their practice and patient population [2], such as a lack of time, patient-related factors and a lack of available evidence. All these barriers [3] are also experienced by GP trainers despite the additional education they received to become trainers. Because GP trainers function as important role models for GP trainees [4, 5], modelling adequate EBM behaviour requires additional education to enhance their EBM competency (i.e. knowledge, skills, attitude and behaviour) [6].

GP specialty training in the Netherlands has a duration of three years. In the first and third years, the trainee works at the practice of a GP trainer. This GP trainer is also the daily supervisor of the trainee. In the second year, the trainee works for periods of three or six months in various clinical settings, such as a psychiatric care hospital or institute, an emergency department or a nursing home. New GP trainers (all of whom have at least five years of clinical experience as GP) follow a four-year training programme that mainly focuses on didactic competencies. EBM as a separate subject is currently not part of this training programme. After these four years, a GP trainer is required to follow eight days (48 h) of refresher courses each year at the training institute.

In today’s healthcare, EBM is considered essential for providing high quality patient care [7, 8]. Practising EBM requires the integration of clinical expertise, patient values and the best available evidence into the decision-making process for patient care [9]. Integrating these three elements into daily practice requires knowledge about and skills in EBM as well as a positive attitude [7, 9].

Fishbein and Ajzen’s [10] theory of reasoned action states that attitude and subjective norms (the influence of others in one’s social environment) are predictors of behaviour. Following this theory, to actually enhance the practice of EBM requires a change in attitude that will result in willingness to change behaviour.

Using education to change the EBM behaviour of GP-trainers thus requires a specified training program. These days, continuing medical education (CME) offers not only face-to-face (F2F) courses but also e-learning (e-courses) and, as the latest development, blended learning courses. E-courses are becoming more popular because of the didactical advantages they offer, such as the possibility to adapt them to individual learning styles [11] and paces, as well as the possibility to repeat courses [12, 13]. The logistic benefits, such as learning at any convenient time and place, are also seen as important advantages of e-learning in the context of CME [14–16]. A study by Kok and colleagues [17] among physicians in a non-hospital based medical specialty concluded that multiple educational methods should be used to change a physician’s behaviour.

Another study among occupational physicians [18] concluded that e-learning alone will increase EBM knowledge but it is not effective in increasing EBM skills and changing EBM behaviour. They recommended a blended learning course. Blended learning includes ‘different learning or instructional methods, different delivery methods, different scheduling and different levels of guidance’ [19]. These learning environments have both didactical and logistic advantages, while also offering learners interaction with peers on complex issues related to behaviour and professional role, and bridging the gap between theory and practice [12, 13, 20]. Blended learning also allows practice to be incorporated more fully into the learning pathway by including practice-based assignments [21].

Ilic and Maloney [22] performed a systematic review including EBM education in several kinds of educational modes for undergraduate and postgraduate medical students. Although they suggested that the findings of their review ‘provides educators with the option of adopting a flexible curriculum in EBM’, in their conclusion they stated that ‘it is not possible to determine which kind of intervention is most effective at increasing EBM competency at this point in time’ and that further studies are required. In a systematic review by Coomarasamy and Khan [23] they stated that EBM teaching should be incorporated into clinical teaching.

On the basis of the above considerations, we designed a blended learning intervention to enhance the EBM competency of GP trainers. In earlier studies, internet-based learning is compared with no educational intervention and/or traditional methods [24], and shown to be as effective as traditional methods. The primary aim of the present study therefore was to assess the magnitude in which this intervention increases GP trainers’ EBM competency. The secondary aims were to explore the relation between formal assessments as outcome measures and self-reported outcomes, and to determine whether specific characteristics of trainers are related to the outcomes of the intervention. We have designed this study as a cohort study as this design suited our objectives well.

Methods

Participants

122 of the 170 GP trainers at the Academic Medical Center, University of Amsterdam (AMC–UvA), and a self-selected group of 7 GP trainers at Leiden University Medical Center (LUMC) participated in the study. All the trainers were informed that participation in the study was voluntary and gave their informed consent. Ethical approval for this study was obtained from the Ethical Review Board of the NVMO (Netherlands Association for Medical Education).

Setting

The use of clinical guidelines is common in Dutch GP practices; 100 guidelines developed by the Dutch College of General Practitioners (NHG) are currently available [25] to facilitate the use of EBM in general practice.

Intervention

In 2012, the GP trainers at the AMC–UvA followed a blended learning course on EBM. The course consisted of four 3-h F2F meetings and a 12-h e-course divided into four 3-h modules spread over 12 months. In preparation for each meeting, trainers were asked to complete one of the e-modules and an assignment in cooperation with their trainees. The assignments for the first and fourth meetings were specifically formulated to stimulate interaction between trainer and trainee on EBM. Each of the four modules had its own learning objectives and learning activities. An overview of the educational intervention is presented in Table 1.

Table 1 Content of the EBM training
Full size table

This blended course was based on educational evidence on adult learning and evidence regarding technical aspects influencing the educational effectiveness of e-learning. A detailed description of the development of this course has been given elsewhere [26]. In short, based on our learning goals – which require a change of behaviour in practice – and audience (experienced GP trainers), we used the social constructivist learning theory [27] and the adult learning theory [28] as learning principles for the design of our course.

We chose the four-component instructional design (4C/ID) model [29] to give direction to the design of the e-learning component of our programme. This model consists of four interrelated components, namely learning tasks, supportive information, just-in-time (JIT) information and part-task practice. It was developed for designing education or training programmes that continue for a longer period of time and focus on complex learning, that is, the integration of knowledge, skills and attitude and transfer to the work situation.

The GP trainers at the AMC–UvA were randomly divided into 12 groups of 8 to 12 participants. We had four teachers and each teacher guided two to four groups. The GP trainers at the LUMC followed the same course-content but had their own teacher. All teachers were GPs and experienced EBM teachers and received specific training before the course and between the F2F sessions.

Study design

This cohort study was performed between February 2012 and April 2013. The study design was chosen because of the possibilities to study changes over time. Personal characteristics and competency (knowledge, skills, attitude and intended behavioural change) were assessed using questionnaires at three moments: before the start of the intervention (= T0), at the end of the last day of the intervention (= T1) and four months after the intervention (= T2). After the second and fourth meetings, the participants were asked to describe three intentions for behavioural change, as these are known to be strongly related to actual behavioural change [10].

Primary outcomes

The first questionnaire sought personal characteristics namely gender, age, year of graduation, experience with research and experience as a trainer. The other questionnaires used are described below and assess 1) EBM knowledge and skills, 2) EBM attitude and 3) EBM behaviour.

  1. 1)

    EBM knowledge and skills

    We used the Fresno test [30, 31] to examine EBM knowledge and skills. This questionnaire has been translated into Dutch and validated for students in allied healthcare [32]. Although the clinical scenarios were adapted for GPs, the same questions and possible answers were used. The questionnaire contains eight open-ended questions with sub-questions, and five multiple-choice questions. Each correctly answered open-ended question generates a maximum of 24 points. The multiple-choice questions are worth between two and four points. The maximum total score is 220 points. The scoring of the open questions of the Fresno test was done in several sessions by three researchers (EtP, MWdW and NvD). During each meeting, scoring was done by one researcher and any ambiguous answers were discussed directly. After all the questionnaires had been scored, a sample of 10 questionnaires from T0, T1 and T2 (a total of 30 questionnaires) were re-examined by another researcher. Scoring of both these questionnaires was compared to assess agreement in scoring.

  2. 2)

    EBM attitude

    Attitude towards EBM was examined using the questionnaire developed by McColl and colleagues for the assessment of EBM attitude in GPs [33], and has been used in multiple studies on EBM [2, 30, 34]. The questionnaire has been forward–backward translated into Dutch [35] and was adjusted, where necessary, for the GP trainers. In the present study, the mean scores on the seven questions assessing the opinion of GP trainers on the attitude towards EBM with the visual analogue scale (VAS) were used to calculate a score on attitude. All questions are scored on a scale of 0 (=very negative) to 100 (=very positive). The scales of negatively formulated questions were inversed for statistical analysis.

  3. 3)

    EBM behaviour

    Behaviour was assessed in two ways. Firstly, six questions on practice behaviour were taken from the McColl questionnaire. These questions focus on EBM behaviour in daily practice and we added the same questions adjusted for their behaviour towards trainees. Also we asked on the practical implementation of the steps of EBM, such as the frequency of formulating a focused search question (a PICO – a question that contains information about Patient details, the Intervention, the Comparison and the Outcome) with the trainee on a practical patient-related question, and the frequency of performing a search and reading an article together.

    Secondly, in the second and fourth meetings, we asked the trainers to name three intended changes in their EBM behaviour in daily practice that the course content up till that moment had provoked (the commitment-to-change method) [10]. The trainers wrote these intentions on cards, which we sent back to them after two months to stimulate actual behavioural change. At T1 and T2, the trainers were asked to what extent they had fulfilled their intentions, and if they had not been completely fulfilled, the reason for that [36, 37].

Secondary outcomes

  1. 1)

    To explore the relation between formal assessments as outcome measures and self-reported outcomes, we formulated two self-report questions on EBM knowledge and EBM attitude using a 5-point Likert scale to judge self-rated knowledge and – attitude.

  2. 2)

    To assess whether the characteristics of the trainers had influenced the outcomes of the intervention, we compared groups of trainers on specific characteristics possibly influencing the effectiveness of the training: 1) gender, 2) age (subdivided into 5-year age groups), 3) research experience, 4) starting level of EBM knowledge based (above or below median), 5) EBM teacher and 6) EBM teaching group. We hypothesized that age might have an effect on the EBM competency because of changes in attention for EBM in medical education over the years; that despite the specific training of the courses’ teachers there might be a difference in teaching style that could influence the outcomes; and that although the trainers were randomly assigned, the composition of the training group might influence the outcomes as a result of interactions between the participants.

Statistical analysis

All data were analysed in SPSS 20.0. An intraclass correlation coefficient (ICC) (two-way mixed model) was calculated to ensure agreement between the examiners of the Fresno test, and was noted as Cronbach’s alpha. Categorical data were summarized as proportions. Continuous data were visually checked for normality. Normally distributed continuous data were expressed as mean with standard deviation (SD), and non-normally distributed data as median and quartiles or as mode. An independent t-test was performed to compare the means of continuous, normally distributed data between groups, and a paired-sample t-test was performed to compare means at different moments of follow-up. To compare the means of multiple groups with continuous, normally distributed data, an ANOVA was performed with a post hoc Bonferroni test. For the prespecified personal characteristics, variables were divided into groups for comparison. To assess whether the increase in knowledge was greater in participants with the lowest starting level, change from T0 to T1 was compared between those who scored high and those who scored low at the start (defined by above or below the median score at T0) using an independent t-test. To assess whether differences between scores were educationally relevant, effect sizes (ES) were calculated using Cohen’s d by dividing the mean difference by the pooled SD. A Cohen’s d ≤ .2 was considered small, .5 was considered medium and ≥ .8 was considered large (Cohen).

Spearman’s rank correlation coefficient was used to assess the relation between non-parametric continuous variables. A correlation of .20–.39 was considered weak, .40–.59 as moderate and .60–.79 as strong.

Results

Because of the small number of participants and the similarity of outcomes at the LUMC, we merged both populations for analysis. Of the 177 GP trainers, 129 (73 %) gave informed consent and filled in the questionnaire at T0, 99 (56 %) did so at T1 and 89 (50 %) did so at T2. Trainers gave no clarification for not participating in the study. The personal characteristics of the included GP trainers are presented in Table 2.

Table 2 Personal characteristics of the responding GP trainers in Amsterdam and Leiden
Full size table

Primary outcomes

Knowledge and skills

The ICC of the Fresno test was high (Cronbach’s α = .96). The mean scores (score 0–220) were T0 = 77.6 (SD ±41.0), T1 = 120.3 (±33.7) and T2 = 96.8 (±38.8) with mean changes on ∆T1-T0 = 40.8 (±36.7, p < .001), ∆T2-T0 = 20.8 (±39.9, p < .001) and ΔT2-T1 = -17.0 (±35.1, p = .001). This results in a proportional 55 % increase in knowledge between T0 and T1 and of 25 % between T0 and T2, and a decrease in knowledge of 24 % between T1 and T2.

Attitude

Mean scores per item of the McColl questionnaire are presented in Table 3. The overall mean scores on the VAS items were T0: 56.2 (SD ±10.7), T1:58.4 (SD ±10.3) and T2:57.4 (SD ±9.3) with mean changes on ∆T1T0 = 2.2 (SD ±12.8, p = .16), ∆T2-T0 = -.87 (SD ±10.0, p = .49) and ΔT2-T1 = -.41 (SD ±9.5, p = .76).

Table 3 Scores per question of McColl test
Full size table

Behaviour

The trainers’ answers to the questions about formulating a PICO, searching for literature and appraising an article with or without their trainee ranged from once a day to once a year. We saw no changes in this behaviour over the time of the intervention.

The five most formulated intentions to change as described after the second and fourth meetings combined were: 1) formulate PICOs, 2) be more critical towards diagnostic tests, 3) calculate the sensitivity and specificity of tests, 4) search in PubMed and make critical appraisal of an article, and 5) pay more attention to EBM in the learning conversations with the trainee. The proportion of GP the trainers who said they fully completed their intentions was 16.7 %, partly completed their intentions 47.6 % and not completed their intentions at all was 35.7 %. The two most frequently mentioned reasons for not fulfilling the intentions were 1) no priority and 2) not having a trainee at the moment. The scores on the questions about fulfilling their intentions showed no differences compared to the questions about fulfilling their intentions to cooperate with their trainees.

Secondary outcomes

Self-rated knowledge and attitude

A weak positive correlation was found between self-rated EBM knowledge and the mean scores on the Fresno test (Table 4). A moderate positive correlation was found between the overall mean score on the McColl test and the self-rated attitude (Table 5).

Table 4 Correlation and mean of the total Fresno scores related to self-rated knowledge
Full size table
Table 5 Correlation and mean of the total McColl scores and self-rated attitude
Full size table

Influence personal characteristics

The mean scores in knowledge, when comparing trainers with different age, teachers, research experience, and teaching groups, showed no difference at any moment during the study. However, female trainers scored significantly higher than the male trainers on the Fresno test at T1 and T2 (Table 6). The ES of the Fresno test was high for GP trainers with a low starting level (≤ median T0) on both T1 and T2 and for the high-starting level a high ES was found on T1, while almost no effect remained at T2 (Table 7).

Table 6 Differences in mean Fresno scores between men and women
Full size table
Table 7 Effect sizes (ES) and mean changes of the Fresno score with low (≤ median) and high (> median) starting level
Full size table

There were no differences in mean scores on the McColl test with regard to gender, age, teacher, teaching groups or research experience.

Discussion

Main findings

There was an absolute increase in knowledge after the educational intervention but no change in attitude or behaviour, although the intentions to change behaviour were fulfilled by some trainers. We found a higher increase in knowledge in female GP trainers and trainers with a low starting level, but none of the other characteristics were shown to influence the outcomes. Finally, the scores on the self-report scales on knowledge and attitude and the Fresno test and the McColl test also showed a weak (Fresno test) to moderate (McColl test) positive correlation.

Comparison to other studies: primary outcomes

We found that a blended learning programme in EBM for GP trainers induces an increase in knowledge.

Four months after the end of the course (T2), our respondents showed a 24 % decrease in the mean scores on the Fresno test compared to directly after the course (T1) (T1 = 120.3 (±33.7); T2 = 96.8 (±38.8)), although there was still a 25 % increase compared to knowledge scores at baseline (T0 = 77.6 (SD ±41.0); T2 = 96.8 (±38.8)). A study by Kok and colleagues [17] among physicians working for the Dutch National Institute of Benefit Schemes started with a higher baseline score on the Fresno test (mean 93.7 (SD 25.9)), which still showed seven months after the intervention an increase (128.2 (SD22.6)) with a slight decrease after 12 months (121.7(25.1)). Literature [38–40] confirms that a response interval (RI) – that is, the time between the last educational intervention and the assessment – negatively influences the retention of knowledge; decay sets in. After a one-year RI, the loss of knowledge gain generally is approximately one third, and within the next year one half [2]. We found after a RI of four months, a loss of knowledge gain of approximately 24 % between T1 and T2, which is comparable to the RI found in other studies [38–40], but not with the study by Kok and colleagues [17]; the loss of knowledge gain was less.

Attitude and behaviour towards EBM show no differences before and after the intervention, although the intention to use EBM more in practice is present and (partly) fulfilled in approximately 50 % of the participants. Earlier studies [41, 42], as the result of a study by Shuval and colleagues [43], suggest a positive influence on attitude and knowledge change after an educational intervention but no actual behaviour change was found. They suggest that an intervention should be both longer (more than three workshops and six out-reach visits) and more extensive, as was done in our study, which however showed similar effects. The theory of reasoned action [10] stated that attitude and subjective norms are predictors of behaviour. Most GP trainers intend to incorporate more EBM behaviour into their daily practice and their role as trainer, but they do not give it any priority. Furthermore, EBM education should be incorporated more fully into the daily routine to minimize the barriers to EBM implementation. Systematic reviews by Ilic and Maloney [22] but also by Coomarasamy and Khan [23] stated that EBM education should integrated theory with ‘real-life’ clinical scenarios’ or should be moved to clinical practice. Although by the practice-based assignments we hoped to integrate EBM more in daily practice, the success of this effort seems limited. Another explanation for the limited rise in attitude in our study, could be that despite our earlier findings on negative attitude in GP trainers [3] our trainers already had a relatively high score on the McColl test at the beginning (all scores were > 50 out of 100; data not reported).

Comparison to other studies: secondary outcomes

Female trainers scored higher than male trainers on the Fresno test. One explanation could be that the Fresno test is quite linguistic which is said to be an advantage for women, although studies on gender differences and performance or ability, showed inconclusive results on verbal abilities between females and males [44–46]. Also, various studies [47–49] on gender differences in relation to self-regulated learning have been performed, but the results of these studies show inconclusive outcomes as well. Virtanen and Nevgi [49] found a slightly higher score on self-regulated learning by female undergraduate students compared to male students. The differences we found in the mean scores on the Fresno test might have arisen because women are a little better at self-regulated learning compared to men [49] and e-courses requires a great deal of self-regulation. The high effect sizes of the group with a low-starting level is comparable to outcomes of studies in groups receiving no EBM education compared to EBM education which results in a knowledge gain for the intervention groups [50].

Our findings based on the self-report scales on knowledge and attitude versus knowledge/skills and attitude are remarkable. We compared the self-report scales on attitude and knowledge with the scores on the McColl test for attitude and the Fresno test for knowledge/skills. Both show a significant positive correlation (Tables 4, 5 and 6). Several studies [51–53] have found that self-report scales on EBM knowledge are not very reliable or even show a negative correlation. The findings on knowledge cannot be explained by the order of the questions, as the self-report question on knowledge was asked before the questions of the Fresno test. The frame of reference of the respondents therefore cannot have been influenced by the their scores on the Fresno test.

Limitations and strengths

An important limitation is our study design; a cohort design. It would have been interesting to compare the learning benefits of this intervention with another educational design. In that case, a RCT design would have been possible. Unfortunately we were not able to use a control group to compare the design of our blended learning intervention with another (blended learning) design. A second limitation is that although the ultimate educational aim is to have GP trainers act as positive role models for GP trainees in EBM behaviour, we did not measure their actual behaviour towards their trainees in daily practice. It is considered very difficult to measure actual EBM behaviour. [54, 55] Existing tools mainly focus on the formulation of PICO and searching for studies, which do not do justice to the more integrative concept of EBM. Within this study, based on the theory of Fishbein, we therefore chose to describe the possible changes in behaviour by expressing intentions. This theory is heavily criticised by Sniehotta, Presseau and Araujo-Soares [56]. Although there are several initiatives to come to an alternative theory, a better explanation of health behaviour change is not yet available. Armitage [57] suggested that the theory of planned behaviour could be one theory within a broader framework and could be used as a benchmark for theories still to come. A third limitation is that this study was performed mostly at one institute in the Netherlands which could have reduced generalizability regarding setting and time.

The strength of this study is that our educational intervention was based on evidence about education especially in blended learning. The group size was another strength: we were able to incorporate 122 GP trainers at the AMC–UvA along with 7 at the LUMC. The validated questionnaires (Fresno and McColl tests) provided a more or less objective assessment of EBM competency.

Relevance and future research

Because of the importance of GP trainers as positive role models for their trainees [4, 5], they should be able to incorporate EBM more fully into their daily practice and keep their EBM knowledge and skills more up to date and ready for deployment. EBM behaviour should be visible in the daily practice of the GP trainer. A blended learning educational intervention could be the first step in this process.

In future programs to enhance the transfer of programs on EBM to daily practice: 1) the F2F part should focus even more on assignments performed in clinical practice to assure integration of EBM in daily practice [36, 38]; and 2) EBM should be a component of all annual refresher courses in order to become standard way of thinking for trainers.

Future research could focus on the contribution of the various components of the blended learning program (e-learning, F2F, assignments), on the development of competency-aspects such as attitude, knowledge and behaviour. Based on those studies it might be possible to design schemes addressing those aspects for developing blended learning.

Conclusion

An intensive blended learning course on EBM for GP trainers induces an increase in knowledge and skills that, although decreased, remains after four months. Attitude and behaviour towards EBM however show no differences before and after the intervention, although GP trainers’ intention to use EBM more often in their practice is present.

References

  1. Mayer J, Piterman L. The attitudes of Australian GPs to evidence-based medicine: a focus group study. Family Pract. 1999;16:627–32.

    Article  Google Scholar 

  2. Zwolsman SE, van Dijk N, te Pas E, Wieringa-De Waard M. Barriers to the use of evidence-based medicine: knowledge and skills, attitude, and external factors. Perspect Med Educ. 2013;2:4–13.

    Article  Google Scholar 

  3. Te Pas E, van Dijk N, Bartelink MEL, Wieringa-De Waard M. Factors influencing the EBM behaviour of GP trainers; a mixed method study. Med Teach. 2013;35:e990–7.

    Article  Google Scholar 

  4. der Leeuw HGAR J-v, van Dijk N, van Etten-Jamaludin FS, Wieringa-De Waard M. The attributes of the clinical trainer as a role model: a systematic review. Acad Med. 2013;88:26–34.

    Article  Google Scholar 

  5. Cruess SR, Cruess RL, Steinert Y. Role modelling - making the most of a powerful strategy. BMJ. 2008;336:718–21.

    Article  Google Scholar 

  6. Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, et al. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5:1–9.

    Article  Google Scholar 

  7. Upton D, Upton P. Knowledge and use of evidence-based practice of GPs and hospital doctors. J Eval Clin Pract. 2005;12:376–84.

    Article  Google Scholar 

  8. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. 1996. Clin Orthop Relat Res. 2007;455:3–5.

    Google Scholar 

  9. Satterfield JM, Spring B, Brownson R, Mullen EJ, Newhouse RP, Walker BB, et al. Toward a Transdisciplinary Model of Evidence-Based Practice. Milbank Q. 2009;87:368–90.

    Article  Google Scholar 

  10. Fishbein, M., & Ajzen, I. Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley. 1975. http://people.umass.edu/aizen/f&a1975.html retrieved 02 February 2015

  11. Nilsson M, Ostergren J, Fors U, Rickenlund A, Jorfeldt L, Caidahl K, et al. Does individual learning styles influence the choice to use a web-based ECG learning programme in a blended learning setting? BMC Medical Education. 2012;12:5.

    Article  Google Scholar 

  12. Sun P, Tsai RJ, Finger G, Chen Y, Yeh D. What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Comput Educ. 2008;50:1183–202.

    Article  Google Scholar 

  13. Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education - Part 1: Learning, teaching and assessment. Medical Teacher. 2008;30:455–73.

    Article  Google Scholar 

  14. Wutoh R, Austin Boren S, Balas EA. eLearning: a review of internet-based continuing medical education. J Contin Educ Health Prof. 2004;24:20–30.

    Article  Google Scholar 

  15. Curran V, Lockyer J, Sargeant J, Fleet L. Evaluation of learning outcomes in Web-based continuing medical education. Acad Med. 2006;81:S30–4.

    Article  Google Scholar 

  16. Choules AP. The use of elearning in medical education: a review of the current situation. Postgrad Med J. 2007;83:212–6.

    Article  Google Scholar 

  17. Kok R, Hoving JL, Smits PB, Ketelaar SM, van Dijk FJ, Verbeek JH. A clinically integrated post-graduate training programme in evidence-based medicine versus ‘no intervention’ for improving disability evaluations: a cluster randomised clinical trial. PLoS One. 2013;8:e57256.

    Article  Google Scholar 

  18. Hugenholtz NI, Sluiter JK, van Dijk FJ, Nieuwenhuijsen K. EBM E-learning: feasible and effective for occupational physicians in different countries. Saf Health Work. 2012;3:199–208.

    Article  Google Scholar 

  19. Pankin J, Roberts J, Savio M. Blended Learning at MIT. 2012. White Paper is sponsored by the MIT Training Alignment Team (TAT)

  20. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning in medical education. Acad Med. 2006;81:207–12.

    Article  Google Scholar 

  21. Ilic D, Hart W, Fiddes P, Misso M, Villanueva E. Adopting a blended learning approach to teaching evidence based medicine: a mixed method study. BMC Medical Education. 2013;13:169.

    Article  Google Scholar 

  22. Ilic D, Maloney S. Methods of teaching medical trainees evidence-based medicine: a systematic review. Med Educ. 2014;48:124–35.

    Article  Google Scholar 

  23. Coomarasamy A, Taylor R, Khan KS. A systematic review of postgraduate teaching in evidence-based medicine and critical appraisal. Med Teach. 2003;25:77–81.

    Article  Google Scholar 

  24. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300:1181–96.

    Article  Google Scholar 

  25. Dutch College of General Practitioners (NHG). https://www.nhg.org/nhg-standaarden. Retrieved 14 January 2014.

  26. Te Pas E, Wieringa-De Waard M, Snijders Blok B, Pouw HT, van Dijk N. Didactic and technical considerations when developing e-learning and CME. Educ Inf Technol. 2014. DOI 10.1007/s10639-014-9364-2.

  27. Brandsford JD, Brown AL, Cocking RR. How people learn: brain, mind, experience, and school. Washington, DC: National Academy Press; 2000.

    Google Scholar 

  28. Trivette CM, Dunst CJ, Hamby DW, O’Herin CE. Characteristics and Consequences of Adult Learning Methods and Strategies. Res Brief. 2009;3.

  29. van Merrienboer JJG, Jelsma O, Paas FGWC. Training for reflective expertise: a four-component instructional design model for training complex cognitive skills. Educ Technol Res Dev. 1992;40:23–43.

    Article  Google Scholar 

  30. Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, et al. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006;296:1116–27.

    Article  Google Scholar 

  31. Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. Br Med J. 2003;326:319–21.

    Article  Google Scholar 

  32. Spek B, de GS W, van Dijk N, Lucas C. Development and validation of an assessment instrument for teaching evidence-based practice to students in allied health care: the Dutch Modified Fresno. J Allied Health. 2012;41:77–82.

    Google Scholar 

  33. McColl A, Smith H, White P, Field J. General practitioners’ perceptions of the route to evidence based medicine: a questionnaire survey. BMJ. 1998;316:361–7.

    Article  Google Scholar 

  34. Mittal R, Perakath B. Evidence-based surgery: knowledge, attitudes, and perceived barriers among surgical trainees. J Surg Educ. 2010;67:278–82.

    Article  Google Scholar 

  35. Knops AM, Vermeulen H, Legemate DA, Ubbink DT. Attitudes, awareness, and barriers regarding evidence-based surgery among surgeons and surgical nurses. World J Surg. 2009;33:1348–55.

    Article  Google Scholar 

  36. Domino FJ, Chopra S, Seligman M, Sullivan K, Quirk ME. The impact on medical practice of commitments to change following CME lectures: A randomized controlled trial. Med Teach. 2011;33:E495–500.

    Article  Google Scholar 

  37. Aiely D, Norton MI. How actions create - not just reveal - preferences. Trends Cognit Sci. 2008;12(1):13–6.

    Article  Google Scholar 

  38. Custers EJFM. Long-term retention of basic science knowledge: a review study. Adv Health Sci Educ. 2010;15:109–28.

    Article  Google Scholar 

  39. Custers EJFM, ten Cate OTJ. Very long-term retention of basic science knowledge in doctors after graduation. Med Educ. 2011;45:422–30.

    Article  Google Scholar 

  40. Altmann EM, Gray WD. FORGETTING TO REMEMBER: The Functional Relationship of Decay and Interference. Psychol Sci. 2002;13:27–33.

    Article  Google Scholar 

  41. Dacey M, Arnstein F, Kennedy MA, Wolfe J, Phillips EM. The impact of lifestyle medicine continuing education on provider knowledge, attitudes, and counseling behaviors. Med Teach. 2013;35:e1149–56.

    Article  Google Scholar 

  42. Olson CA, Shershneva MB, Brownstein MH. Peering inside the clock: using success case method to determine how and why practice-based educational interventions succeed. J Contin Educ Health Prof. 2011;31 Suppl 1:S50–9.

    Article  Google Scholar 

  43. Shuval K, Berkovits E, Netzer D, Hekselman I, Linn S, Brezis M, et al. Evaluating the impact of an evidence-based medicine educational intervention on primary care doctors’ attitudes, knowledge and clinical behaviour: a controlled trial and before and after study. J Eval Clin Pract. 2007;13:581–98.

    Article  Google Scholar 

  44. Weiss EM, Kemmler G, Deisenhammer EA, Fleischhacker WW, Delazer M. Sex differences in cognitive functions. Pers Indiv Differ. 2003;35:863–75.

    Article  Google Scholar 

  45. Hyde JS, Linn MC. Gender differences in verbal-ability - a meta-analysis. Psychol Bull. 1988;104:53–69.

    Article  Google Scholar 

  46. Ryan JJ, Kreiner DS, Tree HA. Gender differences on WAIS-III incidental learning, pairing, and free recall. Appl Neuropsychol. 2008;15:117–22.

    Article  Google Scholar 

  47. Ceresnik A. Self-regulation of adolescents. Gender specifications. Procedia - Social and Behavioral Sciences. 4th International Conference on New Horizons in Education 2013:2365-2374.

  48. Bembenutty H. Self-Regulation of Learning and Academic delay of Gratification: Gender and Ethnic Differences Among College Students. J Adv Acad. 2007;18:586–616.

    Google Scholar 

  49. Virtanen P, Nevgi A. Disciplinary and gender differences among higher education students in self-regulated learning strategies. Educ Psychol. 2014;30:323–47.

    Article  Google Scholar 

  50. Al-Baghli NA. Evidence based medicine workshop. Randomized controlled trial of the efficacy on physician’s knowledge and skills. Saudi Med J. 2013;34:1055–61.

    Google Scholar 

  51. de Smedt A, Buyl R, Nyssen M. Knowledge on evidence-based practice: self-assessment by primary care workers. J Eval Clin Pract. 2007;13:599–600.

    Article  Google Scholar 

  52. Caspi O, McKnight P, Kruse L, Cunningham V, Figueredo AJ, Sechrest L. Evidence-based medicine: discrepancy between perceived competence and actual performance among graduating medical students. Med Teach. 2006;28:318–25.

    Article  Google Scholar 

  53. Colthart I, Bagnall G, Evans A, Allbutt H, Haig A, Illing J, et al. The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10. Med Teach. 2008;30:124–45.

    Article  Google Scholar 

  54. Oude Rengerink K, Zwolsman SE, Ubbink DT, Mol BW, van Dijk N, Vermeulen H. Tools to assess evidence-based practice behaviour among healthcare professionals. Evid Based Med. 2013;18:129–38.

    Article  Google Scholar 

  55. Zwolsman SE, van Dijk N, Wieringa-de Waard M. Observations of evidence-based medicine in general practice. Perspect Med Educ. 2013;2:196–208.

    Article  Google Scholar 

  56. Sniehotta FF, Presseau J, Araujo-Soares V. Time to retire the theory of planned behaviour. Health Psychol Rev. 2014;8:1–7.

    Article  Google Scholar 

  57. Armitage ChJ. Time to retire the theory of planned behaviour? A commentary on Sniehotta, Presseau and Araujo-Soares. Health Psychol Rev. 2014.

Download references

Acknowledgments

This study was financed by CASH (Commission to Stimulate Activities Regarding GP training) of the SBOH. The SBOH is the employer of GPs in training and finances all Dutch GP training programmes.

We thank the staff of the Department of General Practice/Family Medicine, Academic Medical Center, University of Amsterdam for mailing the link of the survey and the invitation of the training to the GP trainers and to organize the meetings of the intervention. We also thank the GP trainers for participating in this study. We are grateful to the EBM teachers for their effort during the intervention and we also thank the sounding board for their input during the development of the intervention

Author information

Authors and Affiliations

  1. Department of General Practice/Family Medicine, Academic Medical Center, University of Amsterdam, location L0-310 Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands

    Ellen te Pas, Margreet Wieringa–de Waard & Nynke van Dijk

  2. Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, The Netherlands

    Wouter de Ruijter

Authors
  1. Ellen te Pas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Margreet Wieringa–de Waard
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wouter de Ruijter
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nynke van Dijk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ellen te Pas.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

EtP participated in the design and coordination of the study, the design of the educational intervention, collected and analysed the data, and drafted the manuscript. MWdW also participated in the design and coordination of the study, the design of the educational intervention and as one of the EBM teachers. WdR was involved in the coordination, the design of the educational intervention and also as one of the EBM teachers. NvD was involved in the design and the coordination of the study and contributed substantially to the interpretation of data. She also participated in the design of the intervention. All authors revised and edited the manuscript and approved the final version.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

te Pas, E., Waard, M.W., de Ruijter, W. et al. Learning results of GP trainers in a blended learning course on EBM: a cohort study. BMC Med Educ 15, 104 (2015). https://doi.org/10.1186/s12909-015-0386-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12909-015-0386-2

Keywords

BMC Medical Education

ISSN: 1472-6920

Contact us