Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Multiple strategy peer-taught evidence-based medicine course in a poor resource setting

  • Ammar Sabouni1,
  • Yamama Bdaiwi2,
  • Saad L. Janoudi1,
  • Lubaba O. Namous1,
  • Tarek Turk2Email author,
  • Mahmoud Alkhatib2,
  • Fatima Abbas2 and
  • Ruba Zuhri Yafi2
BMC Medical EducationBMC series – open, inclusive and trusted201717:82

DOI: 10.1186/s12909-017-0924-1

Received: 7 October 2016

Accepted: 27 April 2017

Published: 4 May 2017

Abstract

Background

Teaching Evidence Based Medicine (EBM) is becoming a priority in the healthcare process. For undergraduates, it has been proved that integrating multiple strategies in teaching EBM yields better results than a single, short-duration strategy. However, there is a lack of evidence on applying EBM educational interventions in developing countries. In this study, we aim to evaluate the effectiveness of a multiple strategy peer-taught online course in improving EBM awareness and skills among medical students in two developing countries, Syria and Egypt.

Methods

We conducted a prospective study with pre- and post- course assessment of 84 medical students in three universities, using the Berlin questionnaire and a set of self-reported questions which studied the students’ EBM knowledge, attitude and competencies. The educational intervention was a peer-taught online course consisting of six sessions (90 min each) presented over six weeks, and integrated with assignments, group discussions, and two workshops.

Results

The mean score of pre- and post-course Berlin tests was 3.5 (95% CI: 2.94–4.06) and 5.5 (95% CI: 4.74–6.26) respectively, increasing by 2 marks (95% CI: 1.112–2.888; p-value <0.001), which indicates a statistically significant increase in students’ EBM knowledge and skill, similar to a previous expert-taught face to face contact course. Self-reported confidences also increased significantly. However, our course did not have a major effect on students’ attitudes toward EBM (1.9–10.8%; p-value: 0.12–0.99).

Conclusion

In developing countries, multiple strategy peer-taught online courses may be an effective alternative to face to face expert-taught courses, especially in the short term.

Keywords

Evidence-based medicine Evidence-based Health Care. Peer-taught, Medical Education. Online courses, Berlin Questionnaire. Undergraduate. Middle East. Syria. Egypt.

Background

Teaching Evidence Based Medicine (EBM) to undergraduate and postgraduate medical students is becoming a priority in the healthcare process. In the UK, EBM has become a part of the foundation year program, [1] and in the US and Canada, accreditation standards for medical schools include the practice of EBM [2, 3].

In 2014, an overview systematically assessed systematic reviews published between 1993 and 2013 on teaching evidence-based medicine (EBM) in a variety of settings [4]. For undergraduates, it proved that integrating multiple strategies (lectures, tutorials, journal clubs, workshops, online courses and integrated methods) produces better results compared to a single, short duration strategy. It took into consideration outcomes such as EBM knowledge, skills, attitude, and practice.

Developing countries have less EBM awareness. In Egypt, there are wide misconceptions about EBM; most physicians consider themselves to be practicing EBM while in fact they are not [5]. In Turkey, only 1% of physicians attended EBM courses during their university life [6] and in Saudi Arabia, 13% of medical students had ever attended a course on EBM [7]. While in the United States, 38.5% of medical schools have a formal EBM curriculum [8, 9].

The barriers to EBM awareness in developing countries are several; in Iran, a systematic review was conducted to investigate obstacles to EBM, to find the most important factors which were; the absence of proper facilities, positive attitudes and adequate training [10]. In Syria, Alahdab et al. explored the barriers to EBM awareness in 2012, and reported that the most important barriers were; the absence of EBM curricula, equipment and facilities, in addition to difficulties in accessing information, institutional subscriptions to medical journals, and sufficient IT hardware [11]. In the Middle East especially, additional challenges have aggravated the situation since the Arab Spring began, such as difficulties in attending face to face contact courses due to lack of safety, resources and infrastructure. The availability of content experts is also an issue with an increase in the number of emigrating doctors; in the last couple of years in Syria alone 80,000 doctors have emigrated [12].

For all these reasons, developing countries should create their own solutions and build their experience using the available means. One possible approach is online courses, which provide flexibility in time, place, and cost [13] leading to a comparable level of knowledge gained compared with lecture-based courses [14]. Another approach is peer organised courses which cost less, need less highly experienced staff, and make it possible to generate new peers from one expert. Yet are effective in increasing self-reported confidences [15]. There is still however a lack of knowledge with regards to the effectiveness of these approaches in developing countries.

In this study, we aim to evaluate the effectiveness of a multiple strategy peer-taught online course in improving EBM awareness and skills among medical students in two developing countries, Egypt and Syria.

Methods

Study design

We conducted a prospective study with pre- and post- course assessment by Berlin questionnaire and a self-reported confidence questionnaire through March and April 2015.

Participants and peers

Eighty four graduate entry medical students and recently graduated final year medical students at the Faculties of Medicine of Damascus University, Syria, Cairo University, Egypt, and Tanta University, Egypt, were enrolled in an online EBM course. Participants had little or no previous EBM skills and knowledge. The course was provided by peers using social media, Facebook and YouTube as the main course platforms. Peers were all medical students who had no previous knowledge of EBM and underwent training in the foundations of EBM by an expert, they later self-developed their EBM skills and created and presented the course.

In the study analysis we included only 48 participants who were able to take both pre and post course questionnaires in person (Table 1). All participants were asked for a verbal consent and the study was approved by the ethical committee of Faculty of Medicine, Damascus University.
Table 1

Basic characteristics of included participants

 

(N)

(%)

Gender

Male

25

52.1

Female

23

47.9

City

Damascus

23

47.9

Cairo

16

33.3

Tanta

9

18.8

Educational level

Basic science (pre-clinical) years

13

27.1

Clinical years

35

72.9

Observed EBM

Never

33

68.8

Once

10

20.8

Many

5

10.4

Regular

0

0.0

Participated in EBM

Never

36

75.0

Once

11

22.9

Many

0

0.0

Regular

1

2.1

Conduct of the educational intervention

A six-session weekly online course was provided by peers, every session included two to four videos lasting approximately 90 min in total. The videos were developed using simple means as mobile phone camera by the peers, who presented them in the participants’ native Arabic language. An evaluation quiz was mandatory at the end of each session. The evaluation included MCQs, true-false questions, open-ended questions, and journal club discussions. The quizzes’ results were not part of the analysis.

Two practical workshops were held at each venue by four of the same peers to boost the educational process, each lasted for around 3 h. In addition to discussions which were held on social media on a Facebook group.

Course curriculum

The course curriculum was covered by a mix of online lectures and interactive workshops covering the following topics: (a) Definition of EBM and formulating clinical questions; (b) Study designs, searching the medical literature, and structure of a scientific paper; (c) Critical appraisal; (d) Analysing the results of studies, interpreting the clinical relevance and precision of the results; (e) Systematic reviews and (f) Diagnostic studies. An Additional file containing the detailed curriculum [see Additional file 1].

All online sessions, journal club discussions and practical workshops were supervised and conducted by peers.

Outcome assessment (the questionnaire)

The Berlin Questionnaire [16] was chosen for appraisal of the participants due to its proven validity, internal consistency, and ability to accommodate for change. It is also able to distinguish participants at different stages of the learning process [17]. The Berlin Questionnaire consists of 15 MCQ-item questions with one point for each question. Questions are based on clinical cases and appraise the learner’s ability to formulate a question, process the suitable evidence, and recognise study design. The questionnaire has a focus on therapeutic and diagnostic studies (4 questions each), and systematic review, prognostic, and harm studies (1 question each) and the rest of the question cover a number of different subjects. The complete score was 15 marks. The Questionnaire was supplemented with a set of self-reported questions to inspect students’ self-reported confidences [18]. Participants completed the pre and post-course survey provided before the first session and after the last session.

Statistical analysis

Data were entered into a Microsoft Excel spreadsheet for each city, combined, and then imported into SPSS version 22.0 to perform the analysis.

Results

A total of 84 students were enrolled in the course, while only 48 students were able to take both pre and post course questionnaires in person and were included in the final analysis with a response rate of 57.1%.

(Table 1) describes the basic characteristics of included participants.

Objective and subjective evaluation of EBM skills and knowledge before and after the course

The mean scores of students’ pre- and post-course Berlin tests were 3.5 (95% CI: 2.94–4.06) and 5.5 (95% CI: 4.74–6.26) respectively, increasing significantly by 2 marks (95% CI: 1.112–2.888; p-value <0.001), (Fig. 1). Further comparisons were made to demonstrate the change between pre- and post-course test’s mean scores among different baseline factors. Females gained significantly more knowledge and skills than males. Comparing residence, education level and the current course’s impact with a previously held face-to-face course did not yield significant results (Table 2).
Fig. 1

Pre- and post-test scores for participants

Table 2

Post-course scores’ mean difference comparing gender, residence, year of study, and current course vs. a previously held course

Variables

Post-course mean scores

Post-course mean difference

Males vs. Females

Males = 1.16

1.75 (P-value = 0.03)

Females = 2.91

Syria vs. Egypt

Syria = 2.52

1 (P-value = 0.25)

Egypt = 1.52

Basic years vs. Clinical years

Basic years = 2.07

0.105 (P-value = 0.69)

Clinical years = 1.9

Current course vs. previously held course

Alahdab et al. course = 2.65

0.65 (P-value = 0.14)

Current course = 2

Perception and self-reported knowledge of EBM

As a self-rating of EBM skills and knowledge, all of the included participants (100%) thought they knew little or nothing at all about evidence-based medicine before the course. This percentage decreased by half (p-value <0.001) after completing the course.

Regarding self-rating of different EBM skills, 40.4% of students reported being confident/extremely confident in their ability to formulate a PICO question. This percentage increased to 74.4% after completing the course (p-value < 0.01). Students’ ability to perform an online literature search, calculating basic statistics, and their ability in applying evidence for patient-centred care increased significantly between the pre- and post- test (22.8% p-value = 0.016; 49.3% p-value <0.001; 30.4% p-value = 0.001 respectively). On investigating students’ critical appraisal skills, we found that the increase by 26% was not statistically significant (p-value = 0.05). Figure 2 emphasizes the key findings and differences between pre- and post-course self-reported confidences.
Fig. 2

Students’ self reported confidence level of certain aspects of EBM before and after the course

Students’ attitudes toward EBM before and after the course

Table 3 summarizes all results related to students’ attitudes toward EBM before and after the course. When asked if evidence-based medicine is time-consuming, 71.7% of the students disagreed with this statement before the course, compared to 69.8% disagreeing with it after the course, this difference in attitude is not statistically significant (p-value >0.05). When students were asked whether EBM relies too much on statistics, 8.7% of them disagreed before the test, and this attitude was did not essentially change, as 19.5% of them disagreed with this statement after the course, a non-statistically significant result (p-value >0.05).
Table 3

Students’ attitudes toward EBM before and after the course

Variables

Pre-course percentage of students disagreeing (%)

Post-course percentage of students disagreeing (%)

Percentages difference (%)

P-value

EBM is time consuming

71.7

69.8

1.9

0.99

EBM is a “cookbook” for medicine

45.7

50

4.3

0.99

EBM relies too much on statistics

8.7

19.5

10.8

0.12

Discussion

We found our multiple strategy peer-taught online course significantly improved medical student EBM skills. Participants improved with a mean increase in the Berlin questionnaire score of 2 marks. Yielding an almost identical increase in EBM knowledge to expert-taught face to face courses [11, 15, 19].

After the course, participants’ self-confidence improved in formulating a PICO research question, performing an online literature search, reading and assessing the results of therapy-related articles, using basic statistical concepts, using likelihood ratio, and applying evidence for patient-centred care.

Multiple strategy course

The strategy we adopted, a multiple strategy online course presented over several weeks, accompanied with several face to face contact workshops, group discussions and assignments, proved effective in line with the findings of Young et al. Young et al. in their overview of systematic reviews suggested that integrating multiple strategies presented over a few weeks, is better than a single strategy method [4]. The results of our social media strategy which was based on Facebook and Youtube were also in line with previous findings [20].

Online platform

Our online course was effective as a comparable alternative that saved on effort, time, and cost. Hadley et al.’s trial also reported no difference between an e-learning EBM course and a lecture-based EBM [14]. Even though we did not carry out a full cost analysis, our course only required the cost of reserving a hall for the two workshops. This corresponds with what Maloney et al. revealed in that a blended e-learning approach cost 24% less than face to face learning [13]. Online courses also serve as a reference which can be used when repeating workshops.

In addition, we expect the fact that our online course was conducted in the participant’s native language (Arabic in our case) enhanced the reach of the course to students of all capabilities and so promoted more students to join.

Peer-taught strategy

Another approach we examined was the peer-taught strategy; we found peer teaching effective in improving students’ EBM knowledge and skill. This follows the results of a peer-taught EBM workshop held at The UK National Institute for Health and Care Excellence (NICE) [15]. We also proved efficacy similar to a two day face to face expert-taught course, previously held in Syria by Al-Ahdab et al. The difference between the two courses was non-statistically significant [11].

Perspective

Many studies have addressed teaching EBM skills and knowledge to medical students and health professionals, and changing attitudes towards EBM in both developed and developing countries [10, 11]. In light of recent lack of safety, and lack of both human and material resources in the Middle East and North Africa region, more flexible and affordable means are necessary. Online and peer taught courses are two possible solutions yet there is limited literature on their effectiveness in teaching EBM in developing countries. Our study addressed this. Studies on effectiveness of EBM courses in general in Syria and Egypt are also scarce and may represent another unique situation in the security situation in countries in crisis and teaching in them. This study also compares between these countries.

Limitations

Our course did not have a significant effect on students’ attitude toward EBM this may be explained by our questions not comprehensively assessing EBM attitude. In addition, our course did not significantly affect the self-confidence of students to critically appraise a study, even though there was a specific focus on critical appraisal, and was addressed in several lectures, workshops and assignments. These finding may be explained by students’ overestimation of their critical appraisal skills pre-test before delving into the details of the course.

We suggest the main limitation of our study is the subjectivity of self-reported attitudes. We cannot judge whether this improvement will be implemented in practice or not. In addition, though students reported no previous background in EBM, students voluntarily participated in the course and this may have imposed a positive bias. We think our sample size and diminished questionnaire response are also limitations. Another important limitation is that the study design did not allow for a live comparison between a face to face and an online course, a controlled trial is the better alternative.

Conclusion

According to our findings, we suggest that to build reliable Evidence-Based Medicine practitioners in developing countries, multiple strategy peer-taught online courses are an effective approach and a comparable alternative to face to face expert-taught courses.

We recommend university role models and EBM experts implement and further assess this choice to provide high quality online courses and higher EBM awareness among medical students in developing countries.

Declarations

Acknowledgement

We would like to thank the Universities of Damascus, Cairo, and Tanta for their efforts in facilitating the conduction of our course and research project, and the AHYA project for organizing the course.

Funding

None.

Availability of data and materials

All data related to this paper’s conclusion are available and stored by the authors. Please contact Sa’ad L. Janoudi for requests to access the data via saadjanoudi@gmail.com.

Authors’ contributions

All authors contributed equally to the conception and design of the study, lecturing in the course, and to the data collection process. In addition, AA and YB drafted the protocol and the discussion section of the final manuscript. SLJ and LON, participated in the data entry process and carried out the data analysis. TT and MK, participated in the data entry process and drafted the results section. FA and RA, drafted the introduction and wrote the abstract. All authors took part in revising and finalizing the final manuscript.

Competing interest

The authors declare that they have no competing interests. The authors alone are responsible for the content and writing of the article.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Verbal consent was asked from all participants prior to taking the course and the study was approved by the ethical committee of the Faculty of Medicine, Damascus University. In Egypt, all study procedures were approved as exempt from the institutional review board in accordance with Title 45 of the Code of Federal Regulations, Part 46.101(b), given the objective to evaluate a method of educational instruction, and that collected data were stripped of any identifiers before analysis.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Kasr Al-Ainy, Faculty of Medicine, Cairo University
(2)
Faculty of Medicine, Damascus University

References

  1. Crilly M, Glasziou P, Heneghan C, Meats E, Burls A. Does the current version of 'Tomorrow's Doctors' adequately support the role of evidence-based medicine in the undergraduate curriculum? Medical Teach. 2009;31(10):938–44.View ArticleGoogle Scholar
  2. Education LCoM. Functions and structure of a medical school: standards for accreditation of medical school programs leading to the M.D. degree. Scientific Method/Clinical/Translational Research, Critical Judgement/Problem-solving skills. http://lcme.org/: The Committee; 2017–2018.
  3. Ahmadi SF, Baradaran HR, Ahmadi E. Effectiveness of teaching evidence-based medicine to undergraduate medical students: a BEME systematic review. Med Teach. 2015;37(1):21–30.View ArticleGoogle Scholar
  4. Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One. 2014;9(1):e86706.View ArticleGoogle Scholar
  5. Mortada EM. Clinicians' self-perceived competency in evidence-based practice at Zagazig University hospitals, Egypt. J Egypt Public Health Assoc. 2013;88(3):121–9.View ArticleGoogle Scholar
  6. Avsar UZ, Avsar U, Cansever Z, Acemoglu H, Cayir Y, Khan AS. Evidence based medicine: teaching, learning and practice: results of a cross-sectional study from Turkey. J Pak Med Assoc. 2014;64(7):762–5.Google Scholar
  7. Bahammam MA, Linjawi AI. Knowledge, attitude, and barriers towards the use of evidence based practice among senior dental and medical students in western Saudi Arabia. Saudi Med J. 2014;35(10):1250–6.Google Scholar
  8. Aiyer M, Hemmer P, Meyer L, Albritton TA, Levine S, Reddy S. Evidence-based medicine in internal medicine clerkships: a national survey. South Med J. 2002;95(12):1389–95.View ArticleGoogle Scholar
  9. Ramos KD, Schafer S, Tracz SM. Validation of the Fresno test of competence in evidence based medicine. BMJ. 2003;326(7384):319–21.View ArticleGoogle Scholar
  10. Ghojazadeh M, Hajebrahimi S, Azami-Aghdash S, Pournaghi Azar F, Keshavarz M, Naghavi-Behzad M, et al. Medical students’ attitudes on and experiences with evidence-based medicine: a qualitative study. J Eval Clin Pract. 2014;20(6):779–85.View ArticleGoogle Scholar
  11. Alahdab F, Firwana B, Hasan R, Sonbol MB, Fares M, Alnahhas I, et al. Undergraduate medical students' perceptions, attitudes, and competencies in evidence-based medicine (EBM), and their understanding of EBM reality in Syria. BMC Res Notes. 2012;5:431.View ArticleGoogle Scholar
  12. Stone-Brown K. Syria: a healthcare system on the brink of collapse. BMJ (Clin Res Ed) 2013;347:f7375.
  13. Maloney S, Nicklen P. A cost-effectiveness analysis of blended versus face-to-face delivery of evidence-based medicine to medical students. J Med Internet Res. 2015;17(7):e182.View ArticleGoogle Scholar
  14. Hadley J, Kulier R, Zamora J, Coppus SF, Weinbrenner S, Meyerrose B, et al. Effectiveness of an e-learning course in evidence-based medicine for foundation (internship) training. J Royal Soc Med. 2010;103(7):288–94.View ArticleGoogle Scholar
  15. Rees E, Sinha Y, Chitnis A, Archer J, Fotheringham V, Renwick S. Peer-teaching of evidence-based medicine. Clin Teach. 2014;11(4):259–63.View ArticleGoogle Scholar
  16. Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH, Kunz R. Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. BMJ (Clin Res ed). 2002;325(7376):1338–41.View ArticleGoogle Scholar
  17. Shaneyfelt T, Baum KD, Bell D, Feldstein D, Houston TK, Kaatz S, et al. Instruments for evaluating education in evidence-based practice: a systematic review. JAMA. 2006;296(9):1116–27.View ArticleGoogle Scholar
  18. Gagliardi JP, Stinnett SS, Schardt C. Innovation in evidence-based medicine education and assessment: an interactive class for third- and fourth-year medical students. J Med Libr Assoc. 2012;100(4):306–9.View ArticleGoogle Scholar
  19. Taheri H, Mirmohamadsadeghi M, Adibi I, Ashorion V, Sadeghizade A, Adibi P. Evidence-based medicine (EBM) for undergraduate medical students. Ann Acad Med Singap. 2008;37(9):764–8.Google Scholar
  20. Puljak L. Using social media for knowledge translation, promotion of evidence-based medicine and high-quality information on health. J Evid Based Med. 2016;9(1):4–7. doi:10.1111/jebm.12175.

Copyright

© The Author(s). 2017