- Open Access
A study on how using an interactive multimedia e-book improves teachers’ ability to teach evidence-based medicine depending on their seniority
BMC Medical Education volume 21, Article number: 547 (2021)
Teaching evidence-based medicine (EBM) is not an easy task. The role of the electronic book (e-book) is a useful supplement to traditional methods for improving skills. Our aim is to use an interactive e-book or PowerPoint to evaluate instructors’ teaching effects on EBM.
Our study group was introduced to learning EBM using an interactive e-book available on the Internet, while the control group used a PowerPoint presentation. We adopted the Modified Fresno test to assess EBM skills both before and after their learning. EBM teaching sessions via e-book or PowerPoint were 20–30 min long, followed by students’ feedback. We adopted Student’s t-test to compare teachers’ evaluation of their EBM skills prior to the class and the students’ assessment of the teachers’ instruction. We also adopted repeated measures ANCOVA to compare teachers’ evaluation of their EBM skills using the Fresno test both before and after the class.
We observed no difference regarding EBM skills between the two groups prior to their experimental learning, which was assessed by the Modified Fresno test. After learning, physicians in the study group ranked higher in choosing a case to explain which kind of research design was used for the study type of the question and explaining their choice (P = 0.024) as assessed by the post-test to pre-test Fresno test. Teaching effect was better in the e-book group than in the control group for the items, “I am satisfied with this lesson,” “The teaching was of high quality,” “This was a good teaching method,” and “It aroused my interest in EBM.” However, no differences were observed between the two groups in physicians who had more than 10 years’ experience.
The use of interactive e-books in clinical teaching can enhance a teacher’s EBM skills, though not in more senior physicians. This may suggest that teaching methodology and activities differ for teachers’ varying years of experience.
Using the best evidence for determining a patient’s therapy is important . Traditionally, the ability to make treatment decisions was limited to a physician’s own experience with the problem, which carries the potential risk of error [2,3,4,5]. The value of evidence-based medicine (EBM), which involves using updated, relevant, and trustworthy evidence to inform medical decisions, has been broadly acknowledged . As a result, teaching EBM has become crucial for medical students’ development into lifelong independent learners and critical thinkers that can offer high-quality patient care . The research problem is that physicians—even senior ones—may miss some of the important nuances of EBM, which subsequently impacts the quality of their teaching . Given the lack of EBM teaching aids, we considered whether an interactive e-learning tool using SimMAGIC software  could be used as an effective teaching aid for EBM. In this study, our aim was to use an interactive e-book or PowerPoint to evaluate instructors’ teaching effects on EBM.
Chiu reported that, according to a national survey in Taiwan, the knowledge, skills, and attitudes regarding evidence-based practice (EBP) improved among physicians and nurses after undergoing training , thus indicating that the practice of EBM is important and can be improved through training . Using Internet information sources to answer patient-related questions has taken an ever more important place in the daily practice of physicians . A practical new instrument for measuring good-quality clinical application is vital for educating teachers and evaluating the effect of their teaching .
Electronic learning (e-learning) differs from traditional educational methods in that it uses digital platforms for learning [13, 14]. Using technology for learning has the benefit of being more student-focused and flexible . The learner becomes a more active participant in the acquisition of knowledge, as opposed to being a passive recipient . Cook reported that e-learning performs on par with traditional classroom teaching . However, other researchers have reported that e-learning is no better than traditional classroom education for improving the proficiency of novices . In modern curricula, e-book technology is considered a useful supplement to traditional methods [18, 19]. However, no studies have yet indicated whether the teaching effect of a lecturer is better after learning from an online e-book than traditional learning.
Our aim was to use an interactive e-book or PowerPoint to evaluate instructors’ teaching effects on EBM in order to evaluate the effectiveness of a clinical teaching EBM e-book to improve teacher-related skills, as well as to foster both teachers’ and learners’ interest in studying EBM. One month later, we arranged for every teacher to teach an EBM class and to have students assess the effectiveness of their teaching. This method may be a promising and effective way to improve physicians’ EBM-teaching skills in the future.
This study was approved by the institutional review board of the ethics and clinical research committee of CGMH (104-9177B). All teachers and students provided their consent prior to participating in the study. To blind the statistician, the data was anonymized, and participants’ names were removed from the questionnaires before data analysis was performed. Data storage, participant recruitment, and data collection were all performed by a research assistant who had no assessment relationship with the participants.
This study was conducted from September 1, 2016 to October 31, 2017 (participants are shown in Table 1). Visiting staff teachers at Chang Gung Memorial Hospital (CGMH) were enrolled in this study, and two hundred teachers in the hospital were randomly selected by our group, choosing one for every 2.5 with the 500 names listed on our hospital’s computer. Those who did not consent to the assessment were excluded. The flow diagram of this study and its procedure is shown in Fig. 1. All teachers and students provided their consent before participation after understanding the study. A research assistant obtained the consent, and participants had the right to withdraw their data and consent at any time.
Tools: e-book or PowerPoint for teachers’ learning and teaching
Our team designed an online e-book using SimMAGIC software. The e-book contains an introduction to the basic concepts of EBM, EBM databases, database literature search skills, critical appraisal, and effectiveness evaluation methods using a repeatable answer-correct quiz [20, 21] (see availability of data and materials). The structure included EBM definition, EBM five steps, searching skills, RCT, systematic review, and four tests. The content for the e-book and PowerPoint was based on the EBM skills found on our hospital’s website at https://www1.cgmh.org.tw/library_s/EBMRdatabase.htm. (This cannot be accessed by Internet Explorer, but only Google Chrome, all rights reserved.)
A repeatable answer-correct quiz was provided to evaluate each of the learning objectives at the end of each topic. The learners can try the test multiple times to have better memory. Problem sets were created for readers and used for self-assessment purposes. They were presented in a multiple-choice question format, and readers could respond by clicking to answer the question. Another traditional method (PowerPoint) was designed for use as a control group (Additional file 1). The PowerPoint learning material was designed with the same contents as that of the e-book and also included the basic concepts of EBM, EBM databases, database literature search skills, critical appraisal methods, clinical application, and effectiveness evaluation. In the PowerPoint presentation, the questions were presented in a multiple-choice format but responses could not be provided by clicking.
In order to provide teachers with a better method of learning EBM and then assess their teaching effects through students’ feedback, a flipped classroom model was designed with an e-book or PowerPoint for teachers’ learning. In order to decrease the influence of students’ characteristics, the effect of teachers’ instruction was assessed by the same three participating 7th year students. It is very difficult to assess teachers’ day of the week and level of fatigue. Nevertheless, the study was appointed before teachers entered the study so they could answer the Fresno test and have a teaching class, thus minimizing the effect of other potentially confounding factors.
The teachers created an answerable question using the PICO model from a true case in the hospital and searched the electronic databases available at CGMH prior to the study. They attempted to consolidate the evidence in order to reach a conclusion and make a recommendation for the question. We then gave them access to the e-book or PowerPoint presentation and encouraged them to practice using it. They re-examined the information they had written by searching, consolidating evidence, and correcting the recommendation. Afterward, the assessment questionnaires shown in Table 2 were completed. The study group utilized the e-book, while the control group used the PowerPoint presentation without the e-book.
A randomized controlled trial was established with a 1:1 allocation ratio to assess the teachers before they were assigned to either the e-book group (n = 40) or the control group (n = 40). Randomization was carried out (allocation concealment) through central randomization performed by an independent randomizer (assistant). Random assignment with a two-blocked design in the order of their entry into the study was adopted using http://www.randomizer.org. The teachers were blinded to the purpose of the study.
We employed an evaluation method using a structured paper questionnaire of the Modified Fresno test (Table 2) to investigate our teachers’ EBM skills both before and after the e-book or PowerPoint learning. The Fresno test questionnaire included three dimensions (knowledge, search abilities, and critical appraisal skills), with a total of 12 items (Table 2). The scale for each item was 0–10, the total scale for one Fresno test was 120. Teachers then practiced the EBM steps to search, critically appraise, and analyze the level of evidence for the article they found, make a recommendation, and apply it to a clinical situation.
One month later, each teacher gave a lesson on EBM according to the e-book (see availability of data and materials) or PowerPoint content shown in the S1 Appendix.
Student assessment of the teachers’ EBM teaching effect
One month later, each teacher gave a 20 to 30-minute lesson on EBM regarding the formulation of problems, database search techniques, and literature appraisal for three students, who then assessed their teaching skills and ability. We provided eight statements for the three students for each teacher that they could answer with the following options: strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. We converted the scale into Likert scale with strongly agree 5, agree 4, neither agree nor disagree 3, disagree 2 and strongly disagree 1. The eight statements were as follows: “I am satisfied with this lesson,” “The teaching was of high quality,” “The teacher had a good attitude,” “This was a good teaching method,” “It helped my understanding,” “I was allowed to ask unlimited questions,” “The teacher listened to my questions,” and “It aroused my interest in EBM.”
The content validity of the EBM skill questionnaire (Table 2) was examined by three experts who had no involvement with the participants and were asked to rate each item. The final survey included only items with strong relevance, which was defined in our previous papers [20,21,22]. The questionnaire had a content validity index of 0.90.
The sample size analysis was calculated using power analysis and sample size (G-Power) software with at least an effective sample size with 80 % power at the 5 % significance level (2-sided test) in each group for EBP, with the hypothesis of no difference between the two groups. Based on previous results (Chen CJ. Journal of Medical Education 2020) that showed an estimated standard deviation of 2.4 for the e-book group, respectively 2.5 for the PowerPoint group, a sample size of 16 teachers in each group would be able to detect a difference of 2.4 between the two groups. . The evaluation of both EBM skills and teaching effects was analyzed as a continuous variable and expressed as mean ± standard error or mean difference. We adopted Student’s t-test to compare teachers’ evaluation of their EBM skills by Fresno test before the class, after the class and the students’ assessment of the teachers’ instruction. We adopted repeated measures ANCOVA to compare teachers’ evaluation of their EBM skills by Fresno test both before and after the class. The teachers were divided into sub-groups according to the length of their clinical experience: less than 5 years, 5–10 years, and more than 10 years of experience. A p-value < 0.05 was considered statistically significant. All statistical analyses were performed using Statistical Package for Social Science (SPSS, version 12) software.
Two hundred teachers were assessed for eligibility, but 120 ultimately did not consent to the assessment due to being too busy with work. The 80 teachers who provided their consent were enrolled and randomly assigned to the e-book group (n = 40) or the traditional PowerPoint group (n = 40). Seven teachers were excluded for incomplete assessments. The final sample included 35 teachers in the e-book group and 38 in the PowerPoint group (Fig. 1). The basic demographic data of all the teachers are shown in Table 1.
Analysis of the EBM skills of teachers with the pre- and post-Modified Fresno test
Prior to the class, the responses of each group assessed by the Modified Fresno test did not differ. After the class, repeated measures ANCOVA have been used in the second part of Table 2: the difference between the post-test value and the pre-test value for every teacher was then used to compare the two groups. The Modified Fresno test showed significant different total scores in the 12 questions between the e-book and PowerPoint groups (88.4 ± 1.75 vs. 82.3 ± 2.04, 95 %CI: 0.710-11.389, effect size = 2.999, p = 0.027). In particular, the fourth question about chose a case to explain which kind of research design was used for the study type of the question and explaining that choice (P = 0.011), and the seventh question about the important results expressed in the articles found (P = 0.023) significantly different between the two groups. (Table 2) But, only the fourth question differed significantly between the two groups compared the post-test to pre-test (P = 0.024) (Table 2).
Teaching effects with feedback from students
Teaching effects were better in the e-book group than in the PowerPoint group according to feedback from students with regard to four items. Those items were as follows: “I am satisfied with this lesson” (mean difference 0.226, P = 0.015), “The teaching was of high quality” (mean difference 0.285, P = 0.004), “This was a good teaching method” (mean difference 0.225, P = 0.027), and “It aroused my interest in EBM” (mean difference 0.358, P < 0.001) (Fig. 2a).
In the sub-group of physicians with less than 5 years’ experience, students’ feedback was better in “It aroused my interest in EBM” in the e-book group than in the PowerPoint group (difference 0.415, P = 0.007) (Fig. 2b).
In the physicians with 5–10 years’ experience, the students gave the e-book group higher ratings than the PowerPoint group for the following six items: “I am satisfied with this lesson” (difference 0.601, P < 0.001), “The teaching was of high quality” (difference 0.593, P = 0.001), “The teacher had a good attitude” (difference 0.390, P = 0.012), “This was a good teaching method” (difference 0.869, P < 0.001), “It helped my understanding” (difference 0.407, P = 0.012), and “It aroused my interest in EBM” (difference 0.569, P < 0.001) (Fig. 2c).
However, we observed no difference in students’ feedback for the physicians with more than 10 years’ experience between the two groups (Fig. 2d).
Improving teaching knowledge and problem-solving skills is important for teachers. Our study showed that the interactive e-book for EBM learning had better learning effects than those in the PowerPoint group according to the Modified Fresno test with regard to choose a case to explain which kind of research design was used for the study type of the question and explaining that choice. The interactive e-book learning improved the effectiveness of teaching in terms of greater satisfaction, better teaching quality, better teaching methods, and arousing students’ interest in EBM when compared to the traditional PowerPoint method. However, the groups with teachers who had less than 5 years’ experience and those with more than 10 years’ experience demonstrated fewer teaching differences.
The cause of the different learning and teaching effects regarding teachers’ seniority is not clear in this study, but may be related to the Taiwan Evidence-Based Medicine Association being established in 2007 . We suppose that the different learning and teaching effects in the teachers’ seniority is the result of the promotion of EBM in Taiwan. One possible reason that physicians with 5–10 years’ experience are better at teaching is that the e-learning tutorial can provide an efficient and effective means of information delivery to junior doctors . Therefore, they can share their experience of learning EBM, making it more interesting to their students.
The most commonly ranked barriers to the application of EBP include insufficient time, lack of skill in literature searching, and lack of skill in critical appraisal [28, 29]. In our study, the e-book group did not experience the problem of time, which offered significant improvement to the teachers because they could use it whenever convenient. The low rate of participation in our study may be due to physicians being too busy with work, and this may be improved for assessment on non-working days.
One previous study reported that approximately 30 % of residents did not complete their EBM exercises . This finding indicates that a better teaching method is needed to improve teachers’ skills for students’ learning. Our interactive multimedia Internet e-book seems to meet this requirement. E-learning with structured materials provides predefined problem sets and has better student learning results [14, 31]. An e-learning approach to educating students to meet the criteria for evidence-based practice can result in higher-quality search strategies and improve confidence in EBM skills [32, 33]. The functions available in our e-book, combined with interactive quizzes and animations, resulted in improved learning effects in certain EBM skills.
In some cases, e-learning that uses e-books may have such challenges as cost and skill training , but our free Internet e-book does not have such problems. We used SimMAGIC to edit the e-book, which is simple and has a clear format. The cost of this software is inexpensive, and learners can use this e-book for free on the Internet . Skills training in our e-book is not difficult, and as Persky has reported, an e-book is convenient in that it allows learners to pace themselves and have an active role in their learning . Clear evidence from our data indicates that the teachers, especially those with 5–10 years’ experience, were able to extrapolate some of the knowledge gained from this practice.
The use of interactive e-books in clinical teaching can enhance a teacher’s EBM skills, especially those with 5–10 years of clinical experience, though not in more senior physicians. This may suggest that teaching methodology and activities differ for teachers’ varying years of experience.
This study has certain limitations that are worth noting. Since it is a single-site study that uses a specifically designed instrument, our findings need to be tested in a large-scale study for generalizability to other populations in different locations. The low rate of participation in our study is another limitation that needs further consideration when interpreting its results.
Availability of data and materials
The Modified Fresno test and students’ feedback data during the current study are available from the corresponding author upon reasonable request. The e-book is available at https://ebooks.hamastar.com.tw/Upload/ebook/20413/Html5/index.html. After downloading SilverLight_x64, go to the web address via Windows Internet Explorer (it is not available via Google Chrome).
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ 1996, 312(7023):71–72.
Grahame-Smith D: Evidence based medicine: Socratic dissent. BMJ 1995, 310(6987):1126–1127.
Davis DA, Thomson MA, Oxman AD, Haynes RB: Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995, 274(9):700–705.
Haigh R, Appleford J, Bond A: The inhumanity of medicine. Medical education is brutalising. BMJ 1995, 310(6978):527.
Sackett DL, Straus SE: Finding and applying evidence during clinical rounds: the “evidence cart”. JAMA 1998, 280(15):1336–1338.
Dickersin K, Straus SE, Bero LA: Evidence based medicine: increasing, not dictating, choice. BMJ 2007, 334 Suppl 1:s10.
McTavish J: Negotiating concepts of evidence-based practice in the provision of good service for nursing and allied health professionals. Health Info Libr J 2017, 34(1):45–57.
Hsiao CC, Tiao MM, Chen CC: Using interactive multimedia e-Books for learning blood cell morphology in pediatric hematology. BMC Med Educ 2016, 16(1):290.
Chiu YW, Weng YH, Lo HL, Shih YH, Hsu CC, Kuo KN: Impact of a nationwide outreach program on the diffusion of evidence-based practice in Taiwan. Int J Quality Health Care 2010, 22(5):430–436.
McGowan JL, Grad R, Pluye P, Hannes K, Deane K, Labrecque M, Welch V, Tugwell P: Electronic retrieval of health information by healthcare providers to improve practice and patient care. Cochrane Database Systematic Reviews 2009(3):CD004749.
Hoogendam A, Stalenhoef AF, Robbe PF, Overbeke AJ: Answers to questions posed during daily patient care are more likely to be answered by UpToDate than PubMed. J Med Internet Research 2008, 10(4):e29.
Leung K, Trevena L, Waters D. Systematic review of instruments for measuring nurses’ knowledge, skills and attitudes for evidence-based practice. J Adv Nurs. 2014;70(10):2181–95.
Alur P, Fatima K, Joseph R: Medical teaching websites: do they reflect the learning paradigm? Med Teach 2002, 24(4):422–424.
Worm BS, Jensen K: Does peer learning or higher levels of e-learning improve learning abilities? A randomized controlled trial. Med Educ Online 2013, 18:21877.
Childs S, Blenkinsopp E, Hall A, Walton G: Effective e-learning for health professionals and students–barriers and their solutions. A systematic review of the literature–findings from the HeXL project. Health Info Libr J 2005, 22 Suppl 2:20–32.
Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM: Internet-based learning in the health professions: a meta-analysis. JAMA 2008, 300(10):1181–1196.
Lahti M, Hatonen H, Valimaki M: Impact of e-learning on nurses’ and student nurses knowledge, skills, and satisfaction: a systematic review and meta-analysis. Int J Nurs Stud 2014, 51(1):136–149.
Stirling A, Birt J: An enriched multimedia eBook application to facilitate learning of anatomy. Anatomical sciences education 2014, 7(1):19–27.
Sinclair P, Kable A, Levett-Jones T: The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol. JBI Database Syst Reviews Implementation Reports 2015, 13(1):52–64.
Huang HL, Chou CP, Leu S, You HL, Tiao MM, Chen CH: Effects of a quasi-experimental study of using flipped classroom approach to teach evidence-based medicine to medical technology students. BMC Med Educ 2020, 20(1):31.
Chen CJ, Tang KS, Huang YH, Tiao MM: An Interactive E-book Learning Model Effectively Trains Pediatric Postgraduate Year 1 Residents in Evidence-based Medicine Skills. J Med Educ 2020, 24(1):39–51.
Tsai JM, Wu YH, Yu S, Li JY, Buttrey MJ: Validated Chinese Translation of the Fresno Test for Evidence-based Health Care Training. Int J Gerontol 2014, 8(4):209–212.
Cheng HM, Guo FR, Hsu TF, Chuang SY, Yen HT, Lee FY, Yang YY, Chen TL, Lee WS, Chuang CL et al: Two strategies to intensify evidence-based medicine education of undergraduate students: a randomised controlled trial. Annals Acad Med 2012, 41(1):4–11.
Johnston JM, Leung GM, Fielding R, Tin KY, Ho LM: The development and validation of a knowledge, attitude and behaviour questionnaire to assess undergraduate evidence-based practice teaching and learning. Med Educ 2003, 37(11):992–1000.
Kersten HB, Frohna JG, Giudice EL: Validation of an Evidence-Based Medicine Critically Appraised Topic Presentation Evaluation Tool (EBM C-PET). J Graduate Med Educ 2013, 5(2):252–256.
Wu PY CC, Yang TY, Yang YC, Tsai JJ, Huang YF: Evidence-based Health Care (EBHC) Teaching and Teachers’ Perspectives towards Teaching EBHC for Undergraduate Medical Students: a Nationwide Survey in Taiwan. J Med Educ 2019, 23(2):85–99.
Egro FM: Basic Burns Management E-Learning: A New Teaching Tool. J Burn Care Res 2017, 38(4):e715-e721.
Chiu YW, Weng YH, Lo HL, Hsu CC, Shih YH, Kuo KN: Comparison of evidence-based practice between physicians and nurses: a national survey of regional hospitals in Taiwan. J Contin Educ Health Prof 2010, 30(2):132–138.
Guyatt GH, Meade MO, Jaeschke RZ, Cook DJ, Haynes RB: Practitioners of evidence based care. Not all clinicians need to appraise evidence from scratch but all need some skills. BMJ 2000, 320(7240):954–955.
Barnes J: Short-term health and social care benefits of the Family Nurse Partnership lack evidence in the UK context but there is promise for child developmental outcomes. Evidence-Based Med 2016, 21(4):145.
Persky AM, Pollack GM: Transforming a large-class lecture course to a smaller-group interactive course. Am J Pharm Educ 2010, 74(9):170.
Schilling K, Wiecha J, Polineni D, Khalil S: An interactive web-based curriculum on evidence-based medicine: design and effectiveness. Fam Med 2006, 38(2):126–132.
Raynor M, Iggulden H: Online anatomy and physiology: piloting the use of an anatomy and physiology e-book-VLE hybrid in pre-registration and post-qualifying nursing programmes at the University of Salford. Health Info Libr J 2008, 25(2):98–105.
We would like to thank Professor Lynn Monrouxe, Margaret Marie Drouaillet, Maggie DeMenna and Li-Tung Huang for their help with the revision and editing of this manuscript to improve its quality. We appreciated the Biostatistics Center, Kaohsiung Chang Gung Memorial Hospital for statistics work.
This work was supported by the Chang Gung Memorial Hospital, Taiwan, with the following grants: CMRPG8J0881-2 and CDRPG8F0021.
Ethics approval and consent to participate
This study was approved by the institutional review board of the ethics and clinical research committee of CGMH (104-9177B). All teachers and students provided their consent prior to participating in the study. All methods were carried out in accordance with the relevant guidelines and regulations.
Consent for publication
The informed consent to participate in this study obtained from participants was voluntary and anonymous. Informed consent was obtained for publication of the images in the supplementary file.
The authors declare that they have no conflicts of interest in relation to this study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Liao, YH., Tang, KS., Chen, CJ. et al. A study on how using an interactive multimedia e-book improves teachers’ ability to teach evidence-based medicine depending on their seniority. BMC Med Educ 21, 547 (2021). https://doi.org/10.1186/s12909-021-02984-2
- Evidenced-based medicine