This article has Open Peer Review reports available.
Implementation of a blended learning approach to teaching evidence based practice: a protocol for a mixed methods study
© Ilic et al.; licensee BioMed Central Ltd. 2013
Received: 7 October 2013
Accepted: 16 December 2013
Published: 19 December 2013
Evidence based practice (EBP) requires that health professionals are competent in integrating the best evidence in their decision making. Being 'evidence-based’ requires skills and knowledge in epidemiology, biostatistics and information literacy. EBP is commonly taught in medical and health sciences degrees, yet there is little evidence to guide educators as to the best teaching modality to increase learner competency in EBP.
This study is mixed methods in design. A randomised controlled trial will examine the effectiveness of blended learning versus didactic approach of teaching EBP to medical students. The primary outcome of the RCT is EBP competency as assessed by the Berlin tool. Focus groups will be conducted to explore student perceptions and attitudes towards implementing a blended learning approach in teaching EBP. A concurrent triangulation design will be implemented, permitting quantitative data to inform the effectiveness of the intervention and qualitative data to contextualise the results.
This study will provide novel evidence on the effectiveness of blended learning in teaching EBP to a cohort of undergraduate and graduate-entry medical students.
Asking a clinical question that is constructed using the PICO (patient, intervention, comparison, outcome) framework (ask);
Acquiring the evidence via a systematic and efficient search of the literature (acquire);
Appraising the evidence through application of critical appraisal techniques (appraise);
Applying the evidence to the clinical scenario (apply); and,
Assessing the EBP process as it relates to the clinical context (assess) .
Each step within the EBP process requires a different level of competency (i.e. integration of knowledge, skill, attitude and behaviour) from the practitioner . Achieving a high level of competency in EBP can only be achieved when the practitioner is able to effectively undertake all five steps, which incorporate adequate levels of knowledge, skills, attitude and behavioural elements . Achieving competency in the principles and practices of EBP provides the practitioner with the ability to know when and how to make evidence-based medical decisions, and also achieve lifelong learning within their medical discipline.
Each step of the EBP process requires a different competency to be learnt. This has led to the suggestion that different teaching modalities could be used in its implementation – be it lecture, tutorial, mini-course, problem based or online . Limited evidence currently exists in order to inform educators as to the most effective method of teaching and increasing practitioner competency in EBP. A 2004 systematic review identified two randomised controlled trials (RCTs) and seven non-RCTs that examined the effectiveness of different teaching modalities in EBP . The authors of that review concluded that standalone teaching improved student knowledge, but not skills, attitudes or behaviour in EBP. Conversely, evidence from the non-RCTs indicated that integrating teaching of EBP with clinical activities (i.e. blended learning) was associated with improvements across all four domains (i.e. knowledge, skills, attitudes and behaviour) .
A 2013 systematic review of the literature identified 9 RCTs that compared the effectiveness of various teaching modalities (lecture, tutorial, self-directed, online, problem-based, uni and multidisciplinary) in training medical students in EBM . The review concluded that training in EBP increased student competency, but identified a lack of evidence to guide educators on which teaching modality is best at increasing EBM competency.
A variety of factors may dictate how an EBP course is implemented in medical education – be it student learning styles, infrastructure or other organisational issues. The ability to achieve competency in EBP also requires that students have a certain level of mastery in epidemiology, biostatistics, informatics and information literacy. Given the impact of these factors, and the challenge in achieving competency in a variety of areas, teaching EBP requires a multidimensional approach. Blended learning attempts to create an optimal learning environment by blended a variety of learning approaches (lecture, tutorial, online, problem-based etc.) .
Determine the competency of students in EBP receiving EBP training via a blended learning approach compared to students receiving EBP training via a didactic approach, and;
Determine student self-efficacy, perceptions and attitudes on EBP teaching delivered through a blended learning approach.
This study is a mixed methods study, incorporating a randomised controlled trial and a qualitative case study at Monash University. A mixed methods approach will permit quantitative data to inform the effectiveness of the intervention, whilst qualitative data will contextualise those results addressing issues of 'how’ and 'why’ . The protocol for this current study was informed by a pilot study on graduate-entry medical students, which has since been accepted for publication by BMC Medical Education.
Settings and participants
A multi-campus study will be conducted with medical students currently enrolled in the MBBS course at Monash University. Monash University runs undergraduate and graduate-entry MBBS programs, in both Australia and Malaysia. Students are assigned to one of seven metropolitan hospitals, or six rural, hospitals in Australia (with one site in Malaysia). Participants for this study will be third year medical students, who are entering their first year of clinically-based training and first year of formal EBP training.
Quantitative research methodology
The following outlines the protocol for the randomised controlled trial aspect of the study.
Third year medical students are randomly placed in small tutorial groups for their EBP teaching (approximately 20–25 in a group). Tutorial groups will be randomised to receive EBP teaching adopting a blended learning approach, or the traditional didactic small group learning approach. Students not wishing to participate in the study will be taught via the traditional didactic approach and not be asked to complete an outcome assessment.
Participants will be randomised according to their tutorial group (i.e. cluster) by a researcher independent to the study utilising a simple cluster randomisation procedure (computerised random numbers). All students will have access to the blended learning materials at the end of the study period to ensure parity between groups.
Difference in learning activities between blended (intervention) and non-blended (comparison) groups
Blended learning (intervention)
Non-blended learning (comparison)
1. Prior to lesson
• Students view specific presentations relevant to the upcoming tutorial via YouTube
2. Designated EBP lesson
• Tutor facilitates discussion with students on key concepts to be covered in the module
• Tutor presents 30–40 PowerPoint presentation on key concepts to be covered in the module
• Students are divided into small groups (4–5 members per group)
• Tutor facilitated discussion of a previously pre-appraised article
• Students divided into small groups (4–5 members per group) and critically appraise an article within the tutorial
• Tutor led discussion on the critical appraisal performed by students in-class
3. Post EBP lesson
• Group members to identify patient during other 'bedside teaching’
• Self-directed learning
• Whilst at the 'bedside’, students to access evidence (via mobile learning) relevant to their patient to inform decision making
• Groups prepare oral presentation, based on their patient scenario, outlining clinical and EBP content learnt
4. Prior to lesson
• Students view specific presentations relevant to the upcoming tutorial via YouTube
5. Designated EBP lesson
• Each group presents learning experience based on their specific patient scenario during 'bedside teaching’
• Lesson repeated as per stage 2 (with new content)
• Tutor facilitates discussion based on content raised in group presentations
• Lesson repeated as per stage 2 (with new content)
Students randomised to the intervention group will receive the same theoretical concepts taught in the control group, but in a blended approach. The blended learning method integrates (i) current classroom activities (lecture/tutorial) with (ii) online and (iii) mobile learning. The online component will be provided through specific resources delivered via the Monash library website, as well as specifically designed online lectures, made available through YouTube, that students view prior to attending the respective two hour teaching block . The mobile learning component will be delivered when students are on their ward rounds and interacting with patients during their existing day-to-day 'bedside teaching’ schedule. During the mobile learning, students will be required to take a detailed medical history from the patient, as they normally would during their 'bedside’ teaching, to which they will apply their EBP learning. Students will present their learning during the next two hour teaching block. The methodology for this intervention has previously been piloted in 2010–11 . Whilst a two-hour block of time will be dedicated to teaching using the blended learning approach, it is anticipated that only one hour of that time will be spent in the tutorial setting, with the other hour available for students to implement their mobile and online learning activities.
Student competency in EBP will be assessed by a blinded outcome assessor using the validated Berlin tool . Student self-efficacy will be assessed using Evidence-Based Practice Question (EBPQ) . The EBPQ is a self-reported measure of implementation of EBP, with measures relating to self-efficacy, behaviour and attitudes toward EBP.
Due to the educational nature of the intervention, it is not possible to blind either the educators or the students. The outcome assessor and data analyst will be kept blinded to the allocation.
Sample size calculation
A minimum of 120 students per arm (40 from each of metropolitan Melbourne (undergraduate), rural Victoria (graduate) and Monash Malaysia (international)), will be recruited to detect a 50% difference in EBP competency (α = 0.05, β = 0.80, σ = 2.8) between groups.
Quantitative data will be analysed using the principle of intention-to-treat. Mean differences in EBP competency, as determined by the Berlin tool, between intervention and control groups will be explored using a Student’s t-test. Differences between intervention/control groups and student type (undergraduate/graduate/international) will be explored using one and two-way ANOVA’s, where appropriate.
Ethical approval for this study has been obtained from the Monash University Human Research Ethics Committee.
Qualitative research methodology
The following outlines the protocol for the qualitative case study of the research.
At the conclusion of the RCT, students from the 13 Australian hospital sites who received the intervention will be invited to participate in focus groups using a convenience sampling approach . Participants will be required to provide written consent prior to their participation in the qualitative case study.
A minimum of three focus groups composed of students based at metropolitan and rural hospital sites (including Australian and Malaysian study sites), will be performed by an independent facilitator. All focus group discussions will be guided by a semi-structured interview schedule (Appendix 1). Each focus group will be comprised of between six and eight students, and will be digitally recorded and transcribed verbatim at the conclusion of the focus groups. Focus groups will be performed until the data reaches a point of theoretical saturation (i.e. no new discussion points are raised, which haven’t already been raised in previous focus groups) .
Transcripts from all focus groups will be analysed independently by two researchers using thematic analysis . The thematic analysis will comprise of a six step approach including; (i) familiarization of the data achieved by reading each transcript, (ii) generation of preliminary codes, (iii) searching for themes from the preliminary codes, (iv) creation of thematic map, (v) specific definition and naming of themes and (vi) final analysis . The two sets of thematic analysis will be compared before a final iteration of thematic analysis is performed and final themes generated and consensus is reached. Thematic analysis will be assisted with the use of the NVivo software program . Data analysis of the quantitative and qualitative results will be performed utilising a concurrent triangulation design . This approach permits immediate collation and analysis of data, promoting triangulation of the data, with the quantitative results determining the effectiveness of the intervention, whilst the qualitative data contextualising the results and probing the participants’ perceptions and attitudes toward the intervention .
It has been more than 20 years since the term 'evidence based practice’ first emerged, yet there is currently little evidence to guide educators on how to teach EBP to health professionals. This study will provide novel RCT evidence regarding the effectiveness of blended learning as a methodological approach for teaching EBP in medical schools. It will also provide evidence on the effectiveness of blended learning across medical student cohorts; be it undergraduate, graduate and international students. It will also provide valuable qualitative data to contextualise results, particularly student perception of competency in EBP.
What type of material do you prefer to be taught in; (and why)
How is EBM best delivered (probe reasoning for responses)
Would 'flip’ learning suit the teaching of EBM (i.e. pre-loading information before class)? (probe reasoning for responses)
Should EBM teaching be blended with;
PBL (problem based learning)
What aspects of the EBM program work best? (Probe responses)
How can the teaching of EBM be improved? (Probe responses)
Support for this publication/activity has been provided by the Australian Government Office for Learning and Teaching. The views expressed in this publication do not necessarily reflect the views of the Australian Government Office for Learning and Teaching. We would like to thank Professor Upton and Dr. Upton for the use of the EBPQ.
- Finkel M, Brown H, Gerber L, Supino P: Teaching evidence-based medicine to medical students. Med Teach. 2003, 25: 202-209. 10.1080/0142159031000092634.View ArticleGoogle Scholar
- Straus S, Glasziou P, Richardson W, Haynes B: Evidence-based medicine: how to practice and teach it. 2011, Edinburgh: Churchill Livingstone: ElsevierGoogle Scholar
- Ilic D: Assessing competency in Evidence Based Practice: strengths and limitations of current tools in practice. BMC Med Educ. 2009, 9: 53-10.1186/1472-6920-9-53.View ArticleGoogle Scholar
- del Mar C, Glasziou P, Mayer D: Teaching evidence based medicine. BMJ. 2004, 329: 989-10.1136/bmj.329.7473.989.View ArticleGoogle Scholar
- Coomarasamy A, Khan K: What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ. 2004, 329: 1017-10.1136/bmj.329.7473.1017.View ArticleGoogle Scholar
- Ilic D, Maloney S: Methods of teaching medical trainees Evidence Based Medicine: A systematic review. Med Educ. In pressGoogle Scholar
- Masie E: Blended learning: the magic is in the mix. The ASTD e-learning handbook. Edited by: Rossett A. 2002, New York: McGraw-HillGoogle Scholar
- Tashakkori A, Creswell J: Designing and conducting mixed methods research. J Mixed Methods Res. 2007, 1: 3-8. 10.1177/2345678906293042.View ArticleGoogle Scholar
- Monash University: Evidence Based Practice. [internet] Accessed 25/06/2013; Available at; http://guides.lib.monash.edu/medicine
- Ilic D, Hart W, Fiddes P, Misso M, Villanueva E: Adopting a blended learning approach to teaching Evidence Based Medicine: a mixed methods study. BMC Med Educ. 2013, 13: 169-10.1186/1472-6920-13-169.View ArticleGoogle Scholar
- Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer H, Kunz R: Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. BMJ. 2002, 325: 1338-1341. 10.1136/bmj.325.7376.1338.View ArticleGoogle Scholar
- Upton D, Upton P: Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006, 53: 454-458. 10.1111/j.1365-2648.2006.03739.x.View ArticleGoogle Scholar
- Silverman D: Qualitative Research: Theory, method and practices. 2004, London: SAGE PublicationsGoogle Scholar
- Braun V, Clarke V: Using thematic analysis in psychology. Qual Res Psychol. 2006, 3: 77-101. 10.1191/1478088706qp063oa.View ArticleGoogle Scholar
- QSR International: NVivo qualitative data analysis software. 2012, Melbourne: QSR International Pty Ltd, 10Google Scholar
- Rauscher L, Greenfield B: Advancements in contemporary physical therapy research: use of mixed methods designs. Phys Ther. 2009, 89: 91-100. 10.2522/ptj.20070236.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/13/170/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.