Characteristics of evidence-based medicine training in Royal College of Physicians and Surgeons of Canada emergency medicine residencies - a national survey of program directors
© Bednarczyk et al.; licensee BioMed Central Ltd. 2014
Received: 9 December 2013
Accepted: 18 March 2014
Published: 21 March 2014
Recent surveys suggest few emergency medicine (EM) training programs have formal evidence-based medicine (EBM) or journal club curricula. Our primary objective was to describe the methods of EBM training in Royal College of Physicians and Surgeons of Canada (RCPSC) EM residencies. Secondary objectives were to explore attitudes regarding current educational practices including e-learning, investigate barriers to journal club and EBM education, and assess the desire for national collaboration.
A 16-question survey containing binary, open-ended, and 5-pt Likert scale questions was distributed to the 14 RCPSC-EM program directors. Proportions of respondents (%), median, and IQR are reported.
The response rate was 93% (13/14). Most programs (85%) had established EBM curricula. Curricula content was delivered most frequently via journal club, with 62% of programs having 10 or more sessions annually. Less than half of journal clubs (46%) were led consistently by EBM experts. Four programs did not use a critical appraisal tool in their sessions (31%). Additional teaching formats included didactic and small group sessions, self-directed e-learning, EBM workshops, and library tutorials. 54% of programs operated educational websites with EBM resources. Program directors attributed highest importance to two core goals in EBM training curricula: critical appraisal of medical literature, and application of literature to patient care (85% rating 5 - “most importance”, respectively). Podcasts, blogs, and online journal clubs were valued for EBM teaching roles including creating exposure to literature (4, IQR 1.5) and linking literature to clinical practice experience (4, IQR 1.5) (1-no merit, 5-strong merit). Five of thirteen respondents rated lack of expert leadership and trained faculty educators as potential limitations to EBM education. The majority of respondents supported the creation of a national unified EBM educational resource (4, IQR 1) (1-no support, 5- strongly support).
RCPSC-EM programs have established EBM teaching curricula and deliver content most frequently via journal club. A lack of EBM expert educators may limit content delivery at certain sites. Program directors supported the nationalization of EBM educational resources. A growing usage of electronic resources may represent an avenue to link national EBM educational expertise, facilitating future collaborative educational efforts.
KeywordsEvidence-based medicine Medical education Emergency medicine E-learning Journal club
Royal College of Physicians and Surgeons of Canada emergency medicine training requirements in evidence-based medicine
Requirements for completion of training
Access and interpret the relevant evidence
Critically appraise evidence to address a clinical question
Integrate critical appraisal conclusions into clinical care
Apply best available medical evidence and management for cost-appropriate care
Plan relevant emergency department operations based upon evidence gathered through the use of information technology
EBM education in EM remains in the early stages of development. A recent survey of the Council of Emergency Medicine Residency Directors in the United States (US) indicates a minority of programs had predefined journal club or EBM curricula (25%) or used structured critical appraisal instruments in their journal clubs (29%) . Kuhn and colleagues reported only 22% of EM program directors in the US conducted more than five didactic EBM sessions per year . In both studies, program directors expressed a desire for a defined curriculum [6, 7]. Such work has established a baseline in teaching methods and outlined a needs assessment for training programs in the US, however this first step has not been performed in Canada.
In EM residencies, EBM content delivery has been described and formats include traditional journal club, structured “evidence-based” journal club, didactic formats, and web based EM blogs [8–11]. Specific concerns have been raised regarding the narrow focus placed on critical appraisal during journal club sessions, however the optimal educational method has not been defined [12, 13]. Adding complexity, the rapid expansion of web based educational technology has changed the way learners access, analyze, and catalogue new evidence .
Before evaluating EBM training in EM, current practices must be defined. Our primary objective was to describe the methods of EBM training in RCPSC-EM residencies. Secondary objectives were to (i) explore attitudes regarding current educational practices, including e-learning, (ii) explore potential barriers to journal club and EBM education, and (iii) assess the desire for national collaboration. By defining educational methods across institutions and identifying areas for growth, we hope to create a platform for local and national quality improvement initiatives relevant to EBM education in EM.
Study design and setting
A paper-based survey of all RCPSC-EM program directors in Canada was performed. Ethics approval was obtained from the University of Manitoba Health Research Ethics Board. Contact information for programs was obtained from the Canadian Resident Matching Service (CaRMS) website . A cover letter stated survey completion was adequate evidence of informed consent. Respondents received a fifty-dollar honoraria directed to their resident education fund.
Survey content and administration
An anonymous 16-question survey containing binary, open ended, and 5 point likert-scale questions was developed by the authors. Survey content was refined by consensus discussion during local focus group meetings and informal collaboration with national EBM experts in EM. We explored the following domains: (i) EBM curricula format and core values, (ii) journal club characteristics and limitations, (iii) use and perceived merit of electronic/web based EBM resources, and (iv) basic demographics. All likert scales provided opportunity for neutral responses. Several questions offered “other” selections and free text to elicit alternate responses.
Program directors were mailed introductions to the research initiative beginning in July 2012. Surveys were mailed one week later with a cover letter indicating estimated completion time, voluntariness, honoraria value, and confidentiality via aggregate data reporting. Potential respondents were provided postage paid return envelopes addressed to an administrative assistant who removed identifying packaging before returning data to the authors. Non-respondents received two bimonthly email requests to complete the survey. Responses were collected until August 2012.
Data were stored using Numbers ’09 (Apple Inc. Version 2.1 (c) 2008-2011). Descriptive statistical analysis was performed. Because of the obligatory small sample size and descriptive nature of the study objective, formal sample size calculations and statistical significance were not performed. To preserve the integrity of data trends in likert scale questions, results are represented in both median and percentage where applicable.
Demographics of respondents
Age of FRCP residency program
Number of residents in program
Article selection for journal club occurred most frequently via resident or attending choice (85% and 54% respectively). Four programs (31%) had predefined journal club curricula, and only one program reported journal club article selection from clinical questions generated on shift. Six programs (46%) indicated journal club sessions were consistently led by an EBM expert, defined as an individual with formal training in EBM, extensive background in clinical research, or a longstanding academic interest in biostatistics and research methodology. Nine programs (69%) used a critical appraisal tool in their journal club sessions. Respondents who specified their critical appraisal tool used the JAMA User’s Guide (5/9) or Oxford critical appraisal worksheets (1/9) [16, 17]. Sessions were held at staff residences or university sites in most cases (38% and 48% respectively), with only 2 programs (15%) holding journal club at public venues.
Perceived limitations of journal club: n (%) respondents
No limitation 1
Significant limitation 5
Public venue (noise, distractions)
Narrow focus on critical appraisal
Lack of expert leadership
Potential barriers to effective EBM teaching
Insignificant barrier 1
Significant barrier 5
Limited EBM trained faculty educators
Lack of funding for EBM training
Lack of interest in department
To our knowledge, this is the first effort to describe the landscape of EBM training in Canadian EM residencies. Compared with previous reports of EM training programs in the US, EBM teaching curricula were prominent in nearly all survey respondents. While the aim of this study was not to thoroughly review individual curricula, several observations may serve as a basis for self-evaluation and quality improvement initiatives.
The EBM skill set enables an EM trainee to make evidenced-based patient care decisions, but begins with the ability to seek out and appraise resources based on a resident’s clinical questions [12, 18]. A successful training program teaches each step of the EBM process, producing physicians who incorporate EBM as a behavior in their daily practice . Although respondents attributed some importance to each component of EBM, critical appraisal was highly valued. This was also evidenced by the predominance of journal club - a critical appraisal exercise - as the most frequently reported educational activity among EBM curricula.
Much research has focused on journal club as an EBM teaching tool in EM [8–10, 19]. The efficacy of this traditional format to teach the breadth of EBM skills or change behavior has been questioned [10, 11, 20]. In a systematic review of journal club in post-graduate medical education, none of the 101 articles retrieved demonstrated translation of evidence from journal club into the clinical practice of learners . Because of a narrow focus on critical appraisal, journal club may be an unsatisfactory method of teaching the full scope of evidence based decision making . Furthermore, as Green argues: “critical appraisal may be the most expendable skill for the ‘front line’ physician practicing evidence-based medicine in the ‘real world’” . Despite these growing concerns, few respondents in our survey identified the narrow focus on critical appraisal as a significant limitation to their journal clubs or EBM curricula.
Recommendations from an extensive review on improving journal club effectiveness included regular meetings, mandatory attendance, and use of a critical appraisal tool . These respective criteria were reported by 92, 100, and 69% of respondents in our survey. Yet perhaps the most consistent suggestion in EBM education literature is to engage learners in the breadth of the EBM process with applied techniques, beginning and ending at the patient’s bedside [11, 23, 24]. Curricula incorporating experiential learning may best meet the needs of adult learners [22, 23, 25]. As demonstrated in an observational cohort, Friedman et al. described a knowledge translation intervention consisting of a 1 hour tutorial and real time EBM exercise in the emergency department . This encouraging work showed a change in patient management in 16.3% of encounters and the potential to impact future practice in resident learners. Of concern, only one of our respondent programs indicated residents generated journal club topics from clinical questions arising on shift. This may represent an important area for growth in Canadian EM curricula.
Additional teaching formats were used with lesser frequency among respondents - including didactic sessions and library tutorials. Such formats are supported by the literature, as Grant demonstrated a didactic intervention was effective in teaching EBM theory to EM residents . Of much interest, however, is the use and support of e- resources by EM programs in resident education, with over half of programs operating EBM websites and cataloging critically appraised topics. Web 2.0, a term encompassing the collaborative use of information technology (blogs, social media, podcasts) has increasing relevance to EM education . With over twelve thousand MEDLINE articles added weekly, EBM curricula must also educate residents to identify sources of pre-appraised evidence [27, 28]. Peer reviewed sources such as Academic Emergency Medicine, Annals of Emergency Medicine, and Emergency Medicine Journal are among the contributors to over 200 EM blogs (catalogued at http://lifeinthefastlane.com) which may help learners filter this vast amount of information [29–32]. Given the inherent variability in quality of such resources, however, PD’s may wish to develop and incorporate methods of critical appraisal of web-based secondary information sources into resident education.
Our survey demonstrated less than half of programs had consistent leadership by EBM experts at journal club sessions. In addition to being accessible, current, and asynchronous, web-based EBM resources also create a non-hierarchal environment where junior learners can interact with experts in the field, and participate in critical appraisal on a broader scale [14, 33]. Linking respective journal clubs or online EBM resources between institutions may create a bridge for smaller centers with fewer EBM educators to larger institutions with surplus leadership and research experience. Based on the demographic data in this survey, respondents varied widely in program maturity and size. The authors speculate that the five respondents that cited lack of EBM expert leadership and limited faculty educators as limitations to EBM training may belong to younger, smaller programs. Indeed, this further supports the case for linking our nations EBM resources, a proposition met with great support by survey respondents. We feel this may represent an exciting opportunity to transform and strengthen EBM training in Canadian EM, by uniting institutions on a web-based EBM educational platform.
Our survey instrument was not prospectively tested for validity. Respondents represented the lead education coordinators for RCPSC-EM residents, but their opinions may differ from those of respective institutional EBM leaders. Furthermore, because of the relative paucity of specialty EM training programs in Canada, our sample size was small. This may have contributed to type I or II error. Future study defining the values and needs of residents in EBM education may be valuable. Ongoing efforts to identify educational interventions that change behavior in learners have great importance for EBM curricula.
The majority of RCPSC-EM programs have prioritized EBM education by developing dedicated curricula. EBM training is diverse, however journal club and critical appraisal appear to be emphasized. This may not optimize the accomplishment of all core EBM training goals valued by respondents. If continued at the reported frequency, journal club sessions designed for resident education should incorporate applied, patient centered methods based on principles of adult learning theory, increase the use of critical appraisal tools, and assure EBM experts are involved in sessions. Lack of EBM leadership at some centers may hinder optimal resident education. A growing usage and support of web-based resources may represent an avenue to link EBM educators across Canada, and respond to the desire expressed by program directors to create a national unified EBM educational resource.
Royal College of Physicians and Surgeons of Canada
Canadian resident matching service
Learning and teaching formats involving electronic media.
The authors wish to extend their gratitude to Darlene Scibak for contributing significant administrative support to this project. The authors also wish to acknowledge the contributions of Brenden Dufault as a consultant biostatistician. Funding was provided by the University of Manitoba Department of Emergency Medicine Resident Education Fund.
- Rutledge T: Emergency medicine training in Canada: learning from the past to prepare for the future. Can J Emerg Med. 2008, 10: 108-110.Google Scholar
- Ip S: Emergency medicine practice and training in Canada. Can Med Assoc J. 2003, 168: 1549-1550.Google Scholar
- Objectives of Training in Emergency Medicine. http://www.royalcollege.ca/.
- Croskerry P, Sinclair D: Emergency medicine: a practice prone to error?. Can J Emerg Med. 2001, 3: 271-276.Google Scholar
- Newman DH, Wyer P, Kaji A: Evidence-based medicine. A primer for the emergency medicine resident. Ann Emerg Med. 2002, 38: 77-80.View ArticleGoogle Scholar
- Carpenter C, Kane B, Carter M, Lucas R, Wilbur L, Graffeo C: Incorporating evidence-based medicine into resident education: a CORD survey of faculty and resident expectations. Acad Emerg Med. 2010, 17 (S2): S54-S61.View ArticleGoogle Scholar
- Kuhn G, Wyer P, Cordell W, Rowe B: A survey to determine the prevalence and characteristics of training in evidence-based medicine in emergency medicine residency programs. J Emerg Med. 2005, 28: 353-359. 10.1016/j.jemermed.2004.09.015.View ArticleGoogle Scholar
- Burstein J, Hollander J, Barlas D: Enhancing the value of journal club: use of a structured review instrument. Am J Emerg Med. 1995, 14: 561-563.View ArticleGoogle Scholar
- Carley SD, Mackway-Jones K, Jones A, Morton RJ, Dollery W, Maurice S, Niklaus L, Donnan S: Moving towards evidence based emergency medicine: use of a structured critical appraisal journal club. J Accid Emerg Med. 1998, 15: 220-226. 10.1136/emj.15.4.220.View ArticleGoogle Scholar
- Carpenter CR, Katz E, Char D: Re: journal club and teaching evidence-based medicine. J Emerg Med. 2006, 31: 307-308. 10.1016/j.jemermed.2006.06.006.View ArticleGoogle Scholar
- Genes N, Parekh S: Bringing journal club to the bedside in the form of a critical appraisal blog. J Emerg Med. 2010, 39: 504-505. 10.1016/j.jemermed.2008.10.011.View ArticleGoogle Scholar
- Hatala R, Guyatt G: Evaluating the teaching of evidence-based medicine. J Am Med Assoc. 2002, 288: 1110-1112. 10.1001/jama.288.9.1110.View ArticleGoogle Scholar
- Hatala R: Is Evidence-based Medicine a Teachable Skill?. Annals of Emergency Medicine. 1999, 34: 226-228. 10.1016/S0196-0644(99)70236-2.View ArticleGoogle Scholar
- Berger E: Web 2.0 in emergency medicine. Ann Emerg Med. 2012, 59: 21A-23A. 15View ArticleGoogle Scholar
- Canadian Resident Matching Service. http://www.carms.ca.
- Oxford Critical Appraisal Sheets. http://www.cebm.net/index.aspx?o=1157.
- JAMA User’s Guide Critical Appraisal Worksheets. http://jamaevidence.com/resource/520.
- Guyatt G, Rennie D, Meade M, Cook D: User’s Guides to the Medical Literature, A Manual for Evidence Based Clinical Practice: McGraw Hill. JAMA Evidence. 2008, 5-6. 6
- Bazarian J, Davis C, Spillane L, Blumstein H, Schneider S: Teaching emergency medicine residents evidence-based critical appraisal skills: a controlled trial. Ann Emerg Med. 1998, 34: 148-154.View ArticleGoogle Scholar
- Ahmadi N, McKenzie M, MacLean A, Brown CJ, Mastracci T, McLeod RS: Teaching evidence based medicine to surgery residents - is journal club the best format? A systematic review of the literature. J Surg Educ. 2012, 69: 91-100. 10.1016/j.jsurg.2011.07.004.View ArticleGoogle Scholar
- Deenadayalan Y, Grimmer-Somers K, Prior M, Kumar S: How to run an effective journal club: a systematic review. J Eval Clin Pract. 2008, 14: 898-911. 10.1111/j.1365-2753.2008.01050.x.View ArticleGoogle Scholar
- Green M: Evidence-based medicine training in graduate medical education: past, present and future. J Eval Clin Pract. 2000, 6: 121-138. 10.1046/j.1365-2753.2000.00239.x.View ArticleGoogle Scholar
- Friedman S, Sayers B, Lazio M, Friedman S, Gisondi MA: Curriculum design of a case- based knowledge translation shift for emergency medicine residents. Acad Emerg Med. 2010, 17: S42-S48.View ArticleGoogle Scholar
- Hardem RD: Teaching and learning evidence based medicine skills in accident and emergency medicine. J Accid Emerg Med. 1998, 16: 126-129.Google Scholar
- Green M, Ellis P: Impact of an evidence-based medicine curriculum based on adult learning theory. J Gen Intern Med. 1997, 12 (12): 742-750. 10.1046/j.1525-1497.1997.07159.x.View ArticleGoogle Scholar
- Grant DC, Keim S, Telfer J: Teaching Bayesian analysis to emergency medicine residents. J Emerg Med. 2006, 31: 437-440. 10.1016/j.jemermed.2006.04.015.View ArticleGoogle Scholar
- Glasziou P: Information overload: what’s behind it, what’s beyond it?. Med J Aust. 2008, 189 (2): 84-85.Google Scholar
- Rowe BH, Diner B, Camargo C, Worster A, Colacone A, Wyer P: Effective Synthesized/Preappraised evidence formats in emergency medicine and the use of supplemental knowledge translation techniques. Acad Emerg Med. 2007, 14: 1023-1028. 10.1111/j.1553-2712.2007.tb02383.x.View ArticleGoogle Scholar
- Academic Emergency Medicine. http://vimeo.com/aem.
- Annals of Emergency Medicine. http://www.annemergmed.com/content/podcast.
- EMJ Podcasts. http://podcasts.bmj.com/emj/.
- Life in the Fast Lane EMCC Blog Index. http://lifeinthefastlane.com/resources/emergency-medicine-blogs.
- Nickson C: Web 2.0 For Emergency Physicians. http://lifeinthefastlane.com/2009/07/web-20-for-emergency-physicians.
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/14/57/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.