Reinforcing outpatient medical student learning using brief computer tutorials: the Patient-Teacher-Tutorial sequence
© Pusic et al.; licensee BioMed Central Ltd. 2012
Received: 20 January 2012
Accepted: 27 July 2012
Published: 8 August 2012
At present, what students read after an outpatient encounter is largely left up to them. Our objective was to evaluate the education efficacy of a clinical education model in which the student moves through a sequence that includes immediately reinforcing their learning using a specifically designed computer tutorial.
Prior to a 14-day Pediatric Emergency rotation, medical students completed pre-tests for two common pediatric topics: Oral Rehydration Solutions (ORS) and Fever Without Source (FWS). After encountering a patient with either FWS or a patient needing ORS, the student logged into a computer that randomly assigned them to either a) completing a relevant computer tutorial (e.g. FWS patient + FWS tutorial = “in sequence”) or b) completing the non-relevant tutorial (e.g. FWS patient + ORS tutorial = “out of sequence”). At the end of their rotation, they were tested again on both topics. Our main outcome was post-test scores on a given tutorial topic, contrasted by whether done in- or out-of-sequence.
Ninety-two students completed the study protocol with 41 in the ‘in sequence’ group. Pre-test scores did not differ significantly. Overall, doing a computer tutorial in sequence resulted in significantly greater post-test scores (z-score 1.1 (SD 0.70) in sequence vs. 0.52 (1.1) out-of-sequence; 95% CI for difference +0.16, +0.93). Students spent longer on the tutorials when they were done in sequence (12.1 min (SD 7.3) vs. 10.5 (6.5)) though the difference was not statistically significant (95% CI diff: -1.2 min, +4.5).
Outpatient learning frameworks could be structured to take best advantage of the heightened learning potential created by patient encounters. We propose the Patient-Teacher-Tutorial sequence as a framework for organizing learning in outpatient clinical settings.
In existing teaching and learning models for outpatient settings, such as the One-Minute Preceptor and SNAPPS, the emphasis is placed on the preceptor-student interaction and how to extract maximal learning from this time-pressured exchange [1, 2]. The One-Minute Preceptor is a teaching method triggered by the event of a learner assessing a patient. Five teaching behaviors are done by the preceptor with the goal of eliciting and modeling ideal clinical reasoning . The SNAPPS model structures the preceptor-student interaction along similar lines but from more of a learner perspective . In these models, outpatient learning occurs mainly through an apprenticeship model where learners are guided by experts [3, 4].
Outpatient medical student learning is typically based on their clinical interviews of actual patients. The medical student-patient encounter serves to set an agenda for learning. It leads to the activation of prior knowledge about the patient’s complaint and serves to focus the attention of the learner on the content to be learned . Ideally, during this phase, the student generates their own initial diagnostic and management formulation to be presented to the preceptor.
Having developed an initial formulation, the student will then discuss the patient with an expert clinician, a process that, in an Emergency Dept, typically takes less than 5 minutes with only 1-2 minutes being actual teaching of the student [6, 7]. The preceptor will verify important parts of the student’s assessment and together they will arrange the necessary care for the patient [1, 8]. This entire teacher-student interaction benefits from the activation of prior knowledge, identification of knowledge gaps, and high motivation. The instructional strategy is the most powerful available: individual tutoring . However, it is limited by the scarce availability of the preceptor [1, 6].
Immediately after the student and preceptor have separated, the optimal conditions for learning are usually still present. Primed by both the patient encounter and the interaction with the teacher, the student is ideally placed to cement their learning. The motivated student may go on to find appropriate learning resources but in a typical clerkship there is considerable variability in the quality of the student’s search and in the resources available [10, 11]. This teachable moment, which we will call the Tutorial Phase (See Figure 1), could be a shared space between the preceptor and learner, an opportunity to agree on a continuing learning agenda. This is explicitly stated in the SNAPPS model where the last “S” stands for “Select a patient-related issue for further study” . However, exactly how this is to be brought about is not specified in existing learning frameworks [1, 2].
In this research, we sought to study strategies for the patient-triggered use of learning resources by medical students in the clinical setting. Specifically, we wondered whether completing a brief computer tutorial right after the patient encounter can increase their learning to a measurable degree compared with completing the same tutorial but outside the Patient-Teacher sequence.
All McGill University third year medical students presenting for their 2-week Pediatric Emergency Department (PED) rotation at the Montreal Children’s Hospital were eligible. There were no specific exclusion criteria.
We designed two computer tutorials to be used by medical students in the PED. Both tutorials present mainly declarative knowledge (facts and concepts). The topics were: Oral Rehydration Solutions (ORS; objective: to describe the physiologic principles of osmosis and facilitated diffusion as they apply to oral rehydration of children – mainly conceptual knowledge) and Fever Without Source (FWS; objective: to enumerate the diagnostic and treatment approach to well-appearing febrile children aged birth-36 months – mainly factual knowledge). The design of these tutorials is reported in detail elsewhere and screen captures are supplied in Additional file 1 . They are brief (30 screens ORS and 42 screens FWS), potentially relevant to a patient just seen, and self-contained. The instructional strategies used included animations, schematic diagrams, interactive exercises and patient presentations but not video. We used the software authoring language Toolbook II Instructor Versions 6.5 – 8.5 (SumTotal Corporation, Mountain View, CA). The tutorials were independently reviewed by content experts and pilot tested to saturation on medical students and residents. They were installed on a single computer in the central area of the nursing station in the Pediatric Emergency Department (PED) and were not available to the students outside the study.
On the first day of their rotation, the students completed the Pre-test (see Outcome Measures below). They started the study procedure when they encountered a patient in the PED with one of the two index conditions for which there were computer tutorials: any patient with vomiting and/or diarrhea ( = ORS Patient ); or any patient under 36 months of age whose fever had no readily demonstrable cause (=Fever Without Source (FWS) Patient ). After discussing the patient’s case with the preceptor and discharging the patient, but before going to see the next patient, they logged into the study computer and were randomized to one of the two study conditions. On the 12th-14th day of the rotation, all participants completed the two post-test examinations. Each participant completed only one of the two tutorials. Subsequent patients with FWS or ORS were managed by the students according to regular practices.
Students randomized to the in sequence condition did a relevant computer tutorial just after having seen the index patient (See Figure 2). For example, if they had seen a patient with fever, they would do the “Fever Without Source” computer tutorial immediately after having completed their patient write-up and discussion with their preceptor. Students randomized to the out of sequence study condition did the opposite computer tutorial that was not relevant to the index patient. For example, if they had seen a patient with fever, they would do the “Oral Rehydration Solutions” tutorial. The timing of presentation of the tutorial in the out of sequence group was allowed to be at a convenient point before the end of their shift and not necessarily tied to when they saw the index patient since we did not know, a priori, which would be best. During the rotation, study participants were only exposed to one of the two tutorials. Which tutorial an individual subject saw was dependent on the diagnosis of the index patient and on the randomized allocation to study condition. (Figure 2).
Allocation and randomization
Allocation to study condition was concealed from the participants until they began the tutorial. Randomization was carried out according to the procedure outlined in Figure 2. Specifically, on the first screen of the tutorial the student indicated the type of patient seen (ORS or FWS). Based on the type of patient, the student was randomized to one of the two study conditions. Randomization was by varying blocks of 6, 8, or 10. Compliance with randomization was confirmed three ways: by comparing the tutorial topics to which an individual student had been randomized to whether they had reported doing the tutorial on the exit survey; by verifying that they had a log file showing completion of the tutorial; by having the student confirm via an on-screen question that they had done the tutorial under the assigned condition for the experiment.
We were able to track all interactions by the students with the computer, including the time and duration of the interaction and the titles of the screens accessed. Tutorial interactions where the student spent fewer than 4 minutes or viewed fewer than five screens were not considered to have completed the study tutorial. Date stamps on the logs allowed us to measure the number of weeks of clerkship experience.
Using the log files, we calculated the length of time that the subject spent interacting with the computer tutorial in minutes. In the PED, interruptions of some of the student sessions were inevitable. Pauses of greater than four minutes were not counted in the estimates of time spent by the students on the tutorials.
The two tests were developed by the investigators. The topics of the questions were the two domains of Pediatric Emergency Medicine relevant to the index patient diagnoses and covered by the tutorials: ORS (6 questions) and FWS (10 questions). The same examinations were used for pre- and post-testing. The questions were a mix of multiple-choice and constructed response formats testing application of declarative knowledge (see Additional file 2). We pilot tested all the questions on 10 medical students, 12 residents, and 3 faculty pediatricians and modified them in consequence. All participants did both examinations time-limited to 15 min. After the study was completed, the examinations were de-identified and marked by a single pediatric emergency physician blind to the pre- or post-test status of the participant and blind to study group allocation.
We calculated the reliability coefficient (Cronbach’s α) for each sub-examination (ORS, FWS) based on the individuals whose data was used in the analysis of the main outcome variable . We dropped 2 questions which would otherwise have impaired overall internal consistency.
In this experiment there were three independent variables with two levels each: tutorial exposure (yes, no), patient exposure (yes, no) and tutorial topic (FWS, ORS). In addition there was one continuous covariate, pre-test score, and one continuous dependent variable: post-test score. We converted the raw scores to z-scores to enable comparisons between the ORS and FWS topics . We compared knowledge gain for students using a three-way (2x2x2) Analysis of Covariance where pre-test score was the covariate. If the ‘in-sequence’ tutorial completion was superior to ‘out-of-sequence’ completion, then we anticipated seeing a statistically significant Patient Exposure by Tutorial interaction. We report the results in terms of means of standardized scores (percentages or z-scores), as well as using effect sizes (Cohen’s d) with 95 % confidence intervals based on the non-central t distribution as calculated using the ESCI software designed by Cummings and Finch .
Students who did not complete the protocol including post-test could not be analyzed on the main dependent variable. For non-completers, we tracked their pre-test scores and their rate of completion of the tutorials for comparison with the group that did complete the post-test.
We had pilot information from a previous trial using similar short-answer questions  and so calculated a sample size based on potential post-test score differences between groups. We chose a minimally important effect size of 0.4 and a sample size of 50 per group as being the best balance of resource use and protection against Type II error, assuming β=0.8 and α=0.05.
Characteristics of study population
The study flow diagram is shown in Figure 2. Between Aug 2008 and June 2009, 157 students did the Pre-Test of whom 96 (61%) saw a patient with one of the two index diseases and logged into the study computer to be randomized. Participants did not differ from non-participants in terms of Sex or Pre-test score.
The randomization resulted in 43 students being assigned to the ‘in sequence’ condition while 53 were in the ‘out of sequence’ condition. Four students did not complete the post-test so that we had 92 students complete the entire study maneuver with 51 in the ‘out of sequence’ condition and 41 ‘in sequence’. These two groups did not differ significantly in terms of Sex, Clerkship Experience or Pre-test Scores.
One question was dropped from each of the ORS & FWS tests due to poor point-biserial correlations. The reliability of the knowledge test as measured on subjects who completed the entire study maneuver (N = 92) was 0.84 (Cronbach’s alpha) for the ORS subscale (5 questions) and 0.63 for the FWS subscale (9 questions).
There were no differences in baseline measures for the randomization groups (in-sequence vs. out-of-sequence) in terms of sex (60% female vs. 57%), and pre-test scores (30% vs. 32%).
Students who were able to complete a tutorial scored higher on the post-test (54.0% (SD 13%) vs. 37.0% (12.5%) than did the 52 who did the post-test but not a tutorial (95% CI for difference +12.5%, +21.5%).
Across all study groups, the 144 participants who completed both tests had an average combined raw score that increased from pre-test 31% (SD = 11%) to post-test 48% (SD = 15.5%). This difference corresponds to a Cohen’s d of 1.1 indicating a large effect size and is statistically significant (p ≪ 0.001).
Effect of time since patient encounter
For the ‘out of sequence’ subjects, the timing of when they did the tutorial was variable with 26 of 51 (49%) doing the non-relevant tutorial immediately after seeing the index patient while the rest did the tutorial some convenient time later in their 8-hour shift. This timing difference did not affect post-test scores to an educationally or statistically significant degree (Post-test Raw Scores: ORS: Immediate 61% vs. Later in Shift 63%; FWS 56% vs. 55%). All subsequent analyses do not distinguish between ‘out of sequence’ subjects on the basis of when they completed the tutorial during the shift.
Knowledge gain by study condition
ANCOVA Table showing the interplay between Patient Exposure (Yes, No), Tutorial Completion (Yes, No) and Topic (Oral Rehydration Solution = 0, Fever Without Source = 1) all adjusted for Pre-Test Score
Partial Sum of Squares
Patient * Tutorial
Patient * Topic
Tutorial * Topic
Patient * Tutorial * Topic
Post-test z-scores by study group
Tutorial + Relevant Patient +
Tutorial + Relevant Patient –
Tutorial – Relevant Patient +
Tutorial – Relevant Patient –
F = 4.0 (1,87); p ≪ 0.05
F = 1.7 (1,87); p = NS
F = 4.8 (1,175); p = 0.03
None of the terms that included Topic (ORS vs. FWS) were statistically significant suggesting that knowledge gain was not moderated by which of the two topics is considered.
When the results are broken out by topic, the same pattern is observed as in the overall model, with increased test scores being associated with the in sequence condition. This effect reached statistical significance for the ORS topic where students who did the tutorial in the in sequence condition scored 19% higher (95% confidence interval for the difference +7%, +31%) corresponding to a large Cohen's-d effect size of 0.85 (95% CI 0.25, 1.5). Students who did the FWS tutorial in the in sequence condition scored an average of 6% HIGHER on the post-test than the comparison group (95% confidence interval for the difference (-3%, +16%) corresponding to a Cohen's-d effect size of 0.38 (95% CI -0.20, +0.97) which is not statistically significant.
Comparison of time on tutorial by study condition
As a planned subgroup analysis, we examined whether the time spent interacting with the tutorial varied according to whether it was done ‘in sequence’. We found estimating and interpreting the actual amount of time spent by the student from the log files difficult. In more than 25% of the cases, there appeared to be long periods between student clicks on the screens. We did not include these periods in our time estimates if they extended beyond four minutes. Our final estimate found that, when adjusted for which tutorial was being done, the ‘in sequence’ group spent 1.6 minutes more with a given tutorial than did the comparison group (95% Confidence Interval -1.2, + 4.5 minutes), which is not statistically significant.
We have proposed a framework for learning in clinical settings which extends current models. We found that presenting a computer tutorial immediately after a patient-preceptor encounter lead to measurably greater learning compared with doing the same tutorial separately from seeing a patient. This confirms the conventional wisdom of reading around cases but also suggests that clinical educators can purposefully design a Patient-Teacher-Tutorial sequence to take advantage of this maxim.
Current models of outpatient teaching and learning describe optimal teacher and learner behaviors that can increase the amount of learning during a given patient encounter. The two most widely studied models, the One-Minute Preceptor and the SNAPPS models, do not specifically describe what is to be done by the student after separation from the preceptor at the end of the patient interaction [2, 8, 16]. Harden et al. at the University of Dundee have done considerable work on the integration of computer-based learning into the clinical rotations of medical students . They enumerated over 130 tasks which could be done by students over the course of their clinical rotations. The tasks are supported with a series of customized study guides that highlight important objectives and specify available learning resources that the student may use in their learning about the task . However, the nature of the interaction between the study guides and the students’ patient experience is not reported in detail.
Our suggested Patient-Teacher-Tutorial (PTT) Sequence, composed of patient teacher and tutorial phases, integrates with and extends these existing models. The PTT model takes into account the constraints imposed by the workplace setting on the learning of conceptual knowledge. Given the time-limited and goal-oriented nature of their interaction, the clinician and student will tend to focus on content that gets the patient encounter done . While this is as it should be, Billett and others have pointed out that, for a student to function completely in the workplace, they need access to the “opaque or hidden” conceptual knowledge that is necessary for a flexible knowledge base . Consider the example of a patient with dehydration due to diarrhea. The student sees the patient and reports back to the preceptor, saying that the patient needs hydration with oral fluids. In the ensuing discussion, the clinician would focus on the choice of the fluid (an Oral Rehydration Solution) and how the patient should take it (e.g. one teaspoon at a time, spaced at 5 min intervals etc…). This directive information will ensure that the patient is correctly instructed. However, what is hidden are the conceptual underpinnings for this directive advice. Oral rehydration solutions are constructed according to a complex understanding of physiologic processes at the level of the intestinal absorptive cells and how they function when damaged by a virus. Understanding these physiology concepts will allow the future clinician to make finer grained judgments as to which fluids to use in which situations.
If we accept that students need to learn both the procedure-specific knowledge and the conceptual underpinnings to optimally function in the clinical setting, then the issue becomes one of ensuring that both types of learning occur. One option might be to simply expand the role of the preceptor to ensure that they include discussions of the conceptual as well as the procedural. However, preceptor time considerations often preclude this more symbolic, conceptual knowledge teaching [6, 7, 18] but, with support, the student can learn this material on their own from well-designed learning aids [12, 20–24].
An attractive aspect of the PTT sequence is that the teacher facilitates the student’s relationship with the declarative knowledge without necessarily being the provider of it. This continually reinforces the way of being of an expert practitioner . Expert practitioners require ready access to domain specific declarative knowledge . Ely estimates that practicing physicians have one clinically important knowledge question for every 3 patients they see [26, 27]. The diligent pursuit of these knowledge needs is a cornerstone of both evidence-based practice and of lifelong learning [28, 29]. Assiduously mandating that medical students do in-stream, focused reading after each patient could potentially establish a lifelong adaptive pattern.
The following limitations should be borne in mind while considering this study. It required the voluntary participation of busy medical students on a difficult rotation. A significant minority did not participate leaving the results susceptible to volunteer bias. We have presented a per protocol analysis. While we analyzed what we randomized, a sizeable minority of the students did not participate. However, this new unconventional technique might not suffer from the same non-completion rate once it is established that it can work and becomes part of the regular workflow, instead of being presented as a supplemental research protocol. We likely had sizeable variability between faculty preceptors in terms of their teaching on these topics. This variability could impact the intervention in unknown ways. Some of our comparisons were under-powered, especially the FWS subanalysis where the multidimensionality of the test would have decreased the power of the comparison.
RCTs in educational settings are controversial with some holding them up as the gold standard  while others deride them as hopelessly confounded . We chose this design in order to tightly contrast the study interventions based on a pre-specified conceptual model.
Our tutorials were invariant “black boxes” in these studies. As such, they used fixed instructional strategies that may not necessarily have been optimal for a given student situation. There are ways in which a computer tutorial can adapt to a student’s particular prior knowledge or personal educational goals . Such adaptations might have made the tutorials more effective and might have interacted with our study manipulations in important ways. We did not have a measure of knowledge retention beyond the rotation.
There may be unforeseen disadvantages to the intervention itself. We postulate that it is a third-best educational option behind learning from patients and learning from preceptors. If the intervention results in the learner seeing fewer patients or spending less time with their preceptors, this would be deleterious. By geographically stationing the computer tutorials right in the nursing station and by temporally presenting them right in the flow of patients, we believe that they can be done with a minimum of disruption to the learner’s interactions with their mentors and patients [12, 17, 33].
We have presented empiric data to support a framework for medical student outpatient learning that extends current models. A computer tutorial done in conjunction with seeing a relevant patient (i.e. the Patient-Teacher-Tutorial sequence) is more effective in raising test scores than the same tutorial done separate from a patient encounter. It could be educationally profitable to restructure outpatient learning frameworks to complement limited preceptor time with focused computer tutorials.
This work was presented as an abstract at the Association of American Medical Colleges Annual Meeting, November 2010, Washington DC.
The authors wish to thank the medical students at McGill for their cheerful participation in this study. We also wish to thank Judy Browning for her invaluable assistance in carrying out the study. We thank David McGillivray, David Kessler, Marc Auerbach, Adina Kalet, Martin Nachbar and Dorene Balmer for providing feedback.
- Irby DM, Bowen JL: Teaching strategies time-efficient strategies for learning and performance. The Clinical Teacher. 2004, 1: 23-10.1111/j.1743-498X.2004.00013.x.View ArticleGoogle Scholar
- Wolpaw TM, Wolpaw DR, Papp KK: SNAPPS: a learner-centered model for outpatient education. Article. Academic Medicine. 2003, 78: 893-898. 10.1097/00001888-200309000-00010.View ArticleGoogle Scholar
- Pratt DD, Arseneau R, Collins JB: Reconsidering “Good Teaching” across the continuum of medical education. J Contin Educ Health Prof. 2001, 21: 70-81. 10.1002/chp.1340210203.View ArticleGoogle Scholar
- Brown JS, Collins A, Duguid P: Situated cognition and the culture of learning. Educational Researcher. 1989, 18: 32-42.View ArticleGoogle Scholar
- Dornan T, Hadfield J, Brown M, Boshuizen H, Scherpbier A: How can medical students learn in a self-directed way in the clinical environment? design-based research. Medical Education. 2005, 39: 356-364. 10.1111/j.1365-2929.2005.02112.x.View ArticleGoogle Scholar
- Innes GD, Stenstrom R, Grafstein E, Christenson JM: Prospective time study derivation of emergency physician workload predictors. Can J Emerg Med. 2005, 7: 299-308.Google Scholar
- Usatine RP, Tremoulet PT, Irby DPD: Time-efficient preceptors in ambulatory care settings. Academic Medicine. 2000, 75: 639-642. 10.1097/00001888-200006000-00016.View ArticleGoogle Scholar
- Neher JO, Gordon KC, Meyer B, Stevens N: A five-step “Microskills” model of clinical teaching. J Am Board Fam Pract. 1992, 5: 419-424.Google Scholar
- Hattie J: Teachers make a difference: what is the research evidence. 2003, Melbourne: ACER Research Conference, 19-21.Google Scholar
- Zayas LE, James PA, Shipengrover JA, Schwartz DG, Osborne JW, Graham RP: Exploring instructional quality indicators in ambulatory medical settings: an ethnographic approach. Fam Med. 1999, 31: 635-640.Google Scholar
- Woods NN, Mylopoulos M, Brydges R: Informal self-regulated learning on a surgical rotation: uncovering student experiences in context. Adv Health Sci Educ Theory Pract. 2011, 16 (5): 643-653. 10.1007/s10459-011-9285-4.View ArticleGoogle Scholar
- Pusic M, Pachev G, MacDonald WA: Embedding medical student computer tutorials into a busy emergency department. Academic Emergency Medicine. 2007, 14 (2): 137-148. 10.1197/j.aem.2007.08.003.View ArticleGoogle Scholar
- Cohen J: Statistical power analysis for the behavioral sciences. 1988, Engelwood: Lawrence Erlbaum Assoc IncGoogle Scholar
- McClelland GH: Working With z-scores and Normal Probabilities. Seeing statistics. Edited by: McClelland GH. 1999, Pacific Grove: Duxbury PressGoogle Scholar
- Cumming G, Finch S: A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement. 2001, 61: 532-574.View ArticleGoogle Scholar
- Heidenreich C, Lye P, Simpson D, Lourich M: The search for effective and efficient ambulatory teaching methods through the literature. Pediatrics. 2000, 105: 231-237.Google Scholar
- Harden RM, Crosby J, Davis MH, Howie PW, Struthers AD: Task-based learning: the answer to integration and problem-based learning in the clinical years. Med Educ. 2000, 34: 391-397. 10.1046/j.1365-2923.2000.00698.x.View ArticleGoogle Scholar
- Kurth RJ, Irigoyen M, Schmidt HJ: A model to structure student learning in ambulatory care settings. Academic Medicine. 1997, 72: 601-606. 10.1097/00001888-199707000-00012.View ArticleGoogle Scholar
- Billett S: Learning in the workplace: Strategies for effective practice. 2001, Australia: Allen & Unwin; Crows NestGoogle Scholar
- Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM: Internet-based learning in the health professions: a meta-analysis. JAMA. 2008, 300 (10): 1181-1196. 10.1001/jama.300.10.1181.View ArticleGoogle Scholar
- Greenhalgh T: Computer assisted learning in undergraduate medical education. BMJ. 2001, 322: 40-44. 10.1136/bmj.322.7277.40.View ArticleGoogle Scholar
- Kerfoot BP, Baker H, Jackson TL, Hulbert WC, Federman DD, Oates RD, et al: A multi-institutional randomized controlled trial of adjuvant web-based teaching to medical students. Acad Med. 2006, 81: 224-230. 10.1097/00001888-200603000-00004.View ArticleGoogle Scholar
- Nardi BA: Studying Context: A Comparison Of Activity Theory, Situated Action Models, And Distributed Cognition. Chapter 4. Context and Consciousness: Activity Theory and Human-Computer Interaction editor Nardi BA. 1996, Cambridge Massachusetts: MIT Press, 69-102.Google Scholar
- Davis B, Sumara DJ: Cognition, complexity and teacher education. Harvard Educational Review. 1997, 67: 105-125.View ArticleGoogle Scholar
- Pratt DD: Five perspectives on teaching in adult and higher education. 1998, Melbourne: Krieger Publishing CoGoogle Scholar
- Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, et al: Analysis of questions asked by family doctors regarding patient care. British Medical Journal. 1999, 319 (7206): 358-361.View ArticleGoogle Scholar
- Coumou H, Zorgverzekeringen A, Meijman F: How do primary care physicians seek answers to clinical questions? a literature review. J Med Libr Assoc. 2006, 94 (1): 55-60.Google Scholar
- Hedberg CA: How the changing face of cme will affect internists. 2005, PA: American College of Physicians, Philadelphia, http://www.acpinternist.org/archives/2005/07/president.htm Accessed 15 May 2012Google Scholar
- American Academy of Family Physicians: Point of Care CME. 2012, Kansas: Practitioners, Leawood, http://www.aafp.org/online/en/home/cme/cmea/cmeapplying/pointofcare.html. 2007. Accessed 15 May 2012Google Scholar
- Slavin R: Evidence-based education policies: transforming educational practice and research. Educational Researcher. 2002, 31: 15-21.View ArticleGoogle Scholar
- Norman G: RCT = results confounded and trivial: the perils of grand educational experiments. Medical Education. 2003, 37: 582-584. 10.1046/j.1365-2923.2003.01586.x.View ArticleGoogle Scholar
- Ruiz JG, Mintzer MJ, Leipzig RM: The impact of e-learning in medical education. Acad Med. 2006, 81: 207-212. 10.1097/00001888-200603000-00002.View ArticleGoogle Scholar
- Bandiera G, Lee S, Tiberius R: Creating effective learning in today's emergency departments: how accomplished teachers get it done. Ann Emerg Med. 2005, 45: 253-261. 10.1016/j.annemergmed.2004.08.007.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/12/70/prepub