This article has Open Peer Review reports available.
Improving education in primary care: development of an online curriculum using the blended learning model
© Lewin et al; licensee BioMed Central Ltd. 2009
Received: 19 December 2008
Accepted: 10 June 2009
Published: 10 June 2009
Standardizing the experiences of medical students in a community preceptorship where clinical sites vary by geography and discipline can be challenging. Computer-assisted learning is prevalent in medical education and can help standardize experiences, but often is not used to its fullest advantage. A blended learning curriculum combining web-based modules with face-to-face learning can ensure students obtain core curricular principles.
This course was developed and used at The Case Western Reserve University School of Medicine and its associated preceptorship sites in the greater Cleveland area. Leaders of a two-year elective continuity experience at the Case Western Reserve School of Medicine used adult learning principles to develop four interactive online modules presenting basics of office practice, difficult patient interviews, common primary care diagnoses, and disease prevention. They can be viewed at http://casemed.case.edu/cpcp/curriculum. Students completed surveys rating the content and technical performance of each module and completed a Generalist OSCE exam at the end of the course.
Participating students rated all aspects of the course highly; particularly those related to charting and direct patient care. Additionally, they scored very well on the Generalist OSCE exam.
Students found the web-based modules to be valuable and to enhance their clinical learning. The blended learning model is a useful tool in designing web-based curriculum for enhancing the clinical curriculum of medical students.
Computer assisted instruction in undergraduate medical education has gained popularity in the last two decades. Web-based learning is attractive for many reasons, including the efficiency of providing content at diverse locations, the flexibility that students have in accessing and reviewing content, and the ability to provide online links to related information [1–3]. Computer-aided instruction is particularly attractive to directors of clinical experiences in complying with LCME standards and regulating the curricula of geographically dispersed students who are exposed to a variety of preceptor styles and patient populations .
Reports of widespread use of web-based curricula in the medical education literature speak to the convenience and feasibility of this method [5–12]. However, there is little description of the way in which didactic content has been reformatted to take advantage of the electronic learning environment [13–17]. Further, the use of computer-based materials to support clinical learning, a form of "blended learning," is new to medical education and has not been well studied [18–20].
The one study that did address the affect of internet-based learning did not specifically target blended learning, but did focus on the affect internet-based instruction had on the learning of health professional students when compared with no intervention and no internet based educational interventions. The authors performed a meta-analysis and concluded that internet-based instruction, when compared with no intervention, was associated with positive learning outcomes, particularly in learner satisfaction and knowledge acquisition, but in the knowledge arena blended learning courses were excluded from the analysis .
To standardize the core curricular concepts for all students.
To design online modules based on adult learning principles and present them in a developmentally appropriate manner, with increasing levels of complexity.
To take advantage of the flexible and interactive nature of the online environment to engage learners and augment and reinforce learning in the clinical setting.
The purpose of this paper is to describe the online curriculum as originally developed, as well as present student feedback and performance data.
The CPCP online curriculum can be viewed at http://casemed.case.edu/cpcp/curriculum. Learning modules can be found under "For Students." The description of the modules below refers to the content of the CPCP course when it was part of the Case Medical
School's Robert Wood Johnson Foundation funded elective Primary Care Track for students interested in exploring careers in primary care (1994–2007). Since then the CPCP curriculum has been amended to be used by the entire student body at Case, and has also been changed to allow educators at other institutions to use its learning activities. For this reason, the interactive parts of the website, described below, are no longer functional, as they required the ability to login to the Case computer system for students to post real-time comments and for course leaders to respond.
A panel of community primary care physicians who served as medical student preceptors, identified the topics included in the curriculum thus ensuring the online component would reflect real clinical scenarios. The modules revolve around a panel of "patients" that remains constant, each with a life story that addresses a portion of the curriculum. They include a two-year-old child, an adolescent girl, a young adult woman, a middle-aged adult man, and an elderly woman. Each "patient" has an online chart that is updated after each module and a variety of interactive methodologies make the patients and their clinical issues come to life.
Several adult-learning principles guided module development. First, adult learners are self-directed, so modules were constructed so that despite the fixed sequence of the modules themselves, students could complete required components of each module in any order. Also, each module includes links to more detailed information so that students can pursue deeper learning in areas of interest.
Further, adult learners are goal and relevancy-oriented, so the information in each module is presented in its most applicable form. Patient scenarios were created to represent common situations, and the practical aspects of responding to those situations are highlighted. Adult learners also bring a wealth of personal experiences to their learning, so the original CPCP modules used open-ended questions and discussion boards to allow students to address clinical issues from their own perspectives. The answers posted to the discussion board were reviewed by the course director and web instructional designer after the students completed each module and relevant feedback and responses were posted for the group to see. As noted above, in order to make this resource available to a larger academic community, the interactive nature of the discussion board has been removed from the current website.
Module One is called "You the Primary Care Physician," and is completed in the first half of the second year of medical school, when students have had limited exposure to clinical practice. The opening page shows a "virtual office" with staff members in place and a chart visible on a counter. Students can click on office staff to learn about their roles and responsibilities. They can go to the virtual chart and read a complete new patient note, see a properly completed problem list, and access tutorials on writing Subjective, Objective, Assessment and Plan (SOAP) notes and prescriptions. There is also a short health insurance tutorial followed by a quiz. Students were required to submit a SOAP note from their preceptorship sites for feedback from a course leader and two prescriptions written in the correct format. All of the activities are practical and designed to give students additional background and experience in the nuts and bolts of working in a medical setting.
Module Two, "The Art of Medicine," presents topics relating to challenging communication issues including maintaining adolescent confidentiality, addressing hidden agendas, conducting mental status exams in the elderly, confronting patients that miss appointments, and obtaining informed consent. Students enter a virtual waiting room and navigate by clicking on each patient. That patient's scenario is then presented along with audio, video, and text examples of possible responses. When the discussion board was in use, students answered open-ended questions after each scenario reflecting their real experiences in their clinical sites, and responses were posted on a discussion board where other students and faculty members could see them and respond.
Module Three is completed in the first half of the third year, when students are comfortable with the basics and can focus on clinical problem solving. It includes the same five patients presenting with common primary care complaints. Each activity includes a discussion of the differential diagnosis, work-up, and treatment of the highlighted condition as well as a related primary care topic such as over-prescription of antibiotics, complementary and alternative medicine, drug interactions, clinical practice guidelines, and use of online resources in real-time patient care. The content, again, is interactive, allowing students to become part of each scenario.
An Overview of the Community Primary Care Preceptorship Modules with Content and Timeline
1st Year of CPCP
2nd Year of CPCP
Second year of medical school
2nd Year of CPCP
Second year of medical school
Third year of medical school
The Art of
Primary care basics
Communicating with the patient
Evidence based medicine in the office setting
Preventable diseases: incidence/prevalence in diverse populations
Introduction to office-based practice – people and procedures
Difficult patient-communication issues and seeing the patient's point of view
Approaching differential diagnoses
Childhood injury, advanced directives, smoking prevention, metabolic syndrome, secondary complications of diabetes, clinical reminder systems
Charting, writing prescriptions, understanding health insurance
Understanding informed consent, performing a mini-mental status exam, maintaining patient confidentiality, addressing noncompliance, identifying hidden agendas
Developing a diagnostic and therapeutic plan for common complaints
Counseling patients for behavior change, performing a foot exam on a diabetic patient, working with other health-care professionals in preventing injury, correctly installing a child safety seat, understanding
Submit a SOAP note, practice prescription writing, and complete a practice profile of
Report on how preceptor handles difficult patient-care communication issues
Review patient cases with common illnesses and identify appropriate care based on clinical evidence
Create a patient education handout written at an appropriate reading level
The CPCP online curriculum was evaluated through feedback from students on the content and function of the modules and their applicability to students' clinical work. Feedback surveys addressed four areas: 1) general information on the time required to complete each module, relevance of the learning objectives, and appropriateness of the online format; 2) applicability of the modules' content to the students' clinical work; 3) module technical performance; and 4) the modules' most and least valuable parts. The first three sections contained Likert scale responses; the questions about the most and least valuable parts of each module were answered in free text. Student responses to the first three sections were tabulated and the percentage indicating each possible response calculated. Comments from section four were reviewed for recurrent themes.
At the end of the third year of medical school, all CPCP students and a convenience sample of students in the traditional curriculum who had no exposure to the CPCP online curriculum or preceptorship completed a 15 station Objective Structured Clinical Examination whose content was based on the CPCP curricular objectives. Stations assessed students' history-taking, physical exam, medical knowledge and patient education skills.
Scores on the exam were collected and analyzed using the SPSS statistical package, v. 14.0. Mean scores on the USMLE Part 1 exam of PCT and control students were compared using ANOVA to look for baseline academic differences between the groups. Levene's test for Equality of Variances was performed on OSCE scores to determine whether the variances between the two groups differed significantly. T-tests were then performed to compare the overall performance of CPCP students and controls, as well as the sub-scores on the history taking, physical exam and patient education stations.
This study was granted "exempt" status by the Institutional Review Board of the MetroHealth Medical Center, a Case affiliate.
CPCP students from the classes of 2005 (16), 2006 (36), and 2007 (37) completed 248 feedback surveys between January, 2002 and July, 2005. All participating students completed all surveys as a requirement of the program. Analysis was performed on 89 surveys for Module 1, 80 for Module 2, 44 for Module 3, and 35 for Module 4. Variation in the numbers is attributable to changes in some students' status as they joined or left the CPCP program.
Survey results showed that each module, in its entirety, took most students approximately 7–9 hours to complete. Greater than 99% of students stated that the learning objectives of each module were clear and 98% agreed that they were met. Students indicated that they would preferentially choose a web-based format to learn the material in each module (>75% for Modules 1–3; 63% for Module 4) with the majority of remaining students preferring small groups.
The overall content of each module was rated as "excellent" or "very good" by the majority of students. The most highly rated components of each module were those that addressed concrete clinical skills such as SOAP note writing, prescription writing, and the health insurance tutorial (Module 1); adolescent confidentiality and hidden agendas (Module 2); treating childhood asthma and the differential diagnosis of URI symptoms (Module 3); and obesity and smoking prevention (Module 4).
Students that Agree or Strongly Agree "I Could Directly Relate what I Learned in the Following Module Components to Real Patient Interactions in my Preceptor's Office"
% Strongly Agree or Agree
Two: The Art of Medicine (n = 50)
Patients who miss appointments*
Obtaining Informed Consent
Finding the Hidden Agenda
The Confidential Adolescent Interview
The Mental Status Exam in the Elderly
Three: The Practice of Medicine (n = 44)
URI/Antibiotic Over prescribing
Treatment of Hypertension in a Diabetic Patient
Use of Asthma Guidelines
Working up Knee Pain/CAM
Weight Loss in the Elderly/Using Online Resources
Four: Preventive Medicine (n = 14)
Secondary Complications of Diabetes
Falls in the Elderly
Patient Education Exercise
Childhood Accidents 50
The technical performance of all components of all four modules was rated as excellent or very good by >70% of students except for the streaming video in Module 3 (54%) and Module 4 (46%) and the streaming audio in Module 4 (45%). More than 75% of students found the modules easy to navigate and more than 80% found the amount of explanation adequate to perform the required tasks.
A total of 41 CPCP students took the Generalist OSCE exam at the end of their second year of the curriculum corresponding to the end of the third year of medical school. Nine volunteer students comprised the control group.
There were no statistical differences between CPCP students (mean = 224, standard dev = 20.4) and controls (mean = 236, standard dev = 15.1) on the USMLE Part 1 exam.
Levene's test for Equality of Variances indicated that variances between the two groups on the gOSCE do not differ significantly (p = .86). Independent samples t-tests on overall exam scores revealed that CPCP students (M = 67.83, SD = 6.4) scored higher than the control group (M = 63.0, SD = 5.85). At an alpha level of .05, this difference was found to be statistically significant, t (48) = 2.08, p = .043 (two-tailed).
Analyses of the subsections (history taking, physical exam, medical knowledge, and patient education) show a trend toward better performance in CPCP students than in controls, especially in the patient education section (p = 0.097), but these differences were not statistically significant.
The use of blended learning is not new, but in the past it consisted primarily of didactic sessions or small groups to bring students in disparate clinical environments together. Using a computer-based model incorporating adult learning principles allows more student flexibility and can present material that simulates actual clinical encounters. Students can work through the material in the way that best suits their learning styles.
Creating online materials allows for a level of creativity absent from writing syllabi or paper cases. We found that the variety of learning activities we could construct was only limited by our imaginations. Striving for the maximum interactivity allowed us to actively engage the students, and their feedback shows that they found the experience to be educationally valuable.
Based on student feedback surveys, we found that the online curriculum was well accepted, despite the extra time students spent to complete it. This likely relates to the students' ability to work on new skills and directly apply them to their work with patients. Having an interactive virtual chart allowed students to practice note and prescription writing on their own, away from the time pressures of the clinical setting. Likewise, seeing approaches to challenging patient interviews online gave students the opportunity to increase their comfort level with such patients and to reflect on their own skills outside of real clinical encounters.
More advanced students could use the curriculum to explore common primary care complaints and their differential diagnoses, tests, and treatments. Some students, however, commented that the information in Module 3 was presented at too basic a level. Reviewing the content with an eye toward its applicability to a wider range of diagnostic abilities might have improved the experience for clinically advanced students.
Although all Case medical students are issued identical laptop computers on matriculation, there were still technologic issues. Some students with slow internet connections at home had trouble viewing the streaming video and listening to the streaming audio. With the increasing availability and affordability of high-speed internet connections, this problem should become less pressing. For medical schools that don't provide or require specific computer specifications, however, care must be taken in creating content that can be used by all students.
There are strengths and limitations to all educational formats, and blended learning is no exception. We have highlighted the ability of an online curriculum to enhance clinical learning by taking advantage of interactive technology and standardizing student experiences in a way that is not place or time dependent. One obvious weakness is the distant nature of the interactions between students and teachers when communicating via discussion board or email. This is balanced, in CPCP, with the face-to-face interactions that the students have weekly with their preceptors. There is also a limited amount of flexibility in the content delivered to individual students. We addressed this by including as many links to further reading and related information as possible so each student could find content that suited his/her level of expertise.
The major limitation to this study is the difficulty in measuring behavioral outcomes. Due to the small number of control students as well as the large number of other clinical experiences during the two years of CPCP participation that could have influenced student performance on the clinical exam, sweeping conclusions are not possible. We were encouraged to see that PCT students performed well on the exam when compared with their peers and that there was a trend toward superior performance in patient counseling and a significant difference in overall performance. The strength of the blended learning model may not be reflected as clearly in performance data as it is in the feedback the students provide on their ability to use the knowledge gained to improve the actual care of patients.
In addition, when Case underwent a major curricular revision in 2006, CPCP and the online curriculum were expanded to include all second year medical students. This expansion of CPCP from an elective for primary care track students to a mandatory course further exemplifies the success of this program.
In summary, a blended learning course using an interactive online curriculum to augment clinical learning in a longitudinal primary care preceptorship was successful in engaging second and third year medical students. Participants rated the content and the format highly and found that they could use the information gained in their real clinical lives. We attribute this success to the creative use of computer technology and the practical nature of the material. Medical educators should consider the blended learning format as they strive to standardize the clinical learning of their students.
We thank Dr. Klara Papp for her statistical support.
The Community Primary Care Preceptorship program was developed with support from a Predoctoral Training in Primary Care grant from the Health Resources and Services Administration (CFDA No. 93.896A).
- Baldwin CD, Gephart D, Maulitz R: Collaborative planning of a Web-based learning resource for primary care education. Acad Med. 2001, 76: 549-50.View ArticleGoogle Scholar
- Fieschi M, Soula G, Giorgi R, Gouvernet J, Fieschi D, Botti G, Volot F, Berland Y: Experimenting with new paradigms for medical education and the emergence of a distance learning degree using the internet: teaching evidence-based medicine. Medical Inform Internet Med. 2002, 27: 1-11. 10.1080/14639230110105301.View ArticleGoogle Scholar
- Harden RM, Hart IR: An international virtual medical school (IVIMEDS): the future for medical education?. Med Teach. 2002, 24 (3): 261-7. 10.1080/01421590220141008.View ArticleGoogle Scholar
- Liaison Committee on Medical Education: Functions and structure of a medical school: standards of accreditation for medical school programs leading to the M.D. degree. [http://www.lcme.org/standard.htm]
- White CB, Albritton TA, Rindt K: MEOW: A web site to improve consistency and communication in clerkships. Academic Med. 2001, 76: 542-10.1097/00001888-200105000-00083.View ArticleGoogle Scholar
- Kaelber DC, Bierer SB, Carter JR: A web-based clinical curriculum on the cardiac exam. Academic Med. 2001, 76: 548-9. 10.1097/00001888-200105000-00091.View ArticleGoogle Scholar
- Kamin C, Deterding R, Lowry M: Student's perceptions of a virtual PBL experience. Acad Med. 2002, 77: 1161-2. 10.1097/00001888-200211000-00028.View ArticleGoogle Scholar
- D'Alessandro DM, Lewis TE, D'Alessandro MP: A pediatric digital storytelling system for third year medical students: the virtual pediatric patients. BMC Med Educ. 2004, 19: 10-10.1186/1472-6920-4-10.View ArticleGoogle Scholar
- Smucny J, Epling JW: A web-based approach to teaching students about diagnostic reasoning. Fam Med. 2004, 36: 622-4.Google Scholar
- Gjerde CL, Pipas CF, Russell M: Teaching of medical informatics in UME-21 medical schools: best practices and useful resources. Fam Med. 2004, 36 (Suppl): S68-73.Google Scholar
- Zebrack JR, Mitchell JL, Davids SL, Simpson DE: Web-based curriculum. A practical and effective strategy for teaching women's health. J Gen Intern Med. 2005, 20: 68-74. 10.1111/j.1525-1497.2005.40062.x.View ArticleGoogle Scholar
- Bernardo V, Ramos MP, Plapler H, De Figueiredo LF, Nader HB, Ancao MS, Von Dietrich CP, Sigulem D: Web-based learning in undergraduate medical education: development and assessment of an online course on experimental surgery. Int J Med Inform. 2004, 73: 1-42. 10.1016/j.ijmedinf.2004.06.002.View ArticleGoogle Scholar
- McKimm J, Jollie C, Cantillon P: ABC of learning and teaching web based learning. BMJ. 2003, 326: 870-3. 10.1136/bmj.326.7394.870.View ArticleGoogle Scholar
- Berry E, Parker-Jones C, Jones RG, Harkin PJ, Horsfall HO, Nicholls JA, Cook NJ: Systematic assessment of World Wide Web materials for medical education: online, cooperative peer review. Journal of American Med Inform Assoc. 1998, 5: 382-9.View ArticleGoogle Scholar
- Cook DA: The research we still are not doing: an agenda for the study of computer-based learning. Academic Med. 2005, 80 (54): 1-8.Google Scholar
- Adler MD, Johnson KB: Quantifying the literature of computer-aided instruction in medical education. Acad Med. 2000, 75: 1025-8. 10.1097/00001888-200010000-00021.View ArticleGoogle Scholar
- Chumley-Jones HS, Dobbie A, Alford CL: Web-based learning: sound educational method or hype? A review of the evaluation literature. Acad Med. 2002, 77: S86-93. 10.1097/00001888-200210001-00028.View ArticleGoogle Scholar
- Osguthorpe RT, Graham CR: Blended learning environments: definitions and directions. Quarterly Review of Distance Education. 2003, 4 (3): 227.Google Scholar
- Barbian J: Blended works: here's proof. Training. 2002, Accessed 9/15/2004, [http://www.trainingmag.com]Google Scholar
- Shaffer K, Small JE, Joseph E, Nancy O: Whitley Award winner. Blended learning in medical education: use of an integrated approach with web-based small group modules and didactic instruction for teaching radiologic anatomy. Acad Radiol. 2004, 11: 1059-70. 10.1016/j.acra.2004.05.018.View ArticleGoogle Scholar
- Cook DA, Levinson AJ, Garside S: Internet-based learning in the health professions: a meta-analysis. JAMA. 2008, 300 (10): 1181-1196. 10.1001/jama.300.10.1181.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/9/33/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.