This article has Open Peer Review reports available.
Teaching differential diagnosis in primary care using an inverted classroom approach: student satisfaction and gain in skills and knowledge
© Bösner et al.; licensee BioMed Central. 2015
Received: 30 April 2014
Accepted: 16 March 2015
Published: 1 April 2015
Differential diagnosis is a crucial skill for primary care physicians. General practice plays an increasing important role in undergraduate medical education. Via general practice, students may be presented with an overview of the whole spectrum of differential diagnosis in regard to common symptoms encountered in primary care. This project evaluated the impact of a blended learning program (using the inverted classroom approach) on student satisfaction and development of skills and knowledge.
An elective seminar in differential diagnosis in primary care, which utilized an inverted classroom design, was offered to students. Evaluation followed a mixed methods design: participants completed a pre- and post-test, a questionnaire, and a focus group discussion. Interviews were transcribed verbatim and answers were grouped according to different themes. Test results were analysed using the Wilcoxon matched-pairs signed-ranks test.
Participants (n = 17) rated the course concept very positively. Especially the inverted classroom approach was appreciated by all students, as it allowed for more time during the seminar to concentrate on interactive and practice based learning. Students (n = 16) showed a post-test significant overall gain in skills and knowledge of 33%.
This study showed a positive effect of the inverted classroom approach on students’ satisfaction and skills and knowledge. Further research is necessary in order to explore the potentials of this approach, especially the impact on development of clinical skills.
One of the essential skills of the clinician is the ability to make an accurate diagnosis. Especially in the field of primary care the process of making a differential diagnosis is a challenging and sometimes daunting task. Diseases often present at an early stage and sometimes in an atypical form, therefore primary care physicians use a broad range of diagnostic strategies . Medical students are usually taught differential diagnosis in the high prevalence setting of a university hospital. In addition, differential diagnosis of disease symptoms with a broad underlying aetiology (e.g. dizziness) will be split among different disciplines (such as neurology, ENT and internal medicine). However, many students will later work as physicians in the low prevalence context of primary care, where they will have to deal with the diagnostic uncertainty that is connected with seeing patients with a broad range of symptoms . General practice/family medicine, which plays an increasingly important role in undergraduate medical education throughout Europe , is in an excellent position to give students an overview of the whole spectrum of differential diagnosis in regard to common symptoms encountered in primary care.
Nowadays, e-learning is an integral part of medical education, and has been shown to be most effective when combined with face-to-face teaching, facilitating the so-called blended learning approach [4-6]. Students increasingly use mobile devices like smartphones for learning purposes, and appreciate being able to utilise multimedia materials like video clips  or podcasts  for the preparation or revision of course content. Undergraduate medical students highly value the use of e-learning, especially when integrated into a blended approach . While blended learning approaches are well established in the postgraduate education of primary care providers [10,11], there are fewer examples in the field of undergraduate medical education in primary care .
The ‘inverted’ or ‘flipped classroom’ approach takes blended learning one step further, as it offers all traditional lectures exclusively via e-learning, and uses the face-to-face sessions for interactive exercises and critical reflection . This concept is already widely used in other disciplines, for example in the social sciences. Different theoretical foundations to justify the inverted classroom approach derive from the literature on student-centered learning including Piaget’s theory of cognitive conflict and Kolb’s theory of experiential learning . It has been shown that this approach positively influences students’ openness for cooperative learning and innovative teaching methods . While students still preferred live lectures to video lectures, they valued the interactive class time higher than in-person lectures .
Inverted classroom education has just started to be incorporated into medical education , and there are so far only few published examples [17-20], however none of these in the field of primary care. In contrast, we consider especially the field of primary care learning as highly suitable for an inverted classroom approach. Primary care includes a broad range of complaints and diseases that can be well taught via e-learning. Class time can then be effectively used to teach the complex process of medical decision making in patients with a broad range of symptoms, the typical setting of primary care, and to explore ways how to deal with the connected uncertainty.
Will students appreciate this approach for studying differential diagnosis in primary care?
What will the gain in skills and knowledge of students learning with the inverted classroom approach be?
Participants and setting
All participants of this elective seminar were in their fourth or fifth year of undergraduate medical studies. The different modules consist of interdisciplinary clinical pictures derived from the everyday routine of primary care. Contrary to other clinical specialties, where students are also presented with very rare diseases which they may rarely -if ever- encounter during their medical career, we primarily concentrated on common underlying aetiologies for lead symptoms such as chest pain, dyspnoea, abdominal pain, or vertigo/dizziness. A special focus was put on the diagnostic accuracy of symptoms and signs in regard to the different underlying disease aetiologies of a given clinical picture. The seminar comprises of 42 hours of face-to-face teaching altogether, divided over 3 hours per week respectively, and closes with an objective structured clinical examination (OSCE). The whole seminar takes place at an interactive skills lab attached to the Marburg University Hospital and utilizes trained simulation patients and different models. Further details of seminar content and underlying didactic considerations have been published elsewhere .
One year ago, we redesigned the whole seminar, using the inverted classroom approach [22,23]. At the faculty of medicine, our seminar is currently the only taught medical course using the inverted classroom design. To our knowledge, this method is so far also not used by other primary care departments in Germany and in general not yet widely established in medical education at German Universities.
Preparation: Several video and audio lectures giving introductory information and key knowledge content are available on the web-based learning platform.
Face-to-face teaching, which takes place in the interactive skills lab: several didactic approaches, such as simulation patients, training models, interactive small group work, and quiz exercises are used.
Follow-up: Additional video and audio lectures present more detailed information in regard to the single leading symptom and its underlying etiologies. Supplementary facultative reading material (e.g. original research on the accuracy of symptoms and signs for a given disease) is also offered via the web-based learning platform.
We used a mixed methods design in order to investigate our study questions.
Student satisfaction (research question 1) was measured using information from different sources. The first source was a standardized questionnaire that is used by the University of Marburg to evaluate seminars. The questionnaire utilizes a rating scale ranging from 1 point (“agree not at all”) to 5 points (“completely agree”) and additional free text answers, and covers four main areas (seminar concept and presentation, interaction with students, level of interest/relevance and difficulty/quantity/speed). It is filled out anonymously and analysed by the university’s evaluation department.
In addition, after the 8th out of 14 course sessions, an evaluation consisting of a focus group discussion and a short questionnaire that was handed out after the focus group were conducted. Topics of the interview and the questionnaire included the perception of the seminar content, a critical reflection of the inverted classroom approach (usefulness of video and audio sessions, connection/synthesis with the face to face sessions) and the individual learning experience of the students. We concentrated on the learning experience of the course participants, with a special focus on how the inverted classroom approach is perceived. For the quantitative evaluation of gain in skills and knowledge (research question 2) we designed a questionnaire consisting of extended matching items (13 items), and key-feature tests (20 items). Both examination formats can measure the process of clinical reasoning, and help to assess clinical decision making skills [24,25]. They are used to test the second level of clinical competence in “Miller’s Pyramid” (“Knows how”) . We found the extended matching questions suitable for testing which clinical findings from the patient’s history and physical examination had the highest diagnostic accuracy.
Together, both question formats covered all major seminar content. Questions were pre-tested on another group of students with comparable pre-existing knowledge. After the pre-test we replaced two key feature cases that were too easy and one extended matching question that was too difficult and in addition modified one key feature question that was misleading. During the seminar, students were asked to complete the pre-test at the beginning of the first session, and the post-test after the last session before starting to learn for the OSCE that was conducted one week later. Tests were completed by students using an unique identifier that allowed the anonymous matching of pre- and post-tests at an individual student level.
Additional file 1 shows example questions of the questionnaire.
For the 4 variables of the standardized questionnaire for student satisfaction means and standard deviations were calculated and plotted.
Results (average in percentage of maximal test score that could be gained) of the pre- and post-test were compared for statistically significant differences using the Wilcoxon matched-pairs signed-ranks test. Error probability with a p-value less than 0.05 was considered significant. Statistical analysis was performed using the GraphPad PRISM (Version 6 Graphpad Software, Inc).
The focus group discussion was taped and transcribed verbatim. For data analysis we utilised a deductive approach based on the questions of the focus group guideline and of the short written questionnaire. Analysis was performed by SB and results discussed among all authors. Responses from the focus groups and the free text answers from the evaluation questionnaires were grouped under different themes.
The entire here presented data are part of the routine evaluation of courses at our faculty. Ethical approval was therefore not required.
Altogether, 17 students applied and were all enrolled in the seminar. One student did not participate in the pre-test. All 17 students took part in the focus group and the final evaluation.
Answers from the focus group discussion and the evaluation questionnaires can be grouped under the following themes:
“Usually you learn from a disease-oriented point of view…but the patient usually presents with a symptom. This is what usually lacks (in other seminars or lectures)… it is difficult to remember all the differential diagnoses, because you have to screen all diseases that you know in order to see whether these contain the symptom… here (in this seminar) it is the other way around.”.
“Epidemiology was a topic which I considered as totally uninteresting, but now its importance has become clearer to me.”
“There is a much more frequent reference to epidemiology (in comparison to other seminars/lectures), which I consider good, as it helps to build a much stronger hierarchy of differential diagnoses.”
Inverted classroom approach
“The things (seminar content) that are in the videos can be outsourced very easily and there is no necessity to cover them here (during the face to face lessons).”
“The summaries of different diseases in the preparatory lessons are very good. The immediate application with the help of simulation patients is very practical.”
“One can be even more radical; we have already seen the video and you then don’t need the slides anymore.”
Acquisition of further key competencies
“It is very good that we have the opportunity to reason together as a team…to brainstorm together in order to consolidate what we have heard.”
“It is also good to get ideas from others. I think that this is also important for being a doctor later, to learn this capacity for teamwork.”
“I learn best when I need to apply my previous knowledge to unknown new situations that I consider difficult. Then I am most creative and it (the knowledge) remains in the long-term memory.”
Gain in skills and knowledge
We aimed to give a first account of the effects of a blended learning program in primary care with this study. To our best knowledge the above presented seminar is the first to teach differential diagnosis in primary care for undergraduate students using an inverted classroom approach. Our evaluation showed both a high satisfaction rate and a significant gain in skills and knowledge.
When planning to introduce a new e-learning program it is crucial to evaluate participants’ reception of the chosen teaching approach [5,27]. The high approval of our blended learning approach is consistent with the available literature. While most studies show a positive effect of blended learning on students’ satisfaction [5,8,28], not all students necessarily appreciate and use multimedia materials such as video clips .
A recently conducted Delphi study on the development of a technology-mediated teaching strategy, the authors called for teaching activities that are “learner-centred, interactive, integrated, reflective and that promote engagement” ; the inverted classroom approach was mentioned among others as one possible strategy to achieve this goal. Our finding of high student satisfaction with this approach especially, is supported by the – presently sparse- literature in the field of medical education. While we could not identify evaluations of other primary care programs that use the inverted classroom model, there have been programs in teaching palliative care skills , cardiovascular, respiratory and renal physiology , renal pharmacotherapy  and perfusion skills . All of the above mentioned programs used the inverted classroom approach and were positively evaluated, which supports our own findings.
An additional finding in our focus group discussion was that the new learning experience during the seminar also triggered part of the participants to critically reflect on their own learning. Biggs refers to this process as ‘metacognition’ which includes the awareness of the own cognitive processes together with exerting control over them. He encourages a deep approach to learning which includes the promotion of guided self-questioning, using other students as a resource or to derive own heuristics to suit a given task . All these mentioned elements can be found in our course and it is interesting to note that an innovative and unusual teaching approach can trigger this kind of critical self-reflection, which we consider an important prerequisite for later professional life.
While a broad acceptance of participating students is surely a precondition for establishing a successful blended learning program , there should also be a positive impact on clinical skills. In one study students rated e-learning just as highly as alternative traditional methods of clinical skills teaching . However, the effectiveness of blended learning is often difficult to quantify [5,6]. We observed a gain in skills and knowledge of the participating students in the course of our seminar, but cannot judge what additional effect the blended learning/inverted classroom approach had in comparison to usual face-to-face teaching. In addition, the results of a systematic review on the role of blended learning in the clinical education of health-care students, which identified only a few high quality studies on this topic, provided only rudimentary evidence that technology-enhanced teaching improves clinical competencies .
There are strengths, but also several limitations to our study. The chosen mixed methods study design is an appropriate way to evaluate a blended learning program, taking qualitative and quantitative information into account . We not only evaluated student satisfaction, but also tried to quantify gain in skills and knowledge at both a group and at an individual level. One limitation is the relatively small number of participants included in the study. As the seminar is highly interactive, consisting mainly of work in small groups with tutors and simulation patients, no more than 18 students can be enrolled. This is also one of the reasons why we could not compare the inverted classroom approach with a second group taught using traditional methods. This would have been desirable from a methodological point of view; however it was not feasible to divide the small group of participants any further. As we do not have long term follow up results (e.g. repetition of the post-test after 6 months), we cannot say how much knowledge was retained by the students. Both examination formats that we used can only test the second level of clinical competence according to “Miller’s Pyramid”. It would have been desirable to perform an OSCE as pre- and post-test. However, from a practical point of view we did not consider it appropriate to confront students with the complex challenges of an OSCE at the start of the seminar. In this instance we had to balance methodological considerations versus the danger of frustrating the participating students right at the beginning of this elective course with an OSCE they would not at all be prepared for.
This study shows that an inverted classroom model is well suited to teach the complex topic of differential diagnosis in primary care. While we perceived high student satisfaction, we could not prove whether the inverted classroom approach leads to a higher gain in skills and knowledge than traditional face-to-face teaching. Future research on our presented concept should therefore aim for a direct comparison of these two teaching approaches on gain in skills and knowledge, ideally in the form of a randomized controlled trial with an OSCE as test format.
The authors thank all students that participated in the seminar.
- Heneghan C, Glasziou P, Thompson M, Rose P, Balla J, Lasserson D, et al. Diagnostic strategies used in primary care. BMJ. 2009;338:b946.View ArticleGoogle Scholar
- Wübken M, Oswald J, Schneider A. Umgang mit diagnostischer Unsicherheit in der Hausarztpraxis. Z Evid Fortbild Qual Gesundhwes. 2013;107:632–7.View ArticleGoogle Scholar
- Brekke M, Carelli F, Zarbailov N, Javashvili G, Wilm S, Timonen M, et al. Undergraduate medical education in general practice/family medicine throughout Europe – a descriptive study. BMC Med Educ. 2013;13:157.View ArticleGoogle Scholar
- Hull P, Chaudry A, Prasthofer A, Pattison G. Optimal sequencing of bedside teaching and computer-based learning: a randomised trial. Med Educ. 2009;43:108–12.View ArticleGoogle Scholar
- Woltering V, Herrler A, Spitzer K, Spreckelsen C. Blended learning positively affects students’ satisfaction and the role of the tutor in the problem-based learning process: results of a mixed-method evaluation. Adv in Health Sci Educ. 2009;14:725–38.View ArticleGoogle Scholar
- Back DA, Haberstroh N, Antolic A, Sostmann K, Schmidmaier G, Hoff E. Blended learning approach improves teaching in a problem-based learning environment in orthopedics - a pilot study. BMC Med Educ. 2014;14:17.View ArticleGoogle Scholar
- Romanov K, Nevgi A. Do medical students watch video clips in eLearning and do these facilitate learning? Med Teach. 2007;29:484–8.View ArticleGoogle Scholar
- Shantikumar S. From lecture theatre to portable media: students’ perceptions of an enhanced podcast for revision. Med Teach. 2009;31:535–8.View ArticleGoogle Scholar
- Gormley GJ, Collins K, Boohan M, Bickle IC, Stevenson M. Is there a place for e-learning in clinical skills? A survey of undergraduate medical students’ experiences and attitudes. Med Teach. 2009;31:e6.View ArticleGoogle Scholar
- Bekkers M, Simpson SA, Dunstan F, Hood K, Hare M, Evans J, et al. Enhancing the quality of antibiotic prescribing in Primary Care: qualitative evaluation of a blended learning intervention. BMC Fam Pract. 2010;11:34.View ArticleGoogle Scholar
- Vollmar HC, Butzlaff ME, Lefering R, Rieger MA. Knowledge translation on dementia: a cluster randomized trial to compare a blended learning approach with a "classical" advanced training in GP quality circles. BMC Health Serv Res. 2007;7:92.View ArticleGoogle Scholar
- Lewin LO, Singh MK, Bateman BL, Bligh Glover P. Improving education in primary care: development of an online curriculum using the blended learning model. BMC Med Educ. 2009;9:33.View ArticleGoogle Scholar
- Lage MJ, Platt GJ, Treglia M. Inverting the classroom: a gateway to creating an inclusive learning environment. J Econ Educ. 2000;31:30–43.View ArticleGoogle Scholar
- Bishop JL, Verleger MA. The Flipped Classroom: A survey of the research. ASEE National Conference Proceedings: 25–26 June 2013, Atlanta.Google Scholar
- Strayer JF. How learning in an inverted classroom influences cooperation, innovation and task orientation. Learn Environ Res. 2012;15:171–93.View ArticleGoogle Scholar
- Prober CG, Khan S. Medical education reimagined: a call to action. AcadMed. 2013;88:1407–10.Google Scholar
- Pierce R, Fox J. Vodcasts and active-learning exercises in a “flipped classroom” model of a renal pharmacotherapy module. Am J Pharm Educ. 2012;76:196.View ArticleGoogle Scholar
- Riley JB, Austin JW, Holt DW, Searles BE, Darling EM. Internet-based virtual classroom and educational management software enhance students' didactic and clinical experiences in perfusion education programs. J Extra Corpor Technol. 2004;36:235–9.Google Scholar
- Periyakoil VS, Basaviah P. The flipped classroom paradigm for teaching palliative care skills. Virtual Mentor. 2013;15:1034–7.View ArticleGoogle Scholar
- Tune JD, Sturek M, Basile DP. Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Adv Physiol Educ. 2013;37:316–20.View ArticleGoogle Scholar
- Bösner S CSMSSTSABE: Differentialdiagnose in der Primärversorung: Konzeption und Durchführung eines neuen Wahlpflichtfaches – Ein Erfahrungsbericht. [http://www.egms.de/static/pdf/journals/zma/2011-28/zma000752.pdf].
- McLaughlin JE, Roth MT, Glatt DM, Gharkholonarehe N, Davidson CA, Griffin LM, et al. The flipped classroom. Acad Med. 2014;89:236–43.View ArticleGoogle Scholar
- Mehta NB, Hull AL, Young JB, Stoller JK. Just imagine. Acad Med. 2013;88:1418–23.View ArticleGoogle Scholar
- Beullens J, Struyf E, van Damme B. Do extended matching multiple-choice questions measure clinical reasoning? Med Educ. 2005;39:410–7.View ArticleGoogle Scholar
- Farmer EA, Page G. A practical guide to assessing clinical decision-making skills using the key features approach. Med Educ. 2005;39:1188–94.View ArticleGoogle Scholar
- Norcini JJ. ABC of learning and teaching in medicine: work based assessment. BMJ. 2003;326:753–5.View ArticleGoogle Scholar
- Gesundheit N, Brutlag P, Youngblood P, Gunning WT, Zary N, Fors U. The use of virtual patients to assess the clinical skills and reasoning of medical students: initial insights on student acceptance. Med Teach. 2009;31:739–42.View ArticleGoogle Scholar
- Perkins GD, Fullerton JN, Davis-Gomez N, Davies RP, Baldock C, Stevens H, et al. The effect of pre-course e-learning prior to advanced life support training: a randomised controlled trial. Resuscitation. 2010;81:877–81.View ArticleGoogle Scholar
- Rowe M, Frantz J, Bozalek V. Beyond knowledge and skills: the use of a Delphi study to develop a technology-mediated teaching strategy. BMC Med Educ. 2013;13:51.View ArticleGoogle Scholar
- BIGGS JB. The role of metalearning in study processes. Br J Educ Psychol. 1985;55:185–212.View ArticleGoogle Scholar
- Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in medical education. Acad Med. 2006;81:207–12.View ArticleGoogle Scholar
- Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: a systematic review. Med Teach. 2012;34:e216.View ArticleGoogle Scholar
- Ilic D, Bin Nordin R, Glasziou P, Tilson JK, Villanueva E. Implementation of a blended learning approach to teaching evidence based practice: a protocol for a mixed methods study. BMC Med Educ. 2013;13:170.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.