This article has Open Peer Review reports available.
Virtual patients: the influence of case design and teamwork on students’ perception and knowledge – a pilot study
© Jäger et al.; licensee BioMed Central Ltd. 2014
Received: 29 December 2013
Accepted: 30 June 2014
Published: 8 July 2014
Virtual patient (VP) cases are an effective teaching method, although little is known about how to design and implement them for maximum effectiveness. The aim of this study was to explore the effect of case design and teamwork on students’ learning outcome.
One hundred forty-six undergraduate medical students participated in a mandatory medical computer science course consisting of five seminars. At the end of each seminar, they worked on one VP case, either in teams of two or individually. Each student filled out an introductory and a final survey and a feedback sheet after completing each case. Additionally, there was a surprise multiple choice (MC) test after the last seminar with three questions regarding each case.
Students with more clinical experience and students who had worked in a team performed significantly better on MC questions. Students with less clinical experience more frequently used information which had been positioned less prominently on the case material. Certain aspects of case design were rated more positively by students who had an interest in e-learning. In general, students preferred to work on cases for less than 15 minutes.
Clinically more advanced students and students working with a partner seem to benefit most from short VP cases with prominently presented information.
In times of digitalization and ubiquitous internet connections, the use of computer-based media is establishing itself as an ever-growing domain in medical education . E-learning allows access that is not limited by time constraints. Furthermore, interactive learning has become an integral part of education at many medical schools . It has also been demonstrated that e-learning can be just as effective as conventional teaching methods . A growing branch of e-learning is the use of virtual patients (VPs). VPs are becoming particularly popular for teaching clinical reasoning , due to the interactive learning experience, which can simulate some of the diagnostic steps and the clinical decision-making processes of physicians’ daily work. Additionally, they have been shown to be effective at propagating knowledge .
Many different approaches and programs for designing VP cases, including different structures and layouts, are currently available . The CASUS platform, for instance, is a system for ‘multimedia paper cases’, which provides a basis for using cases in different countries and languages . This is advantageous as it has been suggested that a simple and easily accessible design of the e-learning platform can improve results when working with VPs .
In order to optimize the effectiveness of VP cases, design and implementation play an important role. Regarding implementation, it seems that relevance to seminars and corresponding tests , as well as a smooth, balanced  and functional integration into the curriculum , are important in order to motivate students to actually use the cases. For case design, a focus group study conducted by Huwendiek et al.  identified important aspects to consider and offered suggestions on how to optimize the design of VPs, also stating that these findings could be followed up in quantitative studies.
Another important aspect is teamwork, which has been shown to activate learners and to enhance knowledge  and, therefore, might also be relevant for learners’ motivation and retention of knowledge when working with VPs. In a study by Edelbring et al. , students reported a perceived benefit to their clinical reasoning skills from working on VP cases with a partner and discussing their patient management approaches. Furthermore, motivation has been identified as being a dependent variable influenced by autonomy , which is also a necessary skill when working on VP cases. However, in one survey among students working with VPs, up to 86% of the participants preferred to work in teams or at least had no preference towards individual work .
With this study, we attempted to identify aspects of case design, as well as aspects of working with VP cases, which might affect students’ perception of and learning success with VP cases. We designed and implemented five VP cases, where three main design aspects (narrative style, question type, question content) from the literature  were modified between cases, and students either worked alone or with a partner in order to answer the following research questions: Is a particular version of these three design aspects of VP cases favoured by students? Does working on a VP case together with a partner result in a better learning outcome? We hypothesize that certain aspects of VP case design are favoured by students and that teamwork will lead to greater retention of knowledge.
Case 1: hepatitis
Case 2: pneumothorax
A patient returns from a trip to Africa with symptoms of an infection.
A passenger on an international flight experiences acute chest pain.
Suggested study time
Objectives and outcomes
Students will learn the differential diagnoses, relevant lab results and serum tests for hepatitis.
Students will learn the first measures for dealing with acute chest pain, as well as causes, symptoms, diagnostics and treatment of a spontaneous pneumothorax.
Case is told from the point of view of a single treating physician.
Case is told from the point of view of a single treating physician.
Patient is presented using precise descriptions and condensed information.
The case is told like a story, describing the setting and background of the situation beyond necessary information.
Pictures of physical findings (e.g. jaundice), tables of lab results
Radiology findings, videos on how to place a chest tube
Nine MC questions, some with multiple correct answers. ‘Mouseover’ explanations, hyperlinks and expert comments
Eight textboxes to answer questions and compare with the suggested answers. ‘Mouseover’ explanations, hyperlinks and expert comments
Facts about the disease, lab results and differential diagnoses; can be answered without reading the case
Diagnostic steps, interpreting findings, how to treat the patient; details which the treating physician would think about in that situation
Linear (string of pearls)
Linear (string of pearls)
Explanations regarding the correctness/incorrectness of each MC option
Reference answers and explanations for the users to compare to their own input
Detailed information on jaundice and hepatitis B serology
Video instructions on placing a chest tube, X-ray studies of pneumothorax
Example MC question
Please interpret the following test results: Anti-Hbc-antibodies: positive; anti-Hbs-antibodies: positive; HBs-antigen: negative
No final conclusion can be made without knowing the patient’s Anti-Hbe status
Acute hepatitis B infection
Chronic hepatitis B infection
Status after hepatitis B vaccination
Status after hepatitis B infection*
Study design and participants
At the Hamburg Medical School, the third, fourth, and fifth years of a six-year medical curriculum include seven different modules of 12 weeks each, which can be completed by the students in any order they choose. Hence, students’ knowledge and experience levels can widely vary depending on the number of modules they have already finished. In October 2012, 146 students entered the module ‘the head’, which includes a course in medical computer science. This course consists of five seminars which are taught in seven groups of about 20 students each and take place in the first five consecutive weeks of this module. Students are randomly assigned to a seminar group at the beginning of the module. For this study, students worked on one of the five newly designed e-learning cases at the end of each seminar, where 30 minutes were dedicated to working with each case. Cases were provided in the order mentioned above. Every week, the students worked on a case either by themselves or with a partner (about 50% of the students from each group had to work with a partner due to the number of computers available for each group), whereby many alternated between solo and teamwork. Participation was voluntary and anonymous. A member of the Ethics Committee of the Chamber of Physicians, Hamburg, confirmed the innocuousness of the research protocol and written informed consent was obtained from participants. Before working with the first case, students filled out the introductory survey. After each case, a feedback sheet was completed and after the fifth case, the final survey and the multiple choice questions were answered.
Only students who handed in the introductory survey, answered the multiple choice questions and completed at least three case feedback sheets were included in the analysis. This selection was made because in order to receive credits for this course it is mandatory to attend at least three seminars. For comparison, participants with complete data sets were dichotomised with respect to the information about their prior knowledge, namely students in their first clinical semester and students in higher clinical semesters. This was done to account for their different perspectives and skills. For evaluation of the multiple choice test, the scores were divided into five groups (one for each case) with three questions each. To estimate the relationship between the different case rating aspects and teamwork as well as the three case design characteristics narrative style, question type and question content fitted mixed model was used while these predictors were modelled as fixed effects. For adjustment of the cluster structure, resulting from the multiple measurements, every student rated at least one VP case and every VP case was rated by several students. Both clusters were modelled as crossed random effects. Further variables of interest or potential confounders, which were also modelled as fixed effects were students’ gender, age, previous knowledge, e-learning interest and e-learning experience. Additionally, an interaction term between teamwork and previous knowledge was included and excluded when not significant. Aspects from the final survey were analysed by regression analysis with the same list of predictors used for the mixed model analysis. Adjusted means and 95%-confidence intervals (95%-CI) are reported. P-values <0.05, two sided, were considered significant. Nominal p-values are reported without correction for multiplicity. All analyses were conducted using Stata 13.1, STATA Corporation, college station, Texas, US.
Students’ characteristics with respect to their prior clinical knowledge and their disposition towards e-learning
Age mean (SD)
E-learning interest mean (SD)
E-learning experience mean (SD)
Prior clinical knowledge
No prior clinical knowledge
Effects of student characteristics on case feedback and MC score
Relevance mean [95%-CI]
Motivation mean [95%-CI]
Perceived learning effect mean [95%-CI]
Narrative style mean [95%-CI]
Question type mean [95%-CI]
Question content mean [95%-CI]
Minutes spent working mean [95%-CI]
MC score mean [95%-CI]
Influence of student characteristics on final evaluation questionnaire
Ideal case duration mean [95%-CI]
More media usage would improve the cases mean [95%-CI]
I could imagine working on similar cases outside of class mean [95%-CI]
I would like a ‘take-home message’ at the end mean [95%-CI]
I read all answer comments thoroughly mean [95%-CI]
I read the ‘expert comment’ (%) mean [95%-CI]
4.2* [4.1; 4.4]
5.0* [4.9; 5.2]
48%** [44%; 53%]
4.0* [3.9; 4.1]
4.8 [4.7; 4.9]
38% [35%; 41%]
13.5* [13.1; 13.9]
4.1* [4.0; 4.2]
48%** [45%; 52%]
12.7 [12.3; 13.2]
5.0* [4.9; 5.1]
3.8 [3.7; 3.9]
33% [30%; 37%]
0.17*1 [0.07; 0.27]
0.42*1 [0.07; 0.77]
0.14*1 [0.04; 0.24]
0.27**1 [0.1; 0.36]
0.09*1 [0.00; 0.19]
-0.10*1 [-0.20; 0.00]
Students answered significantly more questions correctly when they had worked on a case with a partner. Although this effect is only small, it supports our hypothesis that teamwork increases the retention of medical knowledge from VP cases. It is known from continuing medical education that participants consider team-based case discussions to be important to enhance their learning . Interestingly, in our study students perceived a significantly higher learning effect when they worked by themselves even though their test scores demonstrate the opposite. It is possible that the level of difficulty might not have been appropriate for students with less clinical experience, due to the presentation of too much new information at once . This overload of new information might also have led to a more superficial learning approach . However, when interpreting our results and comparing them with the study by Huwendiek et al. , it needs to be taken into account that our study was performed in a medical computer science course and the VPs were not embedded in a clinical setting. This could lead to certain limitations in the comparison.
In our setting, students with no prior clinical knowledge read the ‘expert comments’ more frequently than students with prior knowledge and needed significantly more time to work on the VP cases. This can easily be attributed to needing more time to interpret the given information and skipping less of the explanations. Students without prior knowledge performed significantly worse on the MC test. This underscores the theory that the activation of prior knowledge, in combination with acquiring new knowledge , seems to be a successful learning strategy when working with VP cases. Hence, the design and content of VP cases should be adapted to the placement of the cases in medical students’ curriculum. Another area for improvement could be the placement of relevant information where it cannot be skipped, rather than on optional cards, like the ‘expert comment’, which can be used at the student’s discretion. Skipping information while using VPs has been documented as an issue in other studies as well . Another important factor to prevent students from skipping information is case length. When asked, students in our study wanted to spend only about 13 minutes working on each case. In actuality, students spent about 13 minutes per case. Cases that appear too long may make students inclined to skip over relevant information.
Relevance, a design aspect  that could not be modified in our VP cases has been described as one of the five core attributes of a conceptual framework for designing patient cases for medical education in general  and is one of the ten principles of virtual patient design . In our e-learning setting, participants considered the content of the VP cases to be very relevant independent of case design, prior knowledge or teamwork. Hence, the topics of our VPs seem to have been chosen well. However, the desire for a take-home message at the end of a VP case to summarize the most important aspects of the case reached the highest score in our final questionnaire. This desire was significantly stronger in students with less prior clinical knowledge. This suggests that Huwendiek et al.’s  recommendation for a ‘focus on relevant learning points’ needs further consideration in our case design to reach the greatest learning benefit, especially when we want to use it with students with less clinical experience . Contrary to other findings on team-based learning [13, 16], working in teams had no significant effect on students’ motivation to work with VP cases in our study.
Of the three case design principles that we modified in our VP cases, Huwendiek et al. postulate that VPs should require students to make all the clinical decisions a doctor would make . This, however, could be very time-consuming. In our study, students wished to spend an average of about 13 minutes working with a VP case. In fact, they did work for approximately 13 minutes per case, even though the cases were designed to take up to 30 minutes when worked on thoroughly. When students worked in teams or had less clinical knowledge, they spent a significant but practically irrelevant one to two minutes more on each case. While adequate study time should be provided when integrating e-learning modules , with respect to our findings, it seems necessary to compromise on the design of VP cases between the authenticity of the students’ tasks, the conveyance of relevant aspects of medical knowledge and the time students might be willing to spend on a VP case. Since students’ ratings for the VP case design criterion ‘narrative style’ did not reveal any significant differences – except for being rated more positively by female students - using a short narrative style for VP cases might be an option to save reading time.
Regarding the use of media, Huwendiek et al. recommend ‘adequate use of media’ to make things as realistic as possible by providing a picture or footage of the patient and specific findings . Although our use of media was less than what Huwendiek et al. postulated to be advantageous , the feedback showed that there was only a moderate wish for more media in the VP cases, mostly among students with high e-learning interest or experience. While this makes our findings compatible, a study by Moreno & Mayer  concluded that more media in e-learning did not have a significant effect on test results. A learning-realism trade-off has also been described to be beneficial in a recently published qualitative study on virtual patient design, which explored what concepts work, and for what reasons .
Furthermore, it has been demonstrated that the choice of teaching strategy (i.e. e-learning being mandatory) had a bigger influence on learning outcomes than intrinsic motivation . Students also worked longer on a case when VPs were newly introduced, but lost interest in working with VPs when the cases were not used in a blended learning approach . Students’ time working with VPs also increased when they were aware that the exam covered the learning objectives embedded in the cases . Therefore, the integration of VPs into a mandatory course – as in our study design, even though students were not aware that an exam was to follow – might be of greater influence on students’ learning than the actual time spent on each case. In general, most aspects of our VP cases were rated significantly more positively by female students and by students with an interest in e-learning, even though these groups did not show significantly better test results. Therefore, in order to make students work with VP cases, student perception should be considered in the design. Curricular planners should take into account when and how to work with VP cases for best learning outcomes, considering the students’ level and applying teamwork.
Strengths and limitations
One strength of our study is that the students were not explicitly recruited, but normal course participants who had not been fully briefed on the purpose of their tasks. They were neither informed about the exact implications of their feedback from each VP, nor that working individually or in a team was one variable being tested in our study. Unfortunately, this design also means that there was no pre-test or control group and that the feedback is potentially less reflective than it would have been with a group that was told to pay attention to certain design details of each VP case. A major limitation of our study is that the VPs were used within a medical computer science course. Students in such a course will approach the VPs with a very different focus than if the VPs were integrated into a clinical course where students would be much more motivated to learn from them. The students were not aware that at the end of the course they were going to be tested on the medical knowledge contained in the five VPs. Thus, their attention was not geared towards retaining certain information from the VP cases in the way it is while preparing for an exam. Due to our repeated pattern of case presentation and evaluation, there might also have been a sequencing effect diminishing the level of reflection in the case feedback. Furthermore, it is a limitation that the questionnaires were not validated.
The dropout rate of students who did not participate in the MC test after the fifth seminar was 26%, most likely due to the fact that it is only necessary to attend three of the five seminars in order to earn the credit for the course. Despite this, we still had 108 participants whose results could be used for statistical analysis. Unfortunately, the multiple choice test we used in this study had to be very short, considering that it was a surprise and that the time for the test had to be deducted from the seminar. To improve the reliability and to confirm the results from this study, a longer MC test with a higher number of questions per VP case needs to be designed. There may also have been a setting bias for the time students spent working on the cases because the cases were presented at the end of regular seminars. Thus, finishing the case meant being able to leave, which may have caused some participants to work faster or more superficially than they might have under different circumstances.
Considering that students are willing to spend about 15 minutes on average for working with a VP case, short cases with relevant diseases, strategic placement of important information in prominent places and a take-home message seem to be the most important design aspects. Teamwork and cases adapted to the students’ level seem to enhance knowledge retention and thus should be considered in a blended learning approach. Further studies with VP cases, adapted to these requirements and followed by a more extensive and reliable knowledge test, need to be designed in order to corroborate the findings from this study.
The authors would like to thank all students who participated in this study. Special thanks to Martha M. Gilbert for editing the final draft.
- Ellaway R, Masters K: AMEE guide 32: e-Learning in medical education part 1: learning, teaching and assessment. Med Teach. 2008, 30: 474-489.View ArticleGoogle Scholar
- Triola MM, Huwendiek S, Levinson AJ, Cook DA: New directions in e-learning research in health professions education: report of two symposia. Med Teach. 2012, 34: e15-e20.View ArticleGoogle Scholar
- Davis J, Crabb S, Rogers E, Zamora J, Khan K: Computer-based teaching is as good as face to face lecture-based teaching of evidence based medicine: a randomized controlled trial. Med Teach. 2008, 30: 302-307.View ArticleGoogle Scholar
- Cook DA, Triola MM: Virtual patients: a critical literature review and proposed next steps. Med Educ. 2009, 43: 303-311.View ArticleGoogle Scholar
- Cook DA, Erwin PJ, Triola MM: Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med. 2010, 85: 1589-1602.View ArticleGoogle Scholar
- Huwendiek S, de Leng BA, Zary N, Fischer MR, Ruiz JG, Ellaway R: Towards a typology of virtual patients. Med Teach. 2009, 31: 743-748.View ArticleGoogle Scholar
- Kolb S, Reichert J, Hege I, Praml G, Bellido MC, Martinez-Jaretta B, Fischer M, Nowak D, Radon K, NetWoRM group: European dissemination of a web- and case-based learning system for occupational medicine: NetWoRM Europe. Int Arch Occup Environ Health. 2007, 80: 553-557.View ArticleGoogle Scholar
- Woelber JP, Hilbert TS, Ratka-Krüger P: Can easy-to-use software deliver effective e-learning in dental education? a randomised controlled study. Eur J Dent Educ. 2012, 16: 187-192.View ArticleGoogle Scholar
- Hege I, Ropp V, Adler M, Radon K, Mäsch G, Lyon H, Fischer MR: Experiences with different integration strategies of case-based e-learning. Med Teach. 2007, 29: 791-797.View ArticleGoogle Scholar
- Berman N, Fall LH, Smith S, Levine DA, Maloney CG, Potts M, Siegel B, Foster-Johnson L: Integration strategies for using virtual patients in clinical clerkships. Acad Med. 2009, 84 (7): 942-949.View ArticleGoogle Scholar
- Edelbring S1, Broström O, Henriksson P, Vassiliou D, Spaak J, Dahlgren LO, Fors U, Zary N: Integrating virtual patients into courses: follow-up seminars and perceived benefit. Med Educ. 2012, 46 (4): 417-425. Edelbring 1View ArticleGoogle Scholar
- Huwendiek S, Reichert F, Bosse HM, de Leng BA, van der Vleuten CPM, Haag M, Hoffmann GF, Tönshoff B: Design principles for virtual patients: a focus group study among students. Med Educ. 2009, 43: 580-588.View ArticleGoogle Scholar
- Kühne-Eversmann L, Eversmann T, Fischer MR: Team- and case-based learning to activate participants and enhance knowledge: an evaluation of seminars in Germany. J Contin Educ Health Prof. 2008, 28: 165-171.View ArticleGoogle Scholar
- Edelbring S, Dastmalchi M, Hult H, Lundberg IE, Dahlgren LO: Experiencing virtual patients in clinical learning: a phenomenological study. Adv Health Sci Educ Theory Pract. 2011, 16 (3): 331-345. Edelbring 2View ArticleGoogle Scholar
- Kusurkar RA, Ten Cate TJ, van Asperen M, Croiset G: Motivation as an independent and a dependent variable in medical education: a review of the literature. Med Teach. 2011, 33: e242-e262.View ArticleGoogle Scholar
- Riemer M, Abendroth M: Virtual patients: which kind of teaching method is most attractive to medical students?. GMS Med Inform Biom Epidemiol. 2013, 9: Docx1-Article in GermanGoogle Scholar
- Fischer MR, Schauer S, Gräsel C, Baehring T, Mandl H, Gärtner R, Schwerbaum W, Scriba PC: CASUS model trail. a computer-assisted author system for problem-oriented learning in medicine. Z Arztl Fortbild. 1996, 90: 385-389. Article in GermanGoogle Scholar
- Svirko E, Mellanby J: Attitudes to e-learning, learning style and achievement in learning neuroanatomy by medical students. Med Teach. 2008, 30: e219-e227.View ArticleGoogle Scholar
- Bateman J, Allen M, Samani D, Kidd J, Davies D: Virtual patient design: exploring what works and why. a grounded theory study. Med Educ. 2013, 47: 595-606.View ArticleGoogle Scholar
- Kim S, Phillips WR, Pinsky L, Brock D, Phillips K, Keary J: A conceptual framework for developing teaching cases: a review and synthesis of the literature across disciplines. Med Educ. 2006, 40: 867-876.View ArticleGoogle Scholar
- Cook DA, Dupras DM: A practical guide to developing effective web-based learning. J Gen Int Med. 2004, 19: 698-707.View ArticleGoogle Scholar
- Moreno R, Mayer R: Learning science in virtual reality multimedia environments: role of methods and media. J Educ Psychol. 2002, 94: 598-610.View ArticleGoogle Scholar
- Mahnken AH, Baumann M, Meister M, Schmitt V, Fischer MR: Blended learning in radiology: is elf-determined learning really more effective?. Eur J Radiol. 2011, 78: 384-387.View ArticleGoogle Scholar
- Abendroth M, Harendza S, Riemer M: Clinical decision making: a pilot e-learning study. Clin Teach. 2013, 10: 51-55.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/14/137/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.