Students answered significantly more questions correctly when they had worked on a case with a partner. Although this effect is only small, it supports our hypothesis that teamwork increases the retention of medical knowledge from VP cases. It is known from continuing medical education that participants consider team-based case discussions to be important to enhance their learning . Interestingly, in our study students perceived a significantly higher learning effect when they worked by themselves even though their test scores demonstrate the opposite. It is possible that the level of difficulty might not have been appropriate for students with less clinical experience, due to the presentation of too much new information at once . This overload of new information might also have led to a more superficial learning approach . However, when interpreting our results and comparing them with the study by Huwendiek et al. , it needs to be taken into account that our study was performed in a medical computer science course and the VPs were not embedded in a clinical setting. This could lead to certain limitations in the comparison.
In our setting, students with no prior clinical knowledge read the ‘expert comments’ more frequently than students with prior knowledge and needed significantly more time to work on the VP cases. This can easily be attributed to needing more time to interpret the given information and skipping less of the explanations. Students without prior knowledge performed significantly worse on the MC test. This underscores the theory that the activation of prior knowledge, in combination with acquiring new knowledge , seems to be a successful learning strategy when working with VP cases. Hence, the design and content of VP cases should be adapted to the placement of the cases in medical students’ curriculum. Another area for improvement could be the placement of relevant information where it cannot be skipped, rather than on optional cards, like the ‘expert comment’, which can be used at the student’s discretion. Skipping information while using VPs has been documented as an issue in other studies as well . Another important factor to prevent students from skipping information is case length. When asked, students in our study wanted to spend only about 13 minutes working on each case. In actuality, students spent about 13 minutes per case. Cases that appear too long may make students inclined to skip over relevant information.
Relevance, a design aspect  that could not be modified in our VP cases has been described as one of the five core attributes of a conceptual framework for designing patient cases for medical education in general  and is one of the ten principles of virtual patient design . In our e-learning setting, participants considered the content of the VP cases to be very relevant independent of case design, prior knowledge or teamwork. Hence, the topics of our VPs seem to have been chosen well. However, the desire for a take-home message at the end of a VP case to summarize the most important aspects of the case reached the highest score in our final questionnaire. This desire was significantly stronger in students with less prior clinical knowledge. This suggests that Huwendiek et al.’s  recommendation for a ‘focus on relevant learning points’ needs further consideration in our case design to reach the greatest learning benefit, especially when we want to use it with students with less clinical experience . Contrary to other findings on team-based learning [13, 16], working in teams had no significant effect on students’ motivation to work with VP cases in our study.
Of the three case design principles that we modified in our VP cases, Huwendiek et al. postulate that VPs should require students to make all the clinical decisions a doctor would make . This, however, could be very time-consuming. In our study, students wished to spend an average of about 13 minutes working with a VP case. In fact, they did work for approximately 13 minutes per case, even though the cases were designed to take up to 30 minutes when worked on thoroughly. When students worked in teams or had less clinical knowledge, they spent a significant but practically irrelevant one to two minutes more on each case. While adequate study time should be provided when integrating e-learning modules , with respect to our findings, it seems necessary to compromise on the design of VP cases between the authenticity of the students’ tasks, the conveyance of relevant aspects of medical knowledge and the time students might be willing to spend on a VP case. Since students’ ratings for the VP case design criterion ‘narrative style’ did not reveal any significant differences – except for being rated more positively by female students - using a short narrative style for VP cases might be an option to save reading time.
Regarding the use of media, Huwendiek et al. recommend ‘adequate use of media’ to make things as realistic as possible by providing a picture or footage of the patient and specific findings . Although our use of media was less than what Huwendiek et al. postulated to be advantageous , the feedback showed that there was only a moderate wish for more media in the VP cases, mostly among students with high e-learning interest or experience. While this makes our findings compatible, a study by Moreno & Mayer  concluded that more media in e-learning did not have a significant effect on test results. A learning-realism trade-off has also been described to be beneficial in a recently published qualitative study on virtual patient design, which explored what concepts work, and for what reasons .
Furthermore, it has been demonstrated that the choice of teaching strategy (i.e. e-learning being mandatory) had a bigger influence on learning outcomes than intrinsic motivation . Students also worked longer on a case when VPs were newly introduced, but lost interest in working with VPs when the cases were not used in a blended learning approach . Students’ time working with VPs also increased when they were aware that the exam covered the learning objectives embedded in the cases . Therefore, the integration of VPs into a mandatory course – as in our study design, even though students were not aware that an exam was to follow – might be of greater influence on students’ learning than the actual time spent on each case. In general, most aspects of our VP cases were rated significantly more positively by female students and by students with an interest in e-learning, even though these groups did not show significantly better test results. Therefore, in order to make students work with VP cases, student perception should be considered in the design. Curricular planners should take into account when and how to work with VP cases for best learning outcomes, considering the students’ level and applying teamwork.
Strengths and limitations
One strength of our study is that the students were not explicitly recruited, but normal course participants who had not been fully briefed on the purpose of their tasks. They were neither informed about the exact implications of their feedback from each VP, nor that working individually or in a team was one variable being tested in our study. Unfortunately, this design also means that there was no pre-test or control group and that the feedback is potentially less reflective than it would have been with a group that was told to pay attention to certain design details of each VP case. A major limitation of our study is that the VPs were used within a medical computer science course. Students in such a course will approach the VPs with a very different focus than if the VPs were integrated into a clinical course where students would be much more motivated to learn from them. The students were not aware that at the end of the course they were going to be tested on the medical knowledge contained in the five VPs. Thus, their attention was not geared towards retaining certain information from the VP cases in the way it is while preparing for an exam. Due to our repeated pattern of case presentation and evaluation, there might also have been a sequencing effect diminishing the level of reflection in the case feedback. Furthermore, it is a limitation that the questionnaires were not validated.
The dropout rate of students who did not participate in the MC test after the fifth seminar was 26%, most likely due to the fact that it is only necessary to attend three of the five seminars in order to earn the credit for the course. Despite this, we still had 108 participants whose results could be used for statistical analysis. Unfortunately, the multiple choice test we used in this study had to be very short, considering that it was a surprise and that the time for the test had to be deducted from the seminar. To improve the reliability and to confirm the results from this study, a longer MC test with a higher number of questions per VP case needs to be designed. There may also have been a setting bias for the time students spent working on the cases because the cases were presented at the end of regular seminars. Thus, finishing the case meant being able to leave, which may have caused some participants to work faster or more superficially than they might have under different circumstances.