Phase I: Development of the concept and prototype
The first project year was spent developing a user-friendly and flexible interface framework, based on an authentic pilot case. Through interaction with test students, teachers in dermatology and venereology and other medical-educational expertise, a model gradually emerged.
The user selects a patient in a virtual waiting room and then manages the patient. The following options can be chosen in any order (Fig. 1): suggest questions for medical history, examine the skin (Fig. 2), and suggest further investigations, diagnosis, differential diagnoses and treatment, and writing a medical record. Regardless of which patient is chosen, the same interface meets the user. Patient-specific data is unique for each case and consists of text, images and video clips. When the student has suggested a diagnosis and differential diagnoses, he/she may enter a section called "More about the disease", giving extensive information with a large number of images including differential diagnoses and links to clinical and scientific Internet sites. The concept includes the possibility for the student to: (1) work with authentic patients at his/her own pace, (2) investigate the patients without having to follow a strict point-by-point protocol, (3) write prescriptions and patient records, (4) obtain feedback by comparing his/her own work with that of an experienced colleague (a specialist), and (5) perform a process analysis of the work with each patient, since all transactions within the program are stored in a log file. To be able to retrieve the expert opinion on the various matters, like diagnosis, treatment etc., the student has to enter his/her own suggestions.
Underlying pedagogical considerations
To stimulate student activation and reflection the interactions in NUDOV are designed in a problem-oriented manner where the student leads the investigation and management of the virtual patient. The open structure allows different lines of reasoning, both to reach a diagnosis and in management of the case. So while students mostly arrive at the same diagnosis and roughly the same treatment, they vary in their process of arriving there. To allow individual variation, there is no "one and only" correct way marked out by the software. Instead the student compares his/her actions with those of an expert. This is done when students actively ask for this feedback and is not steered by the system. The expert's actions are normative and correct, but not "the only truth". Consequently, the student is not corrected or scored by the computer, which is an important difference from verification feedback such as "correct" or "wrong answer, try again".
Modern research on the clinical reasoning process and development of expertise asserts that experts develop cognitive instances of typical cases, "illness scripts", and rely on contextual factors in developing and using these [10]. Taking this into account, the case presentations were developed with rich contextual facts for each patient. Another reason to keep contextual factors in the patient presentation is to minimise the risk of depersonalising the students' conception of the case [11]. Students can now anchor the diagnosis to an authentic person. We expect the self-directed usage, the rich presentations and authenticity to support a meaning-oriented approach to learning, which has proven to correlate well with study success for advanced medical students [12].
Technical solutions
The authoring tool Macromedia Authorware 5.2 was used for the overall computer programming of the NUDOV framework. To enhance functionality, additional Windows programming was done using complementary tools. A user's notebook was developed. Video clips were edited and compressed with Microsoft mpeg4 codec. Sound was compressed with mp3 codec. Even though the Authorware authoring system allows the content to be run on the Internet, this was not used in the current version of NUDOV, mostly because of the frequent use of video clips and the download time for these. Although informed consent was obtained from all patients, there was still some hesitation concerning web delivery due to ethical reasons. Other reasons against web delivery included the bandwidth problem with the amount of data involved. Therefore, CD-ROM packaging was decided upon. The CD contains an installation file, which installs the application on an ordinary pc equipped with a sound card.
Phase II: Collection of cases and further feedback from students
During the following two years, typical and authentic cases were recruited. In a venereology case, however, an actor was used. The cases included detailed photographing and video tape-recording of histories and certain practical diagnostic and therapeutic procedures (e.g. "direct microscopy", Doppler sound investigation, taking of skin biopsy specimens, minor skin surgery). In addition, a new extensive digital image archive with differential diagnoses from several hundred patients was collected and organised.
Small groups of students participating in the dermatology and venereology course were recruited to test the new program as it emerged. The students' previous computer experience and "addiction" ranged between diminutive and extensive. Through regular discussions and training sessions, extensive feedback was obtained from the students, enabling us to address didactic problems and to adjust inconsistencies and technical problems.
Phase III: Small-scale implementation and evaluation
The program was evaluated in a prospective study of undergraduates from three consecutive 17-day-courses in dermatology & venereology in the seventh term of the medical programme at Karolinska Institutet. A course administrator randomly assigned the students to one of two study groups: one with conventional teaching (i.e. lectures, seminars/workshops, bed-side teaching, out-patient clinics) without access to NUDOV (non-NUDOV group; n = 85), and another with conventional teaching plus NUDOV (NUDOV-group; n = 31; two 4 h-sessions). This study design was chosen because NUDOV is intended to be a complement to the existing course structure. The NUDOV group had to be smaller for logistical reasons, because a learning laboratory outside the hospital containing only 12 computers had to be used in order to guarantee that no mixing occurred between the two groups. Consequently, no CDs were distributed to any student. There were no significant differences in age or gender between the two study groups. No student complained about the randomisation or objected to participating.
NUDOV was evaluated subjectively with a student questionnaire, completed prior to the course examination. For rating of the program, the students used a semiquantitative 4-stepped scale (e.g. very bad, bad, good, very good). In addition, comparing the scores of the NUDOV and non-NUDOV groups on the final written dermatology and venereology examination made an objective evaluation of the putative impact of NUDOV. A teacher working in a different hospital without any information about NUDOV always constructed the examination, which was assessed under coded conditions.
The examination comprised two sections with 20 questions each, resulting in a possible maximum of 80 points (= 100%). In the first section, the student should give the most likely diagnosis for 20 photographed clinical cases. The second section consisted of short answer questions, both theoretical (e.g. epidemiology, pathophysiology) and with clinical application (e.g. case management, writing of prescriptions).
Ethical considerations
All patients and health care providers gave their written informed consent to participate in the project, and also to appear on the Internet. All students gave their informed consent to participate in the evaluation. Ethical approval from the university or hospital was not required for this type of quality improvement project.
Statistics
The standard deviations from the observed data of the NUDOV group and the non-NUDOV group were used as input in the power calculation. A difference of 5% in performance between the groups was assumed to be of importance. The beta (type II error) and alpha (type I error) values were set to 90% and 5%, respectively. To evaluate the effect of NUDOV on the examination result, a robust linear regression analysis was performed. There was no suspicion of any high leverage points (x-space) and therefore the M-estimator with a bisquare weight function was used to fit the regression line to the data. All analyses were performed using the procedures Power and Robustreg in the SAS software version 9.1.3 (SAS Institute, Inc., N.C., USA).