Development of an e-learning module on leukaemia
Authoring program
Adobe Captivate™ v5.5 was chosen as the authoring program because of its balance of design options and ease of use. Moreover, the published output can be displayed via any web browser, and the product can also be adapted to display on mobile devices (e.g. smart phones), in keeping with the current e-learning trend toward greater use of mobile technology [16].
Module design
A range of key criteria were utilised in the design of the e-learning module on leukaemia. The most important factors contributing to the efficacy of previous e-learning modules include [7, 9, 11, 12]:
□ Authenticity - case based to optimise real-world relevance;
□ Interactivity - responds to users input, as well as being malleable for different types of learners and differing levels of ability;
□ Feedback - facilitates remediation of misconceptions; and
□ Integration - provision of an overarching conceptual framework that takes into account prior learning and curriculum objectives, and brings together concepts in a clinical context.
Framework
The aim of the module was to link the clinical approach to leukaemia with the basic sciences, particularly in relation to diagnostic protocols. This has been successfully achieved in previous UNSW pathology modules on other topics [9, 17].
Learning objectives were emphasised in the introductory screen of the module, which was divided into two main sections: ‘concepts and causes’ and ‘case studies’ (Figure 1). The concepts and causes section commenced with an introduction to leukaemia (definition, the function of normal peripheral blood leucocytes, and an analysis of cell lineages). Following the introduction there were three subsections, which dealt with the major concepts in pathogenesis and diagnosis of distinct disease profiles: acute leukaemia, including acute lymphoblastic leukaemia (ALL) and acute myeloid leukaemia (AML); chronic lymphocytic leukaemia (CLL); and chronic myeloid leukaemia (CML). The ‘acute leukaemia’ section dealt with both AML and ALL in a side-by-side comparison. Throughout the concepts and causes section, interactive tasks linked to learning objectives were included to enhance student engagement.
The concepts and causes section was intended to facilitate understanding of normal haematopoiesis and to identify abnormalities associated with leukaemia. Each of the characteristic disease profiles was discussed using the following structure:
-
1.
Definition;
-
2.
Epidemiology and risk factors;
-
3.
Clinical features;
-
4.
Diagnosis;
-
5.
Staging and prognostic factors.
Four case studies were provided, each extracted from authentic cases of ALL, AML, CLL and CML respectively (these were generously provided by Prof. Fred Dee, University of Iowa). All case studies provided the de-identified patient’s history, relevant findings on examination, and results of diagnostic investigations. As details of each case were revealed on each successive screen, formative assessment questions and relevant feedback were presented regarding differential diagnosis, selection and interpretation of diagnostic investigations, and prognosis.
Interface and navigation
The interface was kept simple, uncluttered and consistent throughout the module, from the introductory pages to the case studies and glossary to prevent distractions and excessive cognitive load that might occur with a complex or unintuitive interface. Text on each screen was kept to a minimum to avoid overwhelming users by creating an on-line textbook environment. Audio was employed to supplement and expand on information provided in text, both to diversify the mode of presentation and to maintain engagement.
Perpetual functions included forward and back options, and a return to main menu button. Occasionally, a side branch from the main content path was utilised to provide additional or background information on a specific concept, for example, explaining the significance of immunoglobulin light chain monoclonality in CLL. These side branches were kept to a maximum of two screens, and subsequently returned the user to the screen from which they diverged.
Feedback and interactive features
In the context of e-learning modules, interactivity and feedback are core features in promoting learning, and immediate feedback is vital for the learning process [18]. Throughout the module, users are encouraged to interact with the concepts presented by answering questions and identifying features on images. Feedback on answers to the questions is provided immediately via audio or by a ‘roll over’ link (Figures 2 and 3). When the user holds the mouse cursor over a highlighted area on the screen (a roll-over) a text box appears with the answer. Additionally, some screens do not permit progress unless the question is attempted and feedback is returned, thereby preventing students from simply clicking through the module without engaging with the content.
Definitions of key terms were provided in a similar manner. Students can also directly access definitions from the screen containing the term through roll over boxes. The immediacy of access to information and feedback reduced the germane load (cognitive load required to process the interface) placed on students, allowing them to focus on the concepts presented [19].
Digital videos
Digital videos were embedded in the module to provide students with information about the diagnosis of common haematological disorders and to portray investigative procedures that they might not have the opportunity to witness during their clinical attachments.
Web access
The World-Wide Web was used as a means of distribution to geographically dispersed participants. The module was converted to a Flash™ file then uploaded to a web server, enabling users to view the module from home, at urban and rural clinical schools, as well as at the main university campus. Access was password-protected for the purposes of this trial. The module is available to view at http://web.med.unsw.edu.au/Pathology/Leukaemia/Leukaemia.htm.
Study Design
Participants
Students enrolled in Phase Three (years 5 and 6, a cohort of 520 students) of the six-year undergraduate medicine program at UNSW in 2011 were invited via email to participate in a randomised controlled trial of the e-learning module on leukaemia, of which 45 responded (37 females, 8 males). These students were matched for academic ability (based on weighted average mark - ‘WAM’ - in the program) and gender (to minimise any gender-related differences). Volunteers were then randomised into either: the Study group (n = 23), who were provided with access to the new e-learning module; or the Control group (n = 22), who were provided with access to existing online resources. Participants were advised that their individual results from the pre-test and post-test would remain confidential and would not impact on their academic standing. The study received ethics approval from the UNSW Faculty of Medicine Human Research Ethics Advisory Panel (Ethics Approval No: 10095).
Instructions to participants
Participants were emailed instructions for the 2-week study period. The Study group email included a link to the module, with each individual receiving a unique username and password for the duration of the study, which they were asked to keep confidential. The control group was provided with links and encouraged to utilise currently available e-learning resources on leukaemia:
-
1.
Robbins Pathologic Basis of Disease (8th Edition) via MD Consult (available via the UNSW Library) - Section on Neoplastic Proliferations of White Cells; and
-
2.
American Society of Hematology (ASH) Teaching Cases URL: http://teachingcases.hematology.org/
These resources were selected as equivalents to the ‘causes and concepts’ and ‘case studies’ components of our module, respectively. Importantly, the online textbook chapter addressed the causes and pathophysiology of leukaemia at the same (or greater) depth compared with our e-learning module. Further, the ASH cases titled ‘Childhood Acute Leukemia’, ‘A Patient with Pancytopenia’, ‘Lymphocytosis’ and ‘Myeloproliferative Disorder’ address CLL, AML, CLL and CML respectively at the same (or greater) depth compared with the cases in our e-learning module.
Pre-test, post-test and questionnaire
The pre-test, post-test and evaluation questionnaire were all designed using Questionmark PerceptionTM (Questionmark, UK), a well-established suite of software for authoring and delivering web-based assessments and surveys. Feedback was provided for all questions upon completion of the pre-test and post-test.
The pre-test and post-test were administered immediately preceding and immediately following the two-week trial period, respectively. Both tests were based around case studies, to reflect the clinically oriented learning of senior medical students. Each case study included a clinical history, followed by several objective items. These were provided in several formats including standard multiple choice (single best answer), multiple response, and drag-and-drop (for image-based questions). Students were presented with haematology and immunohistochemistry results, and histological and radiological images, as appropriate for each case, then asked to answer questions relating to differential diagnosis, diagnostic investigations, provisional diagnosis, pathogenesis, and prognosis. Both the pre-test and post-test were reviewed by two senior members of the academic staff in Pathology at UNSW, as well as a senior clinical haematologist. All of them rated each test as being of equivalent difficulty, and ensured that the material covered by the test was addressed by both the e-learning module and the alternative e-learning resources. The post-test was authored after the e-learning module had been developed, thereby avoiding the potential bias of ‘teaching to the test’.
In addition, all participants were asked to complete a linked questionnaire at the conclusion of the post-test. The questionnaire obtained evaluative feedback regarding the module (Study group) and the alternative e-learning resources (Control group). Five-point Likert scales (1 = strongly disagree, 5 = strongly agree) were used for questions regarding module design and content. Free text responses were utilised to gather information about the most valuable features of the module, as well as suggestions for improvement. A PDF version of the questionnaire is available upon request.
Evaluating the efficacy of the e-learning module
Statistical power
Prior to commencement of the study, it was determined that in order to show a 20 % difference between groups with statistical power > 99 %, a sample size of 15 participants per group was required (n = 30).
Quantitative analysis
All statistical analyses were performed using IBM SPSS Statistics™, version 19. Student’s t tests were performed to compare WAM, pre-test and post-test scores between groups. Stepwise linear regression analysis was performed to determine the factors that contributed significantly to variance in post-test scores. All data regarding WAM, pre-tests and post-tests for both groups are expressed as mean percentage scores ± standard error of the mean.
Data obtained from the evaluation questionnaires was analysed as follows: Kruskal-Wallis test and Dunn’s multiple comparisons tests were employed to compare participants’ ratings of the perceived difficulty of the topic of leukaemia before and after the trial, both within groups and between groups. Mann-Whitney U tests were performed to compare Likert scale data between groups. Likert scale and perceived difficulty data are expressed as median ratings ± interquartile range.
Qualitative analysis
Online evaluation questionnaires were administered immediately following the post-test to gather participants views regarding the e-learning module (Study group) and the alternative online learning resources (Control group). Open-ended questions were analysed for each group.