Comparison of a web-based package with tutor-based methods of teaching respiratory medicine: subjective and objective evaluations
© Smith et al; licensee BioMed Central Ltd. 2007
Received: 13 December 2006
Accepted: 01 November 2007
Published: 01 November 2007
Respiratory disease is a major cause of morbidity and mortality not only in the United Kingdom, but globally. A good understanding of respiratory disease and its treatment is essential for all medical graduates. As a result of changes in clinical practice, patients with some common respiratory illnesses are less often admitted to hospital, restricting the experience available to undergraduate students. Combined with a potential shortage of clinical teachers, this means that new methods of teaching need to be developed and appraised. The aim of this study was to establish whether a web-based package on the diagnosis of respiratory disease would be as effective and as acceptable to final year medical students as tutor-led methods of teaching the same material.
137 out of 315 final year undergraduate students in a single medical school volunteered to take part. Each received up to two hours of tutor-lead interactive, tutor-lead didactic or electronic, Web-based teaching on the accurate diagnosis and management of respiratory disease. Post teaching performance was assessed by multiple true/false questions and data interpretation exercises, whilst students' teaching preferences were assessed by questionnaire.
Despite a high knowledge baseline before the study, there was a small, but statistically significant increase in knowledge score after all forms of teaching. Similarly, data interpretation skills improved in all groups, irrespective of teaching format, Although paradoxically most students expressed a preference for interactive tutor-lead teaching, spirometry interpretation in those receiving web-based teaching improved significantly more [p = 0.041] than in those in the interactive group.
Web-based teaching is at least as good as other teaching formats, but we need to overcome students' reluctance to engage with this teaching method.
The United Kingdom, like many other countries, has a shortage of doctors. As a result, there has been a significant recent increase in the number of students entering medical school. Over the same time there has been a reduction in appointment of academic clinicians [1–3], an increase in pressure on clinical staff to deliver demanding service targets and the introduction of legislation limiting hours in the workplace. Together these factors have combined to reduce the number of staff available to teach. An additional problem for teachers of respiratory medicine is that many respiratory patients are treated within the community, rather than as hospital in-patients, thus potentially limiting the range of direct experience open to undergraduate and junior postgraduate trainees. Another significant pressure at present is the need in the United Kingdom to develop a coherent programme of postgraduate education in response to the recent re-structuring of clinical training in the first two postgraduate years. As a result of these combined pressures, many medical schools are developing e-learning packages which will meet current or anticipated shortfalls in undergraduate teaching and which, in addition, can be made available as postgraduate learning resources.
For the respiratory clinician, a particularly important topic for both undergraduate and postgraduate trainees is the use and interpretation of spirometry (lung function tests). Correctly used, spirometry is a powerful diagnostic tool which can also be used to monitor disease progression and the effectiveness of treatment. Despite this, published evidence suggests that spirometry is under-used both in primary care and hospital settings and is also poorly understood by clinical trainees [4–8]. In addition, a recent review of the literature has highlighted the apparent rarity of teaching on pulmonary function testing in undergraduate curricula .
Therefore the aim of the current study was to develop an electronic module which could be used as a stand-alone session on the correct diagnosis of respiratory illness including the role of spirometry as a diagnostic tool. To our knowledge, there have been no systematic investigations of the value of electronic tools for teaching respiratory medicine. In order to validate the module we performed a controlled trial, comparing subjective and objective performance before and after exposure to the e-module, a didactic lecture or a tutor-lead interactive session on the same academic content.
Subject recruitment and study design
A formal didactic 90 minute lecture (n = 40).
An interactive 90 minute lecture by the same tutor using case studies (n = 40).
A customised web-based interactive learning tool using the same case studies as those used in group two (n = 57). Interactive learning materials were produced in Word and uploaded onto the Imperial College intranet using a WebCT platform. Students were allowed to access the material for two hours and study it at their own pace. This format was slightly longer than the tutor-lead formats, to allow students who might wish to revisit some parts of the material, to do so. They were given neither input from an experienced teacher nor any opportunity to ask questions of a teacher during the session, but they were permitted to discuss the material with their peers and work together if they so desired.
On being allocated to a group, students were told to which room they should report for the additional teaching. Thus, students allocated to the computer lab were able to deduce that they were likely to receive computer based teaching.
Content of learning materials
An independent experienced respiratory teacher who was not involved in the study reviewed all learning and evaluation materials prior to use. In all cases the key learning outcomes for the teaching included the following.
On completion, the student should be able to:
Summarise the national significance of lung disease
Describe common respiratory symptoms
Differentiate asthma from chronic obstructive pulmonary disease (COPD) on the basis of history and investigations including spirometry
Know how spirometry should be performed and define key measurements of lung function
Classify respiratory diseases into obstructive and restrictive (or small lung) disease using spirometric and other data
Be able to interpret common spirometric abnormalities
Assessment of student performance before and after learning session
Pre- and post-assessment of all participants was undertaken to determine the effectiveness of knowledge transfer by the three teaching formats. Participants completed the same ten multiple-true/false questions before and after their teaching session. Two control questions assessed topics which were not covered in any of the teaching formats.
The second part of the pre- and post-assessment concerned data interpretation and because of the nature of the material it was not thought appropriate to use the same data more than once. The data interpretation section which the undergraduates completed before the project therefore included five items associated with the symptom of breathlessness, followed by two sets of spirometry results for them to report. The data which the undergraduates interpreted after the teaching session included three causes of breathlessness and three examples of spirometry.
All students were required to complete the assessments independently, without conferring with colleagues.
All scripts were marked blind using a pre-prepared mark scheme. Differences in knowledge transfer were analysed using ANOVA followed by 2-tailed non-parametric Mann-Whitney tests for each individual item. Since different questions were asked before and after teaching, the effect of teaching within a group was analysed by expressing marks as percentages both for each question and for the entire test and analysing the percentages. Differences in performance on the data interpretation questions between groups were analysed using Repeated MeasuresGeneralized Linear Models.
Finally, at the end of the teaching session, all students were asked to identify their preferred learning format and were asked a series of open questions to elucidate their views about the format of teaching they had experienced. Twenty-five percent of responses were themed by the two lead investigators (SFS and MRP) and comments relating to major themes enumerated for all participants .
This study was approved by the Head of Undergraduate Medicine, Faculty of Medicine and the Head of the National Heart and Lung Institute Division of Imperial College London.
There was no difference in any outcome between the two computer lead groups, suggesting that there was little communication between these groups between sessions. These data were therefore pooled and analysed as a single group. Although access to the E-learning modules was for the same time period for all students, the ways in which they used the package varied enormously. Time actually spent logged onto the relevant webpages varied from less than 5 minutes to the full two hours allocated. Those on line for a very short time were probably working with a colleague, so only one of them remained logged in. Forty six students were logged on for less than 60 minutes in total, 9 for 60 – 120 minutes and 2 for more than 120 minutes. Similarly, whilst some students remained logged in throughout their use of the material, others logged in and out, visiting other sites during the session. The maximum number of hits by one student was 11 in a period of 59 minutes.
Assessment of student performance
Student evaluation of teaching formats
When asked specifically what they liked and disliked about the teaching format they had experienced in the study, many of the students who had been allocated to the computer group reported liking it because it was self-paced (38/57) and because they were able to re-visit material at will (20/57). They disliked the lack of opportunity to ask questions or to pursue an area of interest in more depth (23/57) and several (10/57) commented that they disliked spending a long time looking at the computer screen. In contrast, negative comments about the tutor-lead teaching formats tended to focus on the physical environment (room too warm, uncomfortable chair) with very few comments about the teaching itself.
When asked after the session to list the specific items they could recall from the teaching, most participants from all groups noted points relating to spirometry (123/137), and the classification of lung disorders and the causes of breathlessness (104/137). A significant number also recalled information given on the epidemiology of lung disease (60/137)
In this study, we demonstrated a small, but statistically significant increase in level of knowledge, and a substantial significant improvement in data interpretation skills after all three teaching methods. An improvement in spirometry interpretation was particularly marked in the group experiencing web-based teaching and underlay the overall improvement in data interpretation skills in all three groups. We sought to minimise the risk of students confounding the results by private study by assessing learning immediately after the session, and by assessing enhanced understanding in addition to knowledge transfer.
Using this study design, we demonstrated that despite interactive, face-to-face teaching being the preferred option of the majority of participants, short-term knowledge transfer and a marked improvement in data interpretation skills was equally achievable by all three methods of teaching. One noteworthy feature of the study was that the participants were students only four months away from their final examinations, who had high baseline knowledge prior to the teaching and thus, the margin for improvement in knowledge was small. Despite this, a significant improvement was detected in performance on taught items in all groups. Furthermore, there was no improvement in performance on untaught items, indicating that all three learning formats were of value in this situation. This finding suggests that computer-based teaching may be particularly valuable for senior students who need to consolidate and extend their knowledge in respiratory medicine; whether it is equally suitable for novices in this field needs to be established. We elected to study students at this time, because previous experience has shown them to be eager for additional teaching and thus, it was possible to recruit a substantial number of participants (40% of year group).
To our knowledge this is the first trial designed specifically to examine the effectiveness of a computer-based package at facilitating an improvement in respiratory data handling skills. In line with current findings, most studies designed to investigate educational questions have shown only modest differences or no difference between teaching formats in knowledge transfer [12–24] whilst those considering attitude tend to show a preference for tutor-directed teaching methods even when computer packages are highly rated. However, the ability to interpret lung function data is a crucial skill in respiratory medicine so we were particularly anxious to determine the efficacy of computer-based teaching for this purpose. The study shows that, in this respect, it is at least equivalent to other teaching formats and in this cohort, superior to face-to-face interactive teaching. Close analysis of the data demonstrated that the improvement in performance of the students receiving web-based teaching was largely due to their ability to interpret one particular lung function report illustrating extra-thoracic airway obstruction. This was portrayed along with other examples during both face to face teaching and in the web-based package. However, because this condition is uncommon, the lecturer devoted less time to it than to other, more common, conditions. Our observation suggests that students studying independently weight all the information given equivalently and thus, were more likely to recall a less common area. If this interpretation is correct, it demonstrates the importance of ensuring that materials prepared for private study by students should be focussed on core areas to avoid them being overwhelmed by interesting, but non-core material. We know that the students remained logged on to the e-learning materials for variable lengths of time and whilst some remaining logged on throughout the session, others logged in and out up to 11 times. We do not know, of course, whether they were actually working whilst logged in, or that they had stopped working because they had logged out – they may have been accessing other relevant materials, but the log does demonstrate that the students did personalise their use of the materials, which is one of the strengths of this teaching format. Of further interest is that benefit was gained from this method of teaching despite few students needing the full time period which we had allocated. Indeed many spent less time on the project than those taking part in the tutor lead sessions.
Student preference or perception of performance does not always correlate with objective measurements. In a British based study concerning the teaching of psychiatry Williams et al  found that although students rated their learning as greater in a lecture on panic and anxiety than from a computer package on the same subject, objective assessment showed equivalence of knowledge transfer. Student choice of learning format is likely to be determined by a number of factors including individual learning styles and previous experience. It is likely that many of the participants in our study had only limited experience of interactive, web-based educational methods, since these techniques were not widely used in our undergraduate course at the time of this study. Sub-group analysis of the preferences, showed that in the group that received the web-based teaching, participants were more likely to make electronic teaching their second choice (after interactive tutor-lead) compared with the other two groups. (Figure 6). This bodes well for the future suggesting that, as they become familiar with these, students find e-learning packages more acceptable. Indeed, one respondent who selected e-learning as his preferred format stated this explicitly – "I might not have chosen computers initially, but it was very good."
Advantages of web-based teaching are that it is self-paced and also permits the user to review topics at will – points that were made by a number of our students in their evaluations. However, students studying independently are less likely to receive the benefit of feedback . A partial solution to this problem may be to make modules available for a set time, inviting email questions arising from the module and posting anonymised questions and answers on a virtual notice board. This would permit the tutor to schedule time to answer questions whilst allowing reasonable immediacy of feedback to students. Frequently asked questions could then be used to make improvements to the module for future students. The use of a more interactive form of computer based learning has been shown to improve knowledge retention on neuroanatomy and neurophysiology compared to a computer package with more didactic content.
Whilst it can never replace the inspiration engendered by interaction with an enthusiastic expert teacher, web-based teaching offers the ability for all learners to study flexibly and with repetition as often as they wish. An ideal outcome would be the combining of both formats, so web-based teaching is embedded within the curriculum, for example, by using a face-to-face session to de-brief students following a web-based activity. Previous (unpublished) experience at our institution has shown that offering e-learning opportunities without firmly embedding them in some way, results in very poor use of web-based resources. This study adds further information to reassure us that, at least for teaching senior students, such methods can be as effective as existing techniques and can be used to teach data interpretation skills, as well as knowledge transfer. Whether this is equally true for novice learners remains to be established.
What is known about this area
Most randomised controlled trials of different learning methods show no difference in knowledge transfer between study groups, but, despite this, students often express a preference for tutor-directed learning formats.
What this study adds
A computer-based package is at least as effective as tutor-directed methods for enhancing students' data interpretation skills relating to the diagnosis of respiratory disease.
Suggestions for future research
Future studies should investigate the students' approach to self-directed, computer based learning and establish how they decide to apportion their time and attention between the plethora of material available to them. This would help teachers prepare better materials for student use.
Mr H Brenton provided specialist expertise in mounting the educational material on the Web CT platform. We acknowledge with thanks the receipt of a grant from the European Respiratory Society which enabled us to undertake this study.
- Lloyd T, Philips B, Aber R: Factors that influence doctors' participation in clinical research. Medical Education. 2004, 38: 848-851. 10.1111/j.1365-2929.2004.01895.x.View ArticleGoogle Scholar
- Sheridan DJ: Reversing the decline of academic medicine in Europe. The Lancet. 367: 1698-1701. 10.1016/S0140-6736(06)68739-4.
- Goldacre M, Stear S, Richards R, Sidebottom E: Junior doctors' views about careers in academic medicine. Medical Education. 1999, 33: 318-326. 10.1046/j.1365-2923.1999.00404.x.View ArticleGoogle Scholar
- Fauzi A: Knowledge and practice of medical doctors on chronic obstructive pulmonary disease: a preliminary survey from a state hospital. Med J Malaysia. 2003, 58 (2): 205-212.Google Scholar
- Alvarez Luque I, Flor Escriche X, Rodriguez Mas M, Gallego Alvarez L, Fraga Martinez M, Sanchez Pinacho L, Juvanteny Gorgals J: Do we forget asthma as a chronic illness in our primary care consultations?. Aten Primaria. 2004, 33: 381-386. 10.1157/13060756.View ArticleGoogle Scholar
- Zaas D, Wise R, Wiener C: Airway Obstruction Is Common but Unsuspected in Patients Admitted to a General Medicine Service. Chest. 2004, 125: 106-111. 10.1378/chest.125.1.106.View ArticleGoogle Scholar
- Wickstrom GC, Kolar MM, Keyserling TC, Kelley DK, Xie SX, Bognar BA, Lewis CL, DuPre CT: Confidence of Graduating Internal Medicine Residents to Perform Ambulatory Procedures. Journal of General Internal Medicine. 2000, 15: 361-365. 10.1046/j.1525-1497.2000.04118.x.View ArticleGoogle Scholar
- Smith SF, Brenton H, Roberts NJ, Partridge MR: How do we teach respiratory medicine to undergraduates most effectively. Proceedings of the American Thoracic Society. 2005, 2: A909.Google Scholar
- Yawn B, Yawn R: Spirometry testing education in medical schools: a missed opportunity?. Prim Care Respir J. 2005, 14 (1): 21-24. 10.1016/j.pcrj.2004.10.003.View ArticleGoogle Scholar
- Higgins J, Green S: Assessment of study quality; Cochrane Handbook for Systematic Reviews of Interventions, 6.3 Selection bias updated May 2005; Section 6. 2007, (accessed 24 September 2007), [http://www.cochrane.org/resources/handbook/hbook.htm]Google Scholar
- Bryman A: Qualitative data analysis in Social Research Methods. 2001, Oxford University Press, 387-403.Google Scholar
- Norman GR, Schmidt HG: Effectiveness of problem-based learning curricula: theory, practice and paper darts. Med Educ. 2000, 34: 721-728. 10.1046/j.1365-2923.2000.00749.x.View ArticleGoogle Scholar
- White M, Michaud G, Pachev G, Lirenman D, Kolenc A, Fitzgerald JM: Randomized trial of problem-based versus didactic seminars for disseminating evidence-based guidelines on asthma management to primary care physicians. J Contin Educ Health Prof. 2004, 24: 237-243. 10.1002/chp.1340240407.View ArticleGoogle Scholar
- Bradley P, Oterholt C, Herrin J, Nordheim L, Bjorndal A: Comparison of directed and self-directed learning in evidence-based medicine: a randomised controlled trial. Med Educ. 2005, 39: 1027-1035. 10.1111/j.1365-2929.2005.02268.x.View ArticleGoogle Scholar
- Krishna S, Francisco BD, Balas EA, Konig P, Graff GR, Madsen RW: Internet-enabled interactive multimedia asthma education program: a randomized trial. Pediatrics. 2003, 111: 503-510. 10.1542/peds.111.3.503.View ArticleGoogle Scholar
- McDonough M, Marks IM: Teaching medical students exposure therapy for phobia/panic – randomized, controlled comparison of face-to-face tutorial in small groups vs. solo computer instruction. Med Educ. 2002, 36: 412-417. 10.1046/j.1365-2923.2002.01210.x.View ArticleGoogle Scholar
- Mukohara K, Schwartz MD: Electronic delivery of research summaries for academic generalist doctors: a randomised trial of an educational intervention. Med Educ. 2005, 39: 402-409. 10.1111/j.1365-2929.2005.02109.x.View ArticleGoogle Scholar
- Schol S, Goedhuys J, Notten T, Betz W: Individualised training to improve teaching competence of general practitioner trainers: a randomised controlled trial. Med Educ. 2005, 39: 991-998. 10.1111/j.1365-2929.2005.02263.x.View ArticleGoogle Scholar
- Spickard A, Smithers J, Cordray D, Gigante J, Wofford JL: A randomised trial of an online lecture with and without audio. Med Educ. 2004, 38: 787-790. 10.1111/j.1365-2929.2004.01824.x.View ArticleGoogle Scholar
- Greenhalgh T: Computer assisted learning in undergraduate medical education. BMJ. 2001, 322: 40-44. 10.1136/bmj.322.7277.40.View ArticleGoogle Scholar
- Williams C, Aubin S, Harkin P, Cottrell D: A randomized, controlled, single-blind trial of teaching provided by a computer-based multimedia package versus lecture. Med Educ. 2001, 35: 847-854. 10.1046/j.1365-2923.2001.00960.x.View ArticleGoogle Scholar
- Maiburg BH, Rethans JJ, Schuwirth LW, Mathus-Vliegen LM, Van Ree JW: Controlled trial of effect of computer-based nutrition course on knowledge and practice of general practitioner trainees. Am J Clin Nutr. 2003, 77: 1019S-1024S.Google Scholar
- Seabra D, Srougi M, Baptista R, Nesrallah LJ, Ortiz V, Sigulem D: Computer aided learning versus standard lecture for undergraduate education in urology. J Urol. 2004, 171: 1220-1222. 10.1097/01.ju.0000114303.17198.37.View ArticleGoogle Scholar
- Steele DJ, Johnson Palensky JE, Lynch TG, Lacy NL, Duffy SW: Learning preferences, computer attitudes, and student evaluation of computerised instruction. Med Educ. 2002, 36: 225-232. 10.1046/j.1365-2923.2002.01141.x.View ArticleGoogle Scholar
- Brown S, Knight P: Assessing learners in higher education. 2006, London: Kogan Page Limited, 1994.Google Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/7/41/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.