Case-oriented computer-based-training in radiology: concept, implementation and evaluation
© Dugas et al; licensee BioMed Central Ltd. 2001
Received: 30 July 2001
Accepted: 19 October 2001
Published: 19 October 2001
Providing high-quality clinical cases is important for teaching radiology. We developed, implemented and evaluated a program for a university hospital to support this task.
The system was built with Intranet technology and connected to the Picture Archiving and Communications System (PACS). It contains cases for every user group from students to attendants and is structured according to the ACR-code (American College of Radiology) . Each department member was given an individual account, could gather his teaching cases and put the completed cases into the common database.
During 18 months 583 cases containing 4136 images involving all radiological techniques were compiled and 350 cases put into the common case repository. Workflow integration as well as individual interest influenced the personal efforts to participate but an increasing number of cases and minor modifications of the program improved user acceptance continuously. 101 students went through an evaluation which showed a high level of acceptance and a special interest in elaborate documentation.
Electronic access to reference cases for all department members anytime anywhere is feasible. Critical success factors are workflow integration, reliability, efficient retrieval strategies and incentives for case authoring.
Access to radiological expert knowledge for a broad spectrum of users – from students to senior radiologists – is a continuous challenge in a routine clinical setting of a university hospital. A balance between established and new teaching methods is needed to provide high quality, peer-reviewed content.
The knowledge transfer normally takes place during teaching sessions in small groups, lectures or in the context of a specific case during daily routine by personal interaction e.g. between attendant physicians and residents, house officers and students. For this reason all members of the radiologic department are involved in a general teaching file.
In this context we developed a new software tool to collect and document relevant cases – according to all different levels of radiologic knowledge – from daily routine in an integrated manner by including all types of image sources. Access to this Intranet-based Teaching File – the "Lehrarchiv" – should be provided from any workstation within the department.
The design of this teaching file should – in the long run – integrate all department members both as case authors and normal users. At the same time a high, homogeneous quality level should be guaranteed from the beginning.
By a differentiated authorization system two opposite prerequisites had to be fulfilled: On the one hand the necessary technical infrastructure to provide access to the system should be ensured, i.e. access to the Intranet should be possible anytime anywhere within the department. On the other hand clinical data with patient information and images must be protected against unauthorized use.
Another critical issue concerning the infrastructure is acquisition and archiving of image data, i.e. the pathway of the image from its generation to the teaching file. All radiological techniques should be included, especially CT, MR, conventional X-ray, angiography and ultrasound. In this context our objective was to answer the following questions:
Is a teaching file with access anytime anywhere within the radiologic department and smooth integration into routine workflow feasible from a technical point of view?
What format and data structure is appropriate for a radiologic case description?
What kind of retrieval functions are required?
Is an electronic radiologic teaching file accepted by students and physicians?
Materials and Methods
Secondary digitization of a primary analogous image e.g. a conventional radiographic film and subsequent electronic transfer transfer to the teaching file. This method is rather time-consuming and implies loss of image quality due to digitization, but is typically required when external radiographs or slides are included; the additional effort must be weighed against the relevance for the case presentation on an individual basis.
Transfer of a digital image from an external data source into the teaching file. This may be a digital image primarily stored in an electronic archive or digitized for other reasons.
Direct transfer of primary digitally generated, recent patient images (CT, MR etc.) into the teaching file.
All three methods have been implemented for our teaching file. The problems associated with different media are very common at present and characterize the current change from classical film-based to computerized radiology.
The direct digital transfer is – with respect to workflow integration and image quality – obviously the most attractive method and is the preferred way in the context of the establishment of PACS. This method becomes even more relevant, because the number of images per case is increasing continuously due to sophisticated radiological techniques. For this reason manual transfer of single images becomes more and more impractical.
The technical implementation was realized with a DICOM (Digital Imaging and Communications in Medicine)  interface – an international multi-vendor standard -, which enables the transfer of selected images to the teaching file in a very simplistic manner, similar to printing images. In order to preserve confidentiality the patient's identification is eliminated when a case is published.
An important success factor for a radiologic teaching file consists in efficient retrieval mechanisms, especially when searching for similar cases. Similarity is referring both to localization and pathology of a radiological finding. For this reason the internationally established ACR-Code  was applied, which allows a precise classification of cases according to these dimensions.
The teaching file system acts basically as a DICOM receiver, i.e. the case repository behaves like a printing device.
By means of a web frontend and individual user accounts each author can access and edit his own cases on the teaching file server. Authorized users can release cases for the public case repository.
We applied an iterative software design approach with rapid prototyping, i.e. we built prototypes, tested and continuously refined them; altogether about 10 iteration cycles were required. The data structure consists of two tables (cases and images) with altogether 23 items.
Hard- and Software
The technical concept is based on established Internet tools. An Apache  Webserver (version 1.4.2) on a Linux  machine (Distribution SuSE 6.3) provides PERL (version 5.005) programs accessing a relational database . The department network consists of a Gigabit-Backbone and NT-workstations with 100 MBit connections.
To quantify user acceptance we applied a paper based survey. Categorized items were recorded on a six point Lickert scale, from 1=absolutely correct to 6=completely wrong. Scores are given as mean +/- standard deviation.
After a program development phase of approximately one year and 18 months of clinical routine use there are now approx. 350 released cases accessible (283 are work in progress); 31 users from our radiologic department are registered, but only eight of them contributed to more than 80% of cases.
The radiologist can select relevant images on a DICOM workstation during his routine work and send these to the teaching file through the DICOM-Send procedure in analogy to printing. At a later time he can connect to his teaching file account and edit the case. In addition he can upload secondary digitized images.
The textual description (free text) consists of four sections:
Diagnosis, which is mandatory,
medical history / clinical presentation / laboratory,
radiologic findings and
An efficient retrieval strategy is required to find "similar" cases with both high precision and recall. For this reason a systematic code is necessary. We developed a specific coding tool to enter the ACR-code in both dimensions (localization / pathology) for each case. If several findings are present, up to three ACR code-pairs can be stored. Due to the hierarchical design of the ACR code (e.g. localization Gastrointestinal System – Stomach – Pylorus) similar cases – both in terms of localization or pathology – can be found easily. In addition for each case administrative information (e.g. author ID, date of release) is stored automatically.
In addition to the retrieval function users can select specific case series which are aggregated by authorized radiologists, typically on a specific teaching issue (e.g. selected lung cases). This provides added value for the teaching file: it can be used not only for retrieval of single cases, but also for presentations e.g. in scientific workshops. Individual teaching sessions on specific topics, e.g. for students, are also provided with this tool.
After 18 months there were 583 records in the table 'Cases' consisting of 19 items each, and 4136 records in the table 'Images' with 4 items each.
The system was evaluated by 101 medical students (3rd year) who studied two case series (conventional thorax and liver CT/MR) in small groups (max. 9 persons each, assisted by an instructor) during their mandatory radiological course. We included all students from a semester to avoid a selection bias with respect to individual computer expertise. The learners themselves judged their computer knowledge as average (2,92 +/-1,06).
We also asked the students for the optimal number of images to be contained in a "typical case": 10 +/- 5 (mean +/- S.D.). This is influenced by the selection of cases with a focus on CT and MR – probably less images were appropriate for conventional chest X-rays.
To improve the system the students suggested to apply interactive pointers or similar mechanisms to highlight specific radiological findings. From a technical point of view this can be implemented easily, but the authors argued this would increase the amount of authoring work substantially.
The results from the questionnaire are consistent with the subjective impression gained during the teaching sessions.
We also did an evaluation with residents (n = 15) on a case series consisting of more complicated cases selected by a senior attendant radiologist. There was again a high level of acceptance (comprehensible cases 1,47 +/- 0,74; recommendation of the system to colleagues 1,60 +/- 0,83).
Computer-Based-Training (CBT) in Medicine has proven its clinical potential in several settings (,,,). Compared with the situation 10 years ago , technological problems such as insufficient computer performance, lack of storage capacity or inadequate display devices can now be solved.
Despite these advances electronic peer-reviewed and up-to-date teaching files in radiology which can be accessed anywhere in the department are still very rare. Recently, reports of successful CBT in the field of radiology have been published (,,,), but typically the radiological content of the systems is very limited (both in terms of cases and in terms of authors).
The first objective of this study concerns technical feasibility of a teaching file with access anytime anywhere within the radiologic department and smooth integration into routine workflow. From our experience – after 18 months of routine operation – an Intranet-based system with PACS-interface (i.e. DICOM-compatible) can fulfill this task; it is very important that images can be sent directly from the radiological workstation to the case repository.
There is a wealth of CD-ROMs and other resources (,,) providing 'snapshots' of radiological knowledge, but a collection of routine clinical reference cases for a specific department and its particular devices is needed. Due to the sophistication of imaging methods a clinically relevant teaching file must be updated continuously. This is only feasible by minimizing the effort to create and maintain this database.
The second objective of this study concerns the ideal format and data structure for a radiologic case description. Obviously from an academic point of view a detailed description of the case using controlled vocabularies, including access to a complete electronic patient record would be desirable. In our setting this was not feasible because it is too time-intensive for the authors. Therefore we decided to apply a compact approach with free text data for medical history, radiological findings, diagnosis and comment. Only location and pathology were entered in a coded manner, using the internationally established ACR code.
The third objective of this study addresses retrieval functions, which obviously depend on the data available in the system. From our experience both systematic and free text retrieval mechanisms are important. We applied the ACR code, which is characterized by a high level of granularity, to provide a systematic access to the database. By this means 'similar' cases – concerning both localization or pathology – can be found easily. We also considered to apply a controlled vocabulary for diagnosis, but in our setting the effort for systematic coding would have been too high.
Case series arranged by senior radiologists are another method to organize the case collection and facilitate access.
The fourth objective concerns acceptance of the system by students and physicians. We did not measure the learning success of students, but the questionnaire indicates a high level of acceptance which is in line with observations during the course. The constant growth of peer-reviewed cases over time provides good evidence for acceptance by the physicians.
From our experience, integration into daily routine work is the key success factor for a teaching file. Secondary digitization of images is laborious and implies a loss of quality. Media conversions are such an obstacle that only in a filmless setting with PACS and smooth integration by DICOM-interfaces relevant cases can be collected during daily routine.
To maintain a high level of quality in patient care is a major challenge in the context of labour turnover and continous sophistication of radiological methods. The high degree of specialization makes knowledge transfer an important success factor for the department as a whole. There are many recent publications according to the relevance of knowledge management (,) within organizations.
A technical system like our teaching file is a small, but important building block of a department-wide knowledge management strategy, because it provides access to relevant teaching material anytime anywhere. Our evaluation provides evidence that electronic teaching files are well accepted if quality of content and technical reliability is assured. From a medical informatics point of view, such case repositories offer the opportunity of building decision support systems in the future. Whilst a PACS archive is a huge collection of images, a teaching file is a knowledge database. Due to the systematic coding, an automatic search for similar cases is possible. Intelligent image analysis programs in the past mostly failed because a sufficient knowledge base could not be gathered. Meanwhile sophisticated methods for knowledge discovery and data mining (,) have been developed, but these techniques require both high-volume and high-quality reference datasets.
By national and international cooperation the vision of peer-reviewed, up-to-date and comprehensive teaching files in radiology could become reality.
Mr. Heilmann wrote important computer programs for the prototype system.
- NEMA's OFFICIAL DICOM WEB Page [resource on World Wide Web]. 2001, [http://medical.nema.org/dicom.html]
- The American College of Radiology [resource on World Wide Web]. 2000, [http://www.acr.org/]
- Extensible Markup Language (XML) [resource on World Wide Web]. 2001, [http://www.w3.org/XML]
- The Apache Software Foundation [resource on World Wide Web]. 2001, [http://www.apache.org]
- SuSE Linux [resource on World Wide Web]. 2000, [http://www.suse.de]
- Wall L, Schwartz RL: Programming PERL. O'Reilly & Associates, Sebastopol, CA, USA,. 1992Google Scholar
- PostgreSQL [resource on World Wide Web]. 2001, [http://www.postgresql.org]
- Evans RS, Pestotnik SL, Classen DC, et al: A Computer-Assisted Management Program for Antibiotics and Other Antiinfective Agents. N EngI J Med. 1998, 338: 232-238. 10.1056/NEJM199801223380406.View ArticleGoogle Scholar
- Schwid HA, Rooke GA, Ross BK, Sivarajan M: Use of a computerized advanced cardiac life support simulator improves retention of advanced cardiac life support guidelines better than a textbook review. Crit Care Med. 1999, 27: 821-824. 10.1097/00003246-199904000-00045.View ArticleGoogle Scholar
- Dugas M, Batschkus MM, Lyon HC: Mr. Lewis on the web – how to convert learning resources for intranet-technology. Med Educ. 1999, 33: 42-46. 10.1046/j.1365-2923.1999.00267.x.View ArticleGoogle Scholar
- Lyon HC, et al: PlanAlyzer, an Interactive Computer-assisted Program to Teach Clinical Problem Solving in Diagnosing Anemia and Coronary Artery Disease. Acad Med. 1992, 67: 821-828.View ArticleGoogle Scholar
- Klar R, Bayer U: Computer-assisted Teaching and Learning in Medicine. Int J Biomed Comput. 1990, 26: 7-27.View ArticleGoogle Scholar
- Hornof WJ, allance DW, Brenston PR, Self JA: A client server model to facilitate creation of a medical image teaching library. J Digit Imaging. 1999, 12: 132-137.View ArticleGoogle Scholar
- Zaidel M, Hopper K, lyriboz T: Interactive web-based radiology teaching file. Digit Imaging. 1999, 12;2 Suppl 1: 203-204.View ArticleGoogle Scholar
- Khorasani R, Lester JM, Davis SD, et al: Web-based digital radiology teaching file: facilitating case input at time of interpretation. AJR. 1998, 170: 1165-1167.View ArticleGoogle Scholar
- Tran TH, Roach NA, O'Kane PL, Thune M: Creating a digital radiographic teaching file and database using a PC and common software. AJR. 2000, 175: 325-327.View ArticleGoogle Scholar
- Chew FS, Smirniotopoulos JG: Educational Efficacy of Computer-Assisted Instruction with Interactive Videodisc in Radiology. In: IMIA Yearbook 1994 (ed. v. Bemmel), Schattauer Verlag, Stuttgart, Germany,. 1994, 468-474.Google Scholar
- Eurorad [resource on World Wide Web]. 2001, [http://www.eurorad.org/]
- Richardson ML: World-Wide Web Radiology Teaching File Server on the Internet. AJR. 1995, 164: 479-483.View ArticleGoogle Scholar
- WWW Virtual Library on Knowledge Management [resource on World Wide Web]. 2000, [http://www.brint.com/km/]
- Malhotra Y: Knowledge Management and Virtual Organizations. Idea Group Publishing, Hershey, PA, USA. 2000Google Scholar
- Lavrac N: Selected techniques for data mining in medicine. Artif Intell Med. 1999, 16: 3-23. 10.1016/S0933-3657(98)00062-1.View ArticleGoogle Scholar
- Morik K, Imboff M, Brockhausen P, Joachims T, Gather U: Knowledge discovery and knowledge validation in intensive care. Artif Intell Med. 2000, 19: 225-249. 10.1016/S0933-3657(00)00047-6.View ArticleGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6920/1/5/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.