Conceived and created by Benjamin Williams, Ph.D. in Academic Year 1999–00, the University of Illinois College of Medicine at Urbana-Champaign has carefully and methodically developed a computer based testing software package originally designed to test images in the histology course for first year medical students. Since 1999 histology students have taken their final examinations on-line using the software developed by Dr. Williams. During the past two years, a significant effort to expand the software has resulted in a flexible, powerful and secure program that completely manages all aspects of assessment in the first year medical curriculum in Urbana. The on-line examinations were designed taking into consideration best practice as outlined in the literature [2, 7].
Further, all students become very well acquainted with the computerized format long before taking examinations through the use of on-line practice examinations and on-line grade reporting. Thus, giving all students, and especially females, a level of comfort and familiarity with the on-line format long before an on-line examination is administered.
During Academic Year 2004–05 faculty in biochemistry and neuroscience elected to have their final examinations administered in the new on-line format. The purpose of this study was to determine if students were placed at a disadvantage when taking on-line examinations in lieu of paper-and-pencil examinations and to determine if gender played a role in performance.
Analysis revealed that no difference in overall performance related to gender was found.
No difference in overall class performance was found, closer examination revealed no difference in performance when comparing males and females, regardless of the test format and regardless of discipline. Both the multi-factor ANOVA and the specific t-Tests, performed within each discipline revealed no performance difference between males and females in either one-tail or two-tail distributions.
Neuroscience is a single-semester course delivered during the Spring Semester during the M-1 curriculum. There are two examinations, one in March and one in May. During Academic Year 2004–05, the May examination was administered in an on-line format. When comparisons were made over three academic years of overall student performance, no performance differences were found. However, there were interesting differences between the classes' performance on both the paper-and-pencil comparisons and the paper-and-pencil comparisons to on-line format. Students performed better with the on-line administration of the exam. It should be noted that increases in performance are typically demonstrated on the final examination in this course, so it is not surprising to find a difference in performance as most students attempt to pass this course.
Biochemistry is a semester and one-half course delivered from August until March during the M-1 curriculum. There are four examinations, three during the Fall Semester and one during the Spring Semester. During the Spring Semester the format changes from a traditional lecture format to small group application of biochemistry principles to patient cases. The fall examinations are administered in traditional paper-and-pencil format. The March examination, which is a combination of short-answer and multiple choice questions switched from paper-and-pencil to on-line format during Academic Year 2004–05. ANOVA comparisons of student performance during Academic Years 02, 03 and 04 revealed significant differences in overall performance in the course. ANOVA comparisons on the on-line examination also revealed differences in performance. Similar to the neuroscience performance, it is difficult to draw the conclusion that performance differences between the paper-and-pencil compared to on-line are a result of change in format. These changes could be due to differences in grading of the short answer questions and the change from hand-written responses to word-processed responses.
With regard to gender performance differences in biochemistry, analysis revealed none.
The lack of performance differences in medical students relative to gender is could be explained by the characteristic profile of this population. Regardless of gender, medical students are hardworking, well-informed and technology-capable. Perhaps the general population of females, as opposed to the medical student population of females, is at a greater risk when it comes to utilization of technology in an assessment setting. Further studies designed to address this issue should be made at all levels of the educational/training experience.