Participants
Volunteers for the study were recruited through an announcement in Indian biomedical journals and via email invitations to individuals engaged in biomedical research, and leaders of medical schools and major institutions conducting health research in India. The eligibility criteria for inclusion were: (i) a degree in medicine or a masters' degree in science, (ii) receipt of a graduate or postgraduate degree within the last ten years, (iii) at least one-year of experience in human health-related clinical or social science research, (iv) basic computer skills and availability of broadband internet access, (v) willingness to be randomized and to participate in the study, and (vi) willingness to undertake pre- and post-course evaluations. Interested persons were invited to register at a website, and respond to questions relevant to eligibility. Four investigators (two in India [RA, AA] and two in the US [PM, RB]) reviewed the applications.
Randomization and Study Procedures
The study used a randomized design. Following informed consent, each study participant was allocated to one of two study arms, using a computer-based randomization procedure. Participants in Arm 1 traveled to Lucknow, India for a 3.5-day classroom training in Biostatistics, and a week later participated from their own home or office settings in a 3.5-week on-line training course in Research Ethics (Figure 1). Those in Arm 2 received a 3.5-week on-line training in Biostatistics and then traveled to Lucknow for a 3.5-day on-site training in Research Ethics. Courses were provided at no cost to participating students, and all travel, accommodation, and meal costs for participation in the on-site courses were provided. Participants were not otherwise compensated financially or in kind for loss of time. Before attending the Biostatistics and Research Ethics courses, each participant completed a short course about on-line learning methods; this also helped confirm that they had adequate computer hardware and internet bandwidth required for effective participation in on-line courses.
Course Descriptions
Content for all courses was developed using established principles of curriculum development [14]. Learning objectives, course materials and course lecture slides were identical for the on-line and on-site formats of both the Biostatistics and Research Ethics courses. For the on-line courses, students viewed slide presentations while listening to lectures pre-recorded by faculty experts. Adobe Presenter software program was used to produce the synchronized lecture and slide presentation. Students were provided with readings to accompany or complement each lecture, as well as copies of all lecture slides (PDF files for on-line course; printouts for on-site course). Both the Biostatistics and Research Ethics on-line and on-site courses included structured live group activities and case discussions, in addition to the formal lectures. On-line students also had the option of posting questions through a web portal, to facilitate discussion with fellow students and course faculty. On-site courses included time for questions and discussion during and after lectures.
The Biostatistics course was developed by JM, based on an introductory level course he developed for the Department of Biostatistics at Johns Hopkins Bloomberg School of Public Health, with input on learning objectives, course content and knowledge assessments contributed by faculty experts from India (RA, NG and AA). The Biostatistics course included 17 lectures over 15.5 hours, that covered statistical analysis and study design, including types of data, descriptive statistics, normal distribution, sampling distribution, central limit theorem, confidence intervals, hypothesis testing for continuous and categorical data, simple linear regression, measures of association, survival analysis, and design of observational and interventional studies. Students also participated in 8 interactive group exercise sessions of 45-60 minutes each designed to apply knowledge gained during the lectures. On-line students were provided the same exercises to discuss and solve on-line with other students and the course faculty, during 8 interactive sessions. All lectures for the on-line format were pre-recorded by JM. He was unable to travel to India for the on-site course, and the three Indian faculty experts in biostatistics (RA, NG and AA) delivered lectures, using the same slides as those used in the on-line course. They interacted via email with JM and repeatedly listened to his on-line lectures, in order to deliver the on-site live lectures consistent with JM's on-line lectures. The 8 interactive sessions for both the on-line and on-site formats were conducted by two Indian faculty experts (RA, NG).
The Research Ethics course was developed by faculty experts at the Johns Hopkins Berman Institute of Bioethics (NK and HT), with input on learning objectives, course content and knowledge assessments contributed by a faculty expert from India (AB). The three Research Ethics faculty delivered 15 lectures with 8.75 hours of instruction, covering ethical principles, a framework for ethical analysis, informed consent, the relationship of study design to ethics, risk/benefit assessment, the role of ethics committees, privacy/confidentiality, and honesty in science. In addition, students participated in 5 one-hour interactive case discussions, requiring application of ethical analysis skills. Students also viewed and discussed one 20-minute video on ethical challenges in community-based research. Course participants completed two homework assignments. The same faculty experts from the US and India provided the lectures and cases discussions for both the on-line and on-site formats of the Research Ethics courses.
The interactive sessions for online courses were conducted at pre-specified times and lasted about 60 minutes each. Each session was moderated by one or more course faculty. The course participants were encouraged to log in into these sessions over the Internet, though attendance was not compulsory. During these online 'classrooms', faculty could use audio to address student questions asked via a typed message that appeared on the faculty members computer screen as well as the screen of all other students. The faculty could also share his/her computer screen, on which s/he could write and draw. Further, the moderator could give 'audio' rights to any student, allowing him/her to speak to the entire class.
For the statistics online course, eight online sessions were so arranged that each topic was covered in two consecutive sessions to allow a participant who could not join a particular session to 'attend' the other; each participant was therefore expected to attend four sessions. The participants were encouraged to send to moderators any questions that they wanted discussed. The faculty member reviewed the question and answers for these with the students.
The research ethics faculty held five case-based discussions with the students in the on-line course. Three sessions were held for each case-based discussion, each moderated by a member of the faculty. Students were given the case and a set of questions to answer in advance of the session. The faculty member would review the question and answers with the students and then moderate a discussion on the case.
Outcome Measures
For each training course (on-site or on-line, Biostatistics or Research Ethics), all participants were administered knowledge tests before the course, as well as immediately and three months after course completion. These tests focused on assessment of participants' knowledge related to the specific learning objectives and application of this knowledge to problems raised in case histories. Responses were scored against pre-determined answers, and an overall knowledge score (percent correct responses) was computed for each test. Gains in knowledge scores in each domain (Biostatistics or Research Ethics) were compared between the participants receiving on-site and on-line training. The tests of knowledge were the same for on-line and the on-site platforms in each content domain. The biostatistics knowledge tests consisted of 20 questions in objective format (single response, true/false, or providing answers based on simple calculations). The research ethics knowledge assessments given just before and after the course were unique sets of 41 multiple choice, true/false and short answer questions. The three month follow-up knowledge assessment consisted of the 41 best performing questions from the pre and immediate post-course assessments.
The gain in knowledge at 3 months after completion of each course was used as the primary outcome measure. In addition, at the completion of each course, each study participant was administered a course evaluation questionnaire to assess student acceptability and satisfaction with the course. This assessment utilized a 5-point Likert scale for most items; for a few questions, 4-point and 3-point Likert scales were used.
Ethical Review
The study was approved by ethics committees at both the participating institutions (Institutional Ethics Committee of the Sanjay Gandhi Postgraduate Institute of Medical Sciences, Lucknow, India and Johns Hopkins Medical Institutions Institutional Review Board, Baltimore, Maryland, USA). Each study participant signed a written informed consent prior to randomization, and submitted it by fax, as scanned computer file by email or through postal mail.
Statistical Analysis
Wilcoxon's rank sum test and Wilcoxon's signed rank test were used for inter-group and paired comparisons, respectively. Multivariate quartile regression analyses were used to determine independent predictors of gain in knowledge scores at 3 months following course completion, in each domain. Covariates, including age, number of years since last degree, and mode of training (on-site versus online), were tested in univariate analyses as predictors of gain in knowledge scores in biostatistics and in research ethics separately. Those found significant at p < 0.05 level were included in a multivariable analysis. Data on acceptability of and satisfaction with each course were compared using a trend test for ordinal data. In addition, the effect of demographic factors on the relationship between gain in knowledge at 3 months from baseline and the training platform (on-line or on-site) was examined using a quartile regression analysis; factors found significant on univariate analysis were entered into a multivariate analysis. Data on acceptability of and satisfaction with each course and mode of instruction were compared using a chi-squared test for trend for ordinal data.