Pretest-intervention-posttest designs are uniquely appropriate for investigating the effects of educational innovations and are commonly used in educational research [8–10]. Strict experimental designs suggest the use of a two-group pretest-intervention-posttest design with a control group that receives no training intervention and a group that receives the training. In the occupational training environment required to meet regulatory compliance requirements, it is often not possible to allow differential services (i.e. different levels of training) for the staff. This was the case for radiation safety training at our cancer center where it was decided not to have a control group because the withholding of training from the control group would represent a differential service inconsistent with a worker's 'right to know' about the potential hazards. In this case, a one-group (no control) pretest-intervention-posttest experimental design was utilized. The absence of a control group was not considered a significant threat to the internal validity of the experiment because the likelihood that extraneous factors account for the change was small. It was assumed that in the absence of the intervention (i.e. the specific radiation safety training programs) there were minimal or nonexistent outside variables that would have significantly changed a nurse's cognitive knowledge or attitude with regard to radiation safety regulations and policies over the pretest to posttest interval timeframes. It was concluded that the use of an experimental design without a control group was justified.
The study was carried out in 2004 for inpatient staff registered nurses working with oncology patients at MSKCC, a National Cancer Institute designated comprehensive cancer center in New York City.
All inpatient staff registered nurses (750 nurses) were considered as the study population. Participants were recruited by nurse educators and nurse leaders. A total of 15% of the registered nurses (i.e. 113 nurses) completed the study pretests and posttests. All nurses received the same training information.
The intervention consisted of a multifaceted set of improvements (Figure 1) including: nursing procedure revisions, a core concepts video, two types of inservice training, and enhanced "radiation precaution" signs and labels.
Radiation related nursing procedures were revised to maintain a consistent format and were edited to include only information that was considered to be essential information for nursing care situations. The procedures were validated by nursing and radiation safety experts for both scope and clarity.
A twelve-minute digital video was developed for incorporation into nursing orientation training and annual mandatory training for all nurses. Video topics were selected specifically to address the perceived knowledge gaps as well as to directly address fears of radiation. The video cast included a narrator, a professional male actor chosen by the nurse members of the multidisciplinary team, and representatives of the nursing and radiation safety staff community depicting various radiation safety precaution actions and discussions. An effort was made to include as many staff members as possible to heighten interest in the movie, help validate the material, and instill a sense of ownership. The video was digitized to facilitate presentation during training workshops as well as making the video available on the MSKCC intranet. Hyperlinks from the nursing procedures website allowed nurses to review the video at any time from virtually anywhere in the hospital.
Two types of inservice training sessions were held. The first was an interactive, hands-on workshop developed specifically for nursing leadership (managers, clinical nurse specialists, nurse practitioners, and nurse educators) with the goal of increasing their ability to problem solve in the area of radiation and radioactive precautions. The second was a didactic session for nursing staff on units that regularly housed patients requiring radioactive precautions. This inservice focused on the various radiation treatments and the associated precautions most likely to be experienced by nurses.
Highly visible "radiation precaution" door signs and chart labels were developed to specifically detail the required actions by nursing staff for maintaining the safety of patients and hospital staff while providing care for patients. The improved signs and labels better differentiated among the precautions required for permanent implants, temporary implants, and radiopharmaceutical therapy than the signs previously used.
Two instruments were designed for this evaluation. The first was designed to measure cognitive knowledge of radiation and radiation protection practices. The second was designed to measure attitudes of nurses with regard to radiation.
The present research relied upon a criteria-based multiple-choice/true-false test of cognitive knowledge [11–16]. The same instrument was utilized for both pretest and posttest evaluations. The cognitive test included 15 questions with 4 choices for each question. Face validity for the knowledge instrument was assessed by a local group radiation safety specialists and nursing leaders. Each question was scrutinized to ensure that it represented an accurate measure of desired parameters. Content validity was assessed through the use of radiation safety subject matter experts to ensure that proper topical coverage had been afforded by the overall test. [See additional file 1].
The cognitive questions addressed the following areas of knowledge: background radiation dose, annual limit, when to wear a badge, how to find exposure records, declared pregnant limit, radiosensitivity of the fetus, external beam treatment, seed implant treatment, visitor precautions, temporary implant treatment, permanent implant treatment, systemic radioiodine precautions, systemic radioiodine contamination, monitoring patient dose rate, and contamination cleanup protocols.
A Likert-scaled attitude evaluation included 9 questions. Face validity was assessed by a local group of radiation safety specialists and nursing leaders. [See additional file 1]. The instrument addressed the following attitudinal areas: I feel that radiation safety policies are clear, I know whom to contact for information, I know what steps to take, I can explain precautions well, I feel safe, policies are based on regulations, I am monitored, there is oversight, and I feel safe to have a child.
Data were captured from the response sheets using the ReMark Office OMR software. Two levels of data analysis were performed on the aggregate of pretest and posttest responses. The first used descriptive statistics and the second employed hypothesis evaluation statistics. The significance alpha level was chosen for all statistical tests to be 0.05, the most typical value for social research[6, 18].