Intervention
The PRONTO intervention has been described in detail elsewhere [19,20,21,22]. In brief, the PRONTO training is based on clinical cases, interactive exercises, and communication practices, using an interprofessional approach. The training has minimal didactic content; most teaching occurs through interactive team-building exercises, targeted skills sessions, highly realistic simulations of obstetric and neonatal emergencies, and video-guided debriefings immediately following each scenario. Childbirth and postpartum care simulations, both with and without complications, are conducted. Training sessions are led by a team of nurse midwives, nurses and physicians including at least one PRONTO master trainer and 3–4 local team members that have completed a PRONTO train-the-trainer course. Training occurs in a real work environment (trainees are mixed groups comprised of physicians, nurses and midwives), using resources that are usually available at medical units, providing a highly realistic environment. Simulations are conducted in the emergency area and in labor, delivery, operation, and recovery rooms. PartoPants™, which are hybrid birth simulators made by modifying recycled surgical scrubs, are worn by a participant who plays the role of the patient during an obstetric emergency. A Laerdal NeoNatalie© simulator is used to practice neonatal resuscitation. Participants assume roles according to their profession and work area, therefore rendering a more realistic training experience. The simulations are video recorded for subsequent feedback involving the whole group, which favors constructive learning and the detection and avoidance of potential mistakes [20].
The intervention comprises two modules: Module I (MI: 16 h, conducted over two consecutive days – 8 h each day) is focused on obstetric hemorrhage (OH), neonatal resuscitation (NR), team work, and communication skills. Module II (MII, 8 h, conducted over 1 day) is scheduled 3 months after MI and reinforces the topics in MI, as well as addressing preeclampsia/eclampsia (PE) and shoulder dystocia (ShD) [20]. All training sessions were implemented from 9 AM to 5 PM during weekdays, even for afternoon, night and weekend shift workers.
Participants
The State Ministries of Health and Institutes for Women requested the training to be implemented in 14 Ministry of Health-run hospitals in the Mexican states of Guerrero, Morelos, Oaxaca, Puebla, Quintana Roo, and Veracruz, which provide care for mostly low socioeconomic level population without access to social security. Teaching coordinators or hospital directors were advised by the trainers to invite personnel from different disciplines who worked in childbirth and neonatal care or who were otherwise involved in obstetric emergency care, although each hospital had the final call on who was to attend. Most participants were obstetrician-gynecologists (OBGYN), pediatricians, general practitioners, interns, nurses, or professional midwives. An oral consent letter was read to the potential participants at the beginning of the training. All providers in attendance were offered the option to receive the training regardless of their willingness to participate in the before-after evaluation (and hence the study).
Sample size. A total of 351 participants gave consent to participating in the study and completed pre and post questionnaires, while 55 persons only completed pre-questionnaires and were excluded. According to data from our study, assuming a before-after correlation of 0.56 a pre-treatment mean of 50 and a common standard deviation of 15 points, the minimum detectable effect size with the n = 351 sample size and a power of 80% was 2.1 percentage points in the case of knowledge. In the case of self-efficacy, assuming a before-after correlation 0.5, a pre-treatment mean of 80 and a standard deviation of 11 pre-test and 17 post-test, the minimum detectable effect size with a power of 80% was 2.24 percentage points.
Study variables
Outcomes: Knowledge & Self-efficacy.
Participants completed pre- and post-training questionnaires immediately before and after each module, evaluating knowledge of evidence-based practices in identifying, preventing, and managing obstetric and neonatal emergencies as well as participant confidence in his/her own ability to perform key skills (self-efficacy). The questionnaires were a revised version of those used in the 2010 PRONTO pilot [19]. The self-efficacy scales were based on the model developed by Bandura [23]. Self-efficacy is defined as the sense of security each person experiences in relation to his/her ability to perform the necessary actions during emergencies. Following the standard methodology for measuring self-efficacy, individuals were presented with items portraying different levels of task demands, and they rated the strength of their belief in their ability to execute the needed procedures. In total, the evaluation instrument contained 26 knowledge questions and 27 questions on self-efficacy in five categories: neonatal resuscitation, obstetric hemorrhage, general obstetric emergency, shoulder dystocia, and preeclampsia/eclampsia.
In the case of the knowledge questions, participants’ responses were coded as correct or incorrect, then we obtained a knowledge score (both total and for each category) consisting on the percentage of correct answers by each particular individual in the sample. In the case of self-efficacy items, the participants rated themselves on a scale of 0–100 in which 0 means complete lack of confidence and 100 means total confidence; in this case we defined the self-efficacy score as the arithmetic mean of the participant’s answers, both total and by category.
Covariates
Shift. This variable included information on the work shift of each particular participant, classified as morning, afternoon, night or weekend/holidays. In Mexico, health workers are typically assigned to only one shift, and, although it is theoretically possible to change to a different one, it is difficult to do so and workers tend to remain working in that particular shift indefinitely. Of course, this does not apply to interns, who are at the hospital at all times; in this case we arbitrarily defined them as belonging to the morning shift as in Mexico each intern spends all morning shifts in the hospital, and only covers afternoon, night and holiday/weekend shifts every three to four days (we could say interns work predominantly in the morning). Profession. Participants in the training were classified according to their profession as general practitioners, obstetricians, pediatricians, medical interns, other medical specialists, nurses, obstetric nurses and midwives.
We use the term “nurses” to refer to general nurses; nurse students were included in this category; the term “obstetric nurses” is used to refer to nurses who are certified in obstetric care. We use the term “midwives” to refer to staff that have completed either technical training or a bachelor’s degree in midwifery. As only one obstetric nurse was trained, we decided to reclassify her in the same category as midwives.
Other covariates. Additional covariates included in the model were self-reported gender and age of the participant.
Statistical analyses
Analyses were conducted in three phases. First, baseline participants’ characteristics were described in terms of means and proportions. Secondly, we compared the participants’ characteristics to those who only completed baseline questionnaires but eventually dropped out of the training using a logistic regression model with exclusion of the final sample as the outcome and gender, age, work shift and profession as covariates. Lastly, we modeled knowledge and self-efficacy as a function of participant characteristics and the PRONTO training, both overall and by training topic.
To identify factors related to knowledge and self-efficacy and estimate the before-after changes in outcomes, a set of longitudinal linear regression models with mixed effects (random effect at the individual level) was fitted to the data; the model was used both to estimate the influence of covariates on baseline knowledge and self-efficacy and to model changes in the outcomes after the training using a complete-case approach. All models considered the clustered structure of participants by hospital by including a fixed effects term of the hospital [24]. Models of knowledge and self-efficacy were fitted for each training topic, as well as the whole course average. In all cases, the outcome variable was the knowledge or self-efficacy score obtained (in percentage points), and the main independent variable was a dummy indicating time (i.e., before or after the intervention). We evaluated the effect of gender, age (categorized as < 30, 30–49, and > 50 years), shift during which the participants worked (morning, afternoon, evening, and weekend/holidays), and profession (nurses, midwives, interns, general practitioners, pediatricians, OBGYN, and other medical specialties). The self-efficacy models included the knowledge score in the same topic as a covariate. Interaction terms for time (dummy variable) with gender, age, shift, and profession were included to evaluate possible heterogeneities of effect. The analyses were performed using Stata version 13.0 (StataCorp LP, College Station, TX, USA).