Aims

This study aimed to examine the impact of mental illness related stigma training on third year medical students' knowledge, attitudes and behaviour related to people with mental illness.

Study design

We hypothesised that there would be greater, more favourable change in the knowledge, attitudes and behaviour of the experimental conditions (EC1 and EC2) combined, compared to the control condition (CC). Furthermore, a greater change would be detected in the intervention condition that had the factual content, personal testimonies and role-play training compared to the condition with just the factual content and personal testimonies alone.

Sample

Students were recruited before the beginning of their third year of medicine, which began with a two week introductory course. All students were given the same standard training during this two-week course by the medical school. Material relating to psychiatry and mental illness during this course was minimal and consisted of a brief introduction without any contact from service users or caregivers. Third year medical students were chosen because at this particular medical school their psychiatry rotation/clerkship took place in the third year hence we did not want them to have had clinical contact with patients which could confound the results.

Students were recruited by e-mail. Each student was sent an email introducing the study and what it would entail if they chose to participate. A brief presentation was also given to students at the start of the two-week course telling them about the study. Students were told that their participation in the study was entirely voluntary and their grades would not be affected in any way if they chose not to participate. Students who chose to participate were told that their responses would be kept anonymous and would not affect their grades or course work.

At the end of the academic year, data pertaining to the Psychiatry assessment grades of participants and non-participants were obtained from the registrar of the University to which the medical school belonged. Global scores on the medical school Psychiatry assessments of the participants and non-participants were used to estimate the representativeness of the sample.

Inclusion/exclusion criteria

Students were included in the trial if they were registered for their third year of medicine at the medical school and they: (i) had a valid e-mail address that was on the list provided by the registrar to the researcher; (ii) were attending the two week introductory session. Participants were recruited two weeks prior to the intervention. The recruitment period was short because the introductory timetable and list of registered students was not finalised until the start of the two week introductory period. Students began their psychiatry rotation immediately after the two week introductory period.

Allocation

Allocation into the trial conditions was done by an administrator at the medical school blind to the proposed intervention and independent from the research team. Each trial condition consisted of clusters of firms which were put together pseudo-randomly by the office of the registrar. This meant that student groups of six to ten students were placed together for learning purposes within the medical school curriculum and remained together throughout the year. The basis for allocation was whether the clusters of firms had space in their timetable at the same time when the experimental condition could take place due to logistical issues at the medical school. The nature of allocation of students by the medical school administration, allowing for the preferences of individual students, was not compatible with randomisation.

For the evaluation component of knowledge and attitudes, students were contacted by e-mail. For the evaluation component of behaviour, a role-play assessment of communication skills was allocated by the administrator to a subset from each of the conditions (CC, EC1, EC2) to be examined when they had spaces in their timetable.

Training intervention

The training intervention was developed separately from the evaluation team and was developed by a mental health charity. The interventions were designed to address the knowledge, attitudes and behaviour model with the factual component targeting knowledge, personal testimonies from service users and caregivers targeting attitudes and role-play training targeting behaviour. The intervention in EC1 consisted of a 1-hour time slot including: (i) a 15 minute factual component presentation from the mental health promotion officer of the charity on the social impacts of stigma and discrimination against people with mental illness including themes such as healthcare, employment, civil society participation, personal relationships, medication, media and aggressive behaviour; (ii) a 15 minute personal testimony from a mental health service user who was taking medication for schizoaffective disorder describing their personal perspectives and experiences of having a mental illness, stigma and discrimination; and (iii) a 15 minute personal testimony from a caregiver of a person with schizophrenia discussing their personal perspectives and experiences of caring for a person with a mental illness, stigma and discrimination; (iv) a 10 minute question and answer session between the students and the mental service user and caregiver; (v) a 5 minute allotment to complete a short satisfaction questionnaire about the entire lecture (factual component and personal testimonies).

The additional intervention for EC2 was also developed by the charity. This consisted of two 10-minute role-play scenarios (a parent and daughter seeking help for their daughter's mental health problems and a service user who had a physical health complaint). Role-players were recruited from the role-player company used by the medical school and were instructed about the scenario by the researcher. Students were divided into groups of 8 to 10 and systematically allocated to different classrooms. A facilitator and either a service user or caregiver observer were present among the students. A student was asked to volunteer for the first role-play scenario and the role-players were asked to enter the room by the facilitator. After the role-play, the medical students gave their feedback on how they felt they performed and what they found challenging. This was followed by the role-players, observing service user or caregiver and fellow students sharing their feedback regarding the communication skills used by the medical student with the role-player(s) in the scenario. This process was repeated for the second role-play with a different student volunteer. A satisfaction questionnaire was completed after the role-play training.

Assessments

At baseline the students were asked a number of demographic factors, social contact of someone with mental illness and intended area of specialisation after medical school and had their knowledge and attitudes towards mental illness evaluated. Students were asked to complete the assessments online two weeks before the intervention (baseline) and were sent reminder e-mails from the researcher. On the day of, but prior to, the intervention, students also had the opportunity of completing the instruments in hardcopy. One week after all of the interventions and role-play assessments were completed, students were reassessed using baseline measures, either online or through hard copy. Primary knowledge and attitude outcome data were obtained from the Knowledge Quiz and the Mental Illness: Clinicians' Attitudes (MICA) scale [31]. The secondary outcome measure was a role-play assessment carried out by the role-players of which there were no baseline data.

Measures

Statistical analysis

Analyses were carried out using only data from students who completed instruments at both baseline and post-intervention. Non-completer data were compared to completers to assess bias. Baseline scores were compared between the arms of the trial, and an analyses of covariance (ANCOVA) controlling for baseline scores, gender, and psychiatry assessment grades (from the medical school at the end of the academic year) was conducted to estimate differences across the three conditions regarding knowledge and attitudes. For behaviour, only a post intervention assessment was available. Analysis was carried out using regression in STATA version 10. Residuals from the regressions were examined for normality. Two pre-specified contrasts were tested: 1) to compare the two interventions combined with the control condition, and 2) to compare the two intervention arms with each other.

Ethics

This project was approved by the Local Research Ethics Committee of Bexley and Greenwich (06/Q0707/56) for the National Health Service in the UK.

Knowledge

There were no significant differences in total scores on the Knowledge Quiz at baseline among the control condition, experimental condition 1 and experimental condition 2 (p = 0.668). At follow up there was a significant favourable effect of training on knowledge about people with mental illness and stigma with an adjusted mean difference of 1.19 (95%CI 0.63-1.76) (comparing the two interventions combined with control), but no evidence for a difference between the two different types of intervention).

Regarding knowledge, the Knowledge Quiz was developed solely for the purpose of this study; previous data had not been collected in order to compute a power calculation. A post-hoc power calculation however shows that our study had satisfactory power as we had 80% power to detect a 15.7% change in Knowledge Quiz scores using a two-tailed test in a sample size of 34 students at an alpha level of 0.01.

Attitudes

There was no significant difference between the three trial arms for attitudes at baseline (p = 0.307). There was weak evidence (p = 0.085) that any intervention condition led to less stigmatising scores than the control condition see Table 3. This was despite achieving adequate samples in the CC and EC1 conditions.

Behaviour

In total, 68 students consented to take part in the role-play assessments, 22 of whom were from CC, 21 of whom were from EC1 and 25 of whom were from EC2. The means (sd) for the global role play scores for CC, EC1 and EC2 were 3.14 (0.77) 3.52 (0.87) 3.52 (0.77) respectively. The contrast of the combined experimental conditions (EC1 and EC2) with the CC was significant at a borderline significance level, at p = 0.069, but there was no evidence for a difference between the two intervention conditions.

There were no differences in overall scores when comparing the component scores of the service user scenario with the caregiver scenario or when comparing the global scores of the service user scenario with the caregiver scenario.

Overall satisfaction with training

Methodological limitations

The study was not randomised as students were allocated to firms by the medical school Registrar and this determined whether or not they would be given the intervention since certain clusters of firms had space in their two week introductory session timetable whereas others did not. There were two stages at which selection bais may have occurred. First our participants showed a gender bias in that female students were over-represented. Second, the proportion followed up was also low. While we controlled for some factors associated with loss to follow up, we were not able to characterise these completely.

A number of factors may have contributed to the low follow up rate. We were allowed only one hour for the lecture intervention with personal perspectives and one half hour for the role-play training intervention by the medical school so the instruments were not administered within these session times. Students were e-mailed requesting them to complete the instruments at follow-up which may have also contributed to the modest follow-up rate and lack of long-term follow-up.

Second, some medical students are keener to learn than others and view any sort of training and assessment as valuable to their career as a future doctor. Additionally some medical students may be more interested in the topic of mental illness related stigma. Given that psychiatry and mental illness are often stigmatised, the topic may have not been interesting to other students thereby leading to their non-participation in the follow-up or study overall.

One might surmise that our lack of results may have been because our participants were already more favourably inclined in terms of attitudes and the intervention more likely to have an effect on those who did not participate. Students knew that the training to be evaluated was part of a research project and that they would not be examined on the material of the training. If the training material and the evaluation were formally embedded in the curriculum this may have led to more participants and possibly more observed effectiveness. Much higher rates have been observed in other settings where this was the case [24].

Information about student enrolment as well as student contact information was provided by the registrar therefore there may have been discrepancies in the actual number of students who were eligible as many students still registered during the two week introductory session and not all students may have informed the registrar if they did not attend the two-week introductory session which was recommended but not compulsory. We were conservative in our response rate estimate since the denominator may have been smaller than that used.

It must be noted that our results may be biased given the students were not a representative sample. Participants were different than non-participants which could have led to biased results. For example, we may have not seen a change in attitudes because participants were already inclined to have positive attitudes. Further, the change in knowledge may have been detected because participants were interested in the material. It is also important to note that the results found may also be due to the intervention being very short in duration as the lecture was one hour and the role-play training was 30 minutes while for medical students, a longer intervention may be required however this would need to be determined in a larger, representative sample.

Effectiveness of training

Despite the relatively small sample sizes and limited time for intervention, change was detected in knowledge in medical students who received the experimental conditions. We can be confident that the training was successful in leading to greater knowledge amongst these students than those in the control condition. Changes in attitudes were promising and warrant further investigation but not strong enough to be conclusively demonstrated. There was no evidence that behaviour changed.

This was the first trial with two experimental conditions to examine knowledge, attitudes and behaviour after mental illness related stigma training in medical students. Previous studies have either looked at changes in attitudes after psychiatric clerkships or before or after mental illness related stigma training without a behavioural component [25–27].