Student pharmacists studying in 2019 and 2020, who were in the final year of a four-year Bachelor of Pharmacy (BPharm) or a two-year Master of Pharmacy (MPharm) at The University of Sydney (USYD), as well as student pharmacists in the last 3 years of a six-year Doctor of Pharmacy (PharmD) from Washington State University (WSU) participated in this study. At USYD Pharmacy School, MHFA training is a compulsory component of the curriculum for both undergraduate and postgraduate student pharmacists who are enrolled in the final year of their degree. At WSU, MHFA training is offered as part of a two-credit mental health elective course. Following MHFA training, student pharmacists from Australia were assessed through SPRPs with mental health consumer educators recruited from One Door Mental Health . Similarly, student pharmacists enrolled in the WSU MHFA elective were assessed through SPRPs with consumers and healthcare providers from the National Alliance on Mental Illness (NAMI) and Frontier Behavioural Health [27, 28]. Student pharmacists were randomly allocated to an SPRP assessment, whereby they had to provide MHFA to a mental health consumer/provider enacting one of three cases (Supplementary Material 1–3) as a SP. In both Australia and the US, each randomly selected student pharmacist had their SPRP observed by other fellow student pharmacists. Care was taken to ensure that the role-playing student pharmacist was not allocated to role-play a scenario that they had previously observed.
The SPRP assessments in both countries were audio-recorded. For each case, a case-specific grading rubric was marked out of a total score of 20 by a tutor to assess each student participant’s performance during the SPRPs. The same grading rubric was then given to the role-playing student pharmacist and the SP (consumer) immediately after the simulation, so they could also use it to self-assess and assess, respectively, the student’s performance in the SPRP. Hence, each student pharmacist was marked by three raters (student pharmacist self-assessment, tutor and consumer). While the three rater types (self-assessment, tutor and consumer) were consistent across all role-plays, different individuals contributed in each of these roles.
Student pharmacists were provided with an information sheet which explained the study conduct and purpose, and highlighted the voluntary nature of participation. It is important to note that these activities formed a compulsory part of curricula; however, student pharmacists could volunteer to consent to allow the research team to analyse their scores and audio-recorded assessments.
This study adopted a convergent parallel mixed method design. Between August 2020 to November 2020, data analysis was simultaneously and independently conducted once all student pharmacists’ scores and SPRP audio recordings were available from Australia and the US.
Cases and rubrics
The development of the rubrics was guided by the MHFA Action Plan – ALGEE – along with a scoring system developed by MHFA researchers, and the rubrics have been used to assess student pharmacists participating in SPRPs post-MHFA training, supporting both face and content validity [14, 21, 29, 30]. The rubric has also been modified based on test re-test and interrater reliability analyses, to support its reliability across time and markers . A score ranging from 0 to 2 was assigned to each item out of 10 items within the rubric (0 points = criteria not met, 1 point = some appropriate actions were demonstrated, 2 points = all appropriate actions were applied). To pass the case, student pharmacists needed to receive a score of at least 10 out of 20 and perform all the required actions.
The ‘required actions’ for each scenario are highlighted in grey within the respective grading rubrics, as can be seen in Supplementary Materials 1, 2 and 3. The required actions were all derived from concepts and skills taught during the MHFA training course in tandem with the MHFA manual and were agreed upon by the research team, all of whom were either MHFA instructors or MHFAiders . As the SP admits to experiencing suicidal thoughts in the context of case 1, student pharmacists needed to ask the SP directly about suicidal thoughts, ensure that the SP was safe and not left alone as well as connect the SP with appropriate professional help, in order to pass the case (Supplementary Material 1). For case 2, student pharmacists were also required to assess the risk of suicide directly; however, the SP explains that they are not experiencing suicidal thoughts. Hence, a broader range of supports and professional help recommendations were appropriate for case 2, as the SP was not experiencing a mental health crisis (Supplementary Material 2). For case 3, student pharmacists were required to demonstrate that they ensured the SP received immediate professional help for an acute episode of mania (Supplementary Material 3). Hence, in all three cases, student pharmacists had to ensure the SP’s safety to pass.
Analysis of SPRP scores across raters and countries
Quantitative analyses were conducted using IBM SPSS Statistics 27® (IBM Corp, Armonk, NY). Data used in the quantitative analyses was extracted from each student pharmacists’ rubric scores collected from each of the three raters. Additionally, pass/fail rates derived from the rubric scores of the tutor were used to compare student pharmacist performance between Australia and the US. A one-way analysis of variance (ANOVA) was used to determine the difference in means between the total rater scores by type of rater (student pharmacist, tutor and SP), independent of the case and country. Total rater score means from both countries were also compared using independent samples t-test to identify differences between Australian and US raters, independent of the case. Chi-squared tests were performed to compare pass/fail rates based on tutors’ scores from Australia and the US across cases 1–3. For all analyses, statistical significance was achieved when p ≤ 0.05.
Each audio recording for cases 1 and 2 was transcribed verbatim for qualitative analysis as they were the cases that consisted of a mandatory suicide assessment criterion required to pass the SPRP assessment. A discourse analysis was performed to explore the language and terminology employed by student pharmacists when assessing for suicide. A discourse analysis is an analytical approach that involves deconstructing and critiquing language use, including its social context . This approach was selected as it provided a critical outlook on the nature and form of the language and terminology used by each student pharmacist. Six discursive frames were developed a priori by four authors (WN, SE, RM and CO), adapted from MHFA guidelines as well as previous literature exploring how mental healthcare professionals communicate with patients about suicidal ideation [14, 22, 33]. During the suicide assessment, the student pharmacists’ dialogue was coded according to the following discursive frames: ‘confident’ or ‘timid’, ‘empathetic’ or ‘apathetic’, and ‘direct’ or ‘indirect’.
Student pharmacist dialogue was coded as ‘confident’ if they maintained composure and tone when speaking to the SP. On the other hand, student pharmacist dialogue was coded as ‘timid’ if they spoke with a shaky voice in conjunction with any delays in finishing words, prolonged hesitation, profuse stuttering, long pauses and having multiple disfluencies (e.g., “um…”, “like…”). Any long pauses or delays in speech which occurred during suicide assessment were transcribed and presented as an ellipsis (i.e., “…”). The ‘empathetic’ frame reflected whether student pharmacists displayed a genuine sense of care for the SP while they assessed for suicide. The statements used immediately before, during and after the suicide assessment were used to determine whether student pharmacist dialogue was coded as either ‘empathetic’ or ‘apathetic’. Language considered to be ‘direct’ during suicide assessment was characterised by referring to suicide explicitly as well as using recommended direct terminology, as per MHFA guidance, for example, using terms such as ‘suicide’, ‘killing yourself’ or ‘suicidal thoughts’ . Student pharmacists were coded as using an ‘indirect’ approach to assessing for suicide if they used ambiguous language and terminology that is either obscure, vague and/or convoluted. For example, questions that incorporated words such as ‘harming yourself’, ‘injuring yourself’, ‘negative thoughts’ or ‘dark thoughts’ demonstrated an ‘indirect’ suicide assessment. If a student pharmacist combined direct and indirect terminology (e.g., “Are you thinking of harming yourself or suicide?”) within the same sentence, the dialogue was coded as ‘indirect’. Student pharmacists were considered ‘direct’ if the questions were asked separately (e.g., “Are you thinking about self-harm?”, “Are you thinking about suicide?”) and enough time was allocated for a response from the SP in between each question, in that it was evident that they were asking about thoughts of both non-suicidal and suicidal self-injury.
A decision to not include a ‘neutral’ option as a discursive frame was made by three authors (WN, SE and RM) as discussion between the authors made it possible to reach consensus and code the data within the dichotomous frames. Despite having been introduced as a frame within some previous discourse analysis methodologies, a ‘neutral’ frame is not always used in studies specifically exploring the language used in suicide assessment [22, 33, 34]. Moreover, a ‘neutral’ frame was considered to have been inappropriate if applied to the discursive frames within this study. For example, with regard to the ‘direct’ or ‘indirect’ frames, what constitutes direct assessment of suicide is clearly articulated in the MHFA manual .
One coder (WN) listened to each recording in full and extracted key data relating to the timing of the suicide assessment, the specific terminology used in relation to suicide, as well as coded each suicide assessment as per the aforementioned discursive frames. A second coder (RM) then listened to the specific excerpt of the audio-recordings when the suicide assessment took place and independently coded each student pharmacist’s suicide assessment using the six discursive frames, blinded to the coding of the first coder (WN). To determine the level of inter-rater agreement between the two coders, Cohen’s Kappa values were calculated using IBM SPSS Statistics 28.0. A comparison of the coders’ individual assessments was conducted for each student pharmacist participant, whereby any discrepancies in coding were replayed in a meeting with a third co-author (SE) and discussed until agreement by consensus was reached by three co-authors (WN, RM and SE).
Cross-country comparison of discourse analysis
A chi-squared test was conducted to determine whether there were any significant differences between participants from Australia and the US regarding the proportion of suicide assessments coded across the discursive frames. Independent samples t-tests were performed to determine whether there were any significant differences in the duration of the SPRPs and the timing when suicide assessment occurred, between Australian and US data.
Considerations relating to authors’ reflexivity
WN, SE and RM led the analyses and made the core decisions regarding the discursive frames. SE is an accredited MHFA instructor, registered pharmacist and educator. RM is MHFA-trained and has had over 2 decades of experience in community pharmacy practice and as an academic educator. SE and RM were part of the team that led the development of the first simulation assessments, and they have both taken part in these assessments since. Both SE and RM have played the role of the tutor as well as the simulated patient, in previous assessments. Their contribution was influenced by their knowledge of MHFA guidance, as well as their experience of community pharmacy practice and of assessing student pharmacists during simulated patient assessments. At the time of data analysis, WN was a MHFA-trained final year student pharmacist undertaking his honours research project. He had first-hand experience with supporting consumers living with mental illness in his role as a student working in community pharmacy.