- Research article
- Open Access
- Open Peer Review
Effectiveness of an interprofessional patient safety team-based learning simulation experience on healthcare professional trainees
BMC Medical Education volume 18, Article number: 192 (2018)
Although the American Council of Graduate Medical Education (ACGME) mandates formal education in patient safety, there is a lack of standardized educational practice on how to conduct patient safety training. Traditionally, patient safety is taught utilizing instructional strategies that promote passive learning such as self-directed online learning modules or didactic lectures that result in suboptimal learning and satisfaction.
During the summer of 2015, 76 trainees consisting of internal medicine interns and senior-level nursing students participated in an interactive patient safety workshop that used a flipped classroom approach integrating team based learning (TBL) and interprofessional simulated application exercises.
Workshop trainees demonstrated an increase in knowledge specifically related to patient safety core concepts on the Team Readiness Assurance Test (TRAT) compared to the Individual Readiness Assurance Test (IRAT) (p = 0.001). Completion rates on the simulation application exercises checklists were high except for a few critical action items such as hand-washing, identifying barriers to care, and making efforts to clarify code status with patient. The Readiness for Interprofessional Learning Scale (RIPLS) subscale scores for Teamwork and Collaboration and Professional Identity were higher on the post-workshop survey compared to the pre-workshop survey, however only the difference in the Positive Professional Identity subscale was statistically significant (p = 0.03). A majority (90%) of the trainees either agreed that the safety concepts they learned would likely improve the quality of care they provide to future patients.
This novel approach to safety training expanded teaching outside of the classroom and integrated simulation and engagement in error reduction strategies. Next steps include direct observation of trainees in the clinical setting for team-based competency when it comes to patient safety and recognition of system errors.
Preventable medical errors are the third leading cause of death in the U.S., claiming the lives of over 400,000 patients each year . These numbers underscore the need for patient safety training and education for healthcare professionals. Although the principles and concepts of patient safety are required as a critical part of healthcare education and training, many healthcare educators are uncertain how best to integrate patient safety training into their programs and curricula . Patient safety training and education of healthcare professionals have not kept pace with advances in workforce requirements regarding patient safety .
At Stony Brook University the academic health science schools and hospital share common elements of their missions: to educate and provide excellent quality clinical care. Although there have been successful efforts to promote interprofessional learning and practice, there are still too few opportunities to systematically bring our faculty and trainees from multiple disciplines together in a meaningful way for the purpose of improving the quality of patient care by learning about each other’s roles, cultivating communication, practicing collaboration and fostering teamwork. To address this need, we designed and developed an interprofessional, collaborative, team-based learning simulation experience with the goal of promoting safe and quality patient care among healthcare professional trainees. While interprofessional simulation strategies are becoming more widely used [3,4,5], this training exercise demonstrates a unique way of teaching patient safety by integrating three fundamental instructional strategies: 1) team-based learning pedagogy, 2) interprofessional education (IPE) of trainees, and 3) simulation-based education (SBE).
Medicine has traditionally relied on an apprentice-style approach to learning and experience . This inevitably exposes patients to inexperienced healthcare practitioners, and the dangers and harm associated with this are increasingly unacceptable . Simulation is a technique to replace or amplify real-patient experiences with supervised and guided experiences, artificially contrived to evoke or replicate substantial aspects of the real world in a fully interactive manner . Furthermore, interprofessional communication and collaboration is crucial to patient safety training, especially when they enter clinical practice . According to the Interprofessional Education Collaborative (IPEC) , the ability to work with other health professionals to maintain a climate of mutual respect and shared values; understand one’s own role and those of other professions in order to appropriately address the healthcare needs of the patients; use a team approach to communicate with patients and their families; and perform a variety of team roles to deliver patient-centered care are the four core competencies for inteprofessional collaborative practice. Training future healthcare providers to work in interdisciplinary teams that help enhance these competencies should facilitate improved healthcare outcomes for patients . Hence, interprofessional education coupled with simulation application allows for the collaboration of expertise from different fields and the demonstration of these core competencies in safety education.
Team-based learning (TBL) is an evidence-based, multiphase pedagogical approach that requires active learner/trainee participation and collaboration . This instructional strategy is an ideal way to promote team science and collaborative efforts aimed to address a scientific challenge that leverages the strengths and expertise of professionals trained in different fields . TBL is a flipped classroom model that utilizes a readiness assurance process intended to hold learners accountable for coming to class prepared. Prior to class, the learners are provided materials to study based on the learning objectives of the class. The class begins with a readiness assurance process that consists of an Individual Readiness Assurance Test (IRAT) given to individual learners to assess key concepts that they are expected to acquire from the pre-class study materials. Immediately after the IRAT, the learners take the same test in teams (Team Readiness Assurance Test) by discussing amongst themselves and coming to a consensus on the answer to each question. After the readiness assurance process, the teams then work on application-based exercises that promote higher level learning and student engagement.
The objective of this study was to design, implement, and evaluate the effectiveness of a simulation training model that incorporates interprofessional learning and TBL for interdisciplinary faculty to teach core concepts of patient safety.
During the months of July and August 2015, we conducted five 3.5-h interprofessional patient safety-training workshops. These workshops were scheduled during residents’ protected didactic time. At each workshop, trainees (incoming internal medicine interns and senior-level nursing students) were randomly grouped into pre-determined interprofessional teams consisting of 4–6 trainees per team. The size of each team varied depending on the number of trainees available for the workshop. Each team had either one or two internal medicine residents and between three or four nursing students. Prior to this workshop the trainees had not worked together before and were not given the team assignments in advance. The teams remained together for the duration of the workshop, rotating through all of the clinical simulation scenarios as a unit (Fig. 1). By the end of the workshop, trainees were expected to be able to demonstrate the following learning objectives: 1) develop a root cause analysis and generate an action plan, 2) conduct a patient hand-off as the transmitter and receiver, and 3) demonstrate patient teach-back and safe discharge instructions .
Our study participants consisted of a convenient sample of 26 first year internal medicine residents and 50 undergraduate nursing students enrolled in an upper-division, accelerated one-year baccalaureate nursing program. According to the Joint Commission, communication failures have been identified as the top contributing factor to sentinel events reported . Therefore these two groups were chosen based on a call from the Joint Commission asking for increased collaborative training in physician-nurse communication. Within one year post-study, the nursing student participants will be eligible to sit for State licensure and subsequently be employed as Registered Nurses, where the need to communicate using succinct and effective language is critical and will directly impact on patient safety. The internal medicine residents have just started their intern year and therefore such training is essential for safe clinical practice. Prior to this workshop, these participants did not have any formal patient safety education.
Intervention and data collection
The pre-workshop study materials included readings of evidence-based literature and voice over powerpoints on topics regarding effective handoffs, safe discharge and systemic analyses of medical errors. In adherence with team-based learning pedagogy, an Individual Readiness Assurance Test (IRAT) was completed by each trainee, followed by an identical Team Readiness Assurance Test (TRAT) that was completed by each interprofessional team. The IRAT/TRAT consists of 10 questions, each question worth 1 point for a total possible score of 10 points. The content of the test is based on patient safety literature contained in pre-reading assignments given to the trainees one week prior to the workshop. A content expert reviewed the IRAT/TRAT for face and content validity. At the end of the TRAT, faculty instructors reviewed the correct and incorrect answers and provided feedback, rationale, and offered clarifications to the trainees. The purpose of the IRAT/TRAT was to hold the trainees accountable for learning the pre-reading materials and to demonstrate understanding of the concepts with their team members in preparation for the simulation case scenarios.
Following the readiness assurance process, however, rather than a TBL application-based exercise on paper, the interprofessional teams engaged in three simulations using high fidelity manikins and standardized patients designed to reinforce the workshop learning objectives through hands-on teamwork and communication. These simulation cases included: 1) systematic analysis of a medical error with the use of a high fidelity mannequin, 2) interprofessional I-PASS mnemonic handoff  with the use of a high fidelity mannequin, and 3) safe discharge using patient teach-back techniques with the use of a standardized patient (Table 1). Performance checklists were developed for each simulation case based on the learning objectives completed in the pre-reading materials, as well as components of TeamSTEPPS . After each simulation case the trainees participated in immediate post-scenario debriefing with interprofessional faculty instructors using a critical action checklist developed for each scenario. This provided the interprofessional team members with the opportunity to reflect upon and discuss their experiences and consider what they might do differently in the future. It also gave faculty instructors to the opportunity to comment on certain interactions they observed such as communication and interpersonal skills.
The simulation training program was evaluated using data collected from trainees on their knowledge, skills, and attitudes. Specifically, patient safety knowledge-based learning outcomes were evaluated using the IRAT and TRAT. The rationale for comparing IRAT and TRAT was to evaluate the contribution to learning from the interprofessional team experience and to provide corrective feedback to the team.
In terms of attitudinal data, the Readiness for Interprofessional Learning Scale (RIPLS), a validated 19-item questionnaire developed by Parsell and Bligh [13, 14], was administered immediately before and after the workshop. RIPLS uses a 5-point Likert scale to provide healthcare professional students the opportunity to self-rate from Strongly Agree (5) to Strongly Disagree (1) on statements regarding their knowledge, skills, and attitudes on working with other healthcare professionals. McFayden et al.  adopted the original version of the questionnaire to reflect four subscales: (a) Teamwork and Collaboration (9 items) that measures the knowledge and skills needed to participate in interprofessional learning, (b) Negative (3 items) and Positive Professional Identity (4 items) that measures perception of the values and benefits of interprofessional learning, and (c) Roles and Responsibilities (3 items) that measures perceptions of what can be performed in an interprofessional learning environment. Note that for Negative Professional Identity subscale, two statement items are negatively worded and therefore lower ratings reflect more favorable attitudes.
The trainees completed a post-workshop self-assessment survey that we developed that asked them to rate on a scale of 1 (Not Confident at All) to 5 (Very Confident) their confidence level in carrying out the concepts learned during the simulation workshops. Finally, completion rates of the performance checklist for each simulation case allowed trainees to apply and demonstrate hands-on skills of what they have learned from the pre-workshop study materials and then reinforced through the readiness assurance process.
Data were analyzed using IBM SPSS Statistics for Windows, Version 22.0 (Armonk, NY: IBM Corp). Descriptive statistics were used to calculate mean scores, standard deviations and percentages. As individual identifiers were not collected on the readiness assurance tests, an independent samples t-test was used to compare the overall mean IRAT scores with the overall mean TRAT scores to see if there was a difference between individual and team performances. Chi-square tests were conducted to compare the number of correct responses for each question item between the IRAT and TRAT. Aggregate pre-post responses for the Readiness for Interprofessional Learning Scale (RIPLS)  were analyzed using an independent samples t-test. A significant level of p < 0.05 was considered to be statistically significant.
A convenience sample of a total of 76 trainees (26 medicine interns and 50 senior-level nursing students) participated in the workshops, creating twenty interprofessional teams. All analyses were calculated based on data collected from the 76 workshop trainees.
The TRAT scores (Mean = 7.7, SD = 1.8) were significantly higher than the IRAT scores [Mean = 5.6, SD = 1.7; t(94) = − 4.9, p = .001]. A significantly higher number of correct responses were selected on the TRAT compared to the IRAT on questions related to these patient safety concepts: preventable adverse events, cognitive biases, and patient handoff communication .
Critical action checklists
Performance of each interprofessional team on the three simulation/standardized patient application case scenarios was assessed through direct observation by MD and RN faculty instructors using checklists that included critical action items. After each interprofessional team performance, the faculty instructors would immediately discuss what they observed and come to a 100% full consensus on which items were completed by the team. A majority of the critical action items on the checklists were completed by the interprofessional teams (see Table 2).
Although the RIPLS scores for the Teamwork and Collaboration and Professional Identity subscales (Negative and Positive combined) were higher on the post-workshop survey (Mean = 42.9, SD = 3.5; Mean = 22.8, SD = 2.7 subsequently) compared to the pre-workshop survey (Mean = 41.9, SD = 4.4; Mean = 22.0, SD = 2.6 subsequently), a statistical difference was seen only in the Positive Professional Identity subscale from before (Mean = 17.4, SD = 2.2) to after the workshop (Mean = 18.3, SD = 2.2; t (140) = − 2.2, p = 0.03) (Table 3).
In the post-workshop self-assessment survey, over 90% of the participants either ‘agreed’ or ‘strongly agreed’ to the statement “The safety concepts I learned will impact my clinical practice”. Additionally, a majority of the workshop participants agreed that they felt confident in “conducting error analysis and root cause analysis” (78%), “their ability to provide proper hand-off using IPASS” (86%), and “their ability to provide a safe discharge plan” (86%).
Learning about patient safety requires the understanding of relevant concepts and principles as well as the demonstration of teamwork, communication skills, and error reduction strategies. This study used simulation-based education that integrated two instructional strategies to teach patient safety: team-based learning and interprofessional, collaborative learning. Additionally, the simulation application is centered on commonly encountered medical errors. Simulation is a powerful and frequently used technique that can help healthcare professionals achieve higher levels of competence and safer care . Our study incorporates structured team-based pedagogy with interprofessional learning in order to emphasize realistic team communication, decision-making, judgment, and leadership skills necessary to handle medical errors .
Team performance on the TRAT for patient safety knowledge was higher than that of the individual performance on the IRAT. The team-based learning approach was used to ensure that trainees would begin the workshop with a basic understanding of patient safety concepts including systematic analyses of medical errors, effective handoffs, and patient education teach-back techniques. The simulation/standardized patient application case scenarios that followed required the trainees to access and apply patient safety, communication, and teamwork skills. One of the benefits of the TBL approach ensures that misunderstanding and/or lack of understanding of certain concepts at the individual level, is addressed in discussion and peer learning during the team readiness assurance process. Previous research also showed that learners perform better on the TRAT if they complete the IRAT first because learners would need to consider the questions on their own first and therefore feel inclined to contribute to the team discussion to share their thoughts as well as listen to the thoughts of others . This process facilitates acquisition of common and complete understanding of the important concepts among team members as they move forward with the application exercises. The TBL approach also encourages interprofessional communication and learning to occur; allowing different healthcare professionals the opportunity to bring their discipline-specific perspective to the team discussions so that as a team everyone is in agreement about how to best coordinate and provide patient care. Higher performance on the TRAT was to be expected in our study and supports findings from the literature that TBL is an effective strategy to promote interprofessional collaboration .
We found high completion rates for a majority of the critical safety actions across all interprofessional teams for each case scenario. There were a few action items that were not performed by at least half of the interprofessional teams such as hand washing (45% of interprofessional teams completed), clarifying code status with the patient (40% of interprofessional teams completed); and identifying barriers of care with the patient (50% of interprofessional teams completed). Additionally, only 55% of the interprofessional teams were able to provide appropriate illness severity and give a patient summary that is concise and relevant. This information can inform health professions educators as they design and develop training programs and curriculum. Hand washing, in particular, is a simple, yet critical, action item that should be continuously emphasized in all areas of simulated training programs, including patient safety curricula.
While increases in the RIPLS Teamwork and Collaboration and Professional Identity subscales were noted after workshop participation, the only significant difference occurred in the Positive Professional Identity subscale. According to the definition of each subscale and description of its psychometrics properties provided by Parsell and Bligh , these results suggest that after participating in the workshop, trainees’ perceptions of shared learning with other healthcare professionals positively increased, viewing interprofessional learning as useful for improving communication and problem solving skills . Additionally, although the Roles and Responsibilities subscale overall scores did not change (Mean = 9.8), the subscale score increased for the medicine interns (from 10.1 to 10.7) and decreased for the nursing students (from 9.6 to 9.3) after the workshop compared with before. Lower scores on the Roles and Responsibilities subscale imply a clearer perception of one’s role and those of others . While McFadyen  suggests that the internal consistency for this subscale is weak if it is used to examine undergraduate students (due to a lack of a clear understanding of what their roles and responsibilities are), it is interesting that the medicine interns, who are post graduate level trainees, were less clear of their role and responsibilities compared to the senior level nursing students. This may be because the interns were just beginning their training compared to the nursing students who were starting their senior year of training. Participation in the interprofessional simulation exercises may have reinforced perceptions of one’s contributions to the team, or reinforced the concept that delivery of safe patient care is the responsibility of the team as a whole, and not by individuals working independently. Trainees reported a very positive experience with the workshop and indicated that the patient safety content covered will likely impact the care they provide to patients in the future.
This study has several limitations that include a small sample size from each profession at a single institution and the use of a pre-posttest study design which does not assess the long-term effect of the intervention. Some caution is therefore advised in interpreting the results of this study as these results may not be generalizable to the general population in each profession. Some existing literature also questions the use of the RIPLS instrument and its sensitivity to detect meaningful change over time since researchers frequently find no significant difference between pre-and post IPE intervention . In our study, however, we did see statistically significant improvements for the Positive Professional Identity subscale after a brief 3.5 h workshop. Last, limitations to implementing this approach to patient safety training include accessibility of a simulation environment, standardized patients, and faculty availability in which replication at other institutions that do not have such resources may be challenging. Furthermore, the trainees’ ability to apply key principles of patient safety in actual clinical settings was not assessed in this study.
Next steps include repeat interprofessional, TBL simulations throughout the course of training, follow-up with trainees to obtain their perspectives on whether this activity has impacted patient care, and direct observation of residents in actual patient care settings including discharge and hand-off experiences. In this way, trainees can be further observed in actual clinical settings for team-based competency in patient safety and recognition of systems errors.
Future directions should include the development of patient safety direct observation tools to evaluate whether trainees are applying the learned simulation skills in their clinical practice. These tools can be used to measure skills during routine patient care on clinical units. Patient care events including error disclosure, multidisciplinary meetings, patient discharge meetings, and patient counseling about safety issues may be observed and evaluated.
Through interprofessional, TBL simulation, trainees actively participate, engage in, and demonstrate learning concepts related to patient safety. Implementation of such an approach is an important teaching strategy which can increase healthcare trainees’ interest and motivation to learn about key patient safety concepts. This training program will fulfill educational requirements and promote necessary skills among trainees to develop lifelong habits to effectively reduce medical errors. In this study we found that a simulation model in an interprofessional team setting can teach healthcare professionals realistic, hands-on principles of patient safety.
Individual Readiness Assurance Test
Readiness for Interprofessional Learning Scale
Team Readiness Assurance Test
Leotsakos A, Ardolino A, Cheung R, Zheng H, Barraclough B, Walton M. Educating future leaders in patient safety. J Multidiscip Healthc. 2014;7:381–8.
Aggarwal R, Mytton OT, Derbrew M, Hananel D, Heydenburg M, Issenberg B, et al. Training and simulation for patient safety. Qual Saf Health Care. 2010;19(Suppl 2):i34–43.
Bridges DR, Davidson RA, Odegard PS, Maki IV, Tomkowiak J. Interprofessional collaboration: three best practice models of interprofessional education. Med Educ online. 2011;16. https://doi.org/10.3402/meo.v16i0.6035.
Rider EA, Brashers V. Team-based learning: a strategy for interprofessional collaboration. Med Educ. 2006;40(5):486–7.
Palaganas JC, Brunette V, Winslow B. Prelicensure simulation-enhanced Interprofessional education: a critical review of the research literature. Simul Healthc. 2016;11(6):404–18.
Lateef F. Simulation-based learning: just like the real thing. J Emerg Trauma Shock. 2010;3(4):348–52.
Gaba DM. The future vision of simulation in health care. Qual Saf Health Care. 2004;13(Suppl.1):2–10.
World Health Organization. Patient Safety Curriculum Guide. Multi-Professional Edition, 2011. Available from: http://www.who.int/patientsafety/education/curriculum/Curriculum_Tools/en/ Accessed 9 Jan 2018.
Interprofessional Education Collaborative Expert Panel. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. Washington, DC: Interprofessional Education Collaborative; 2011.
Lu W-H, Goolsarran N, Hamo CE, Frawley SM, Rowe C, Lane S. Teaching patient safety using an interprofessional team-based learning simulation model in residency training. MedEdPORTAL Publications. 2016;12:10409.
The Joint Commission on Accreditation of Healthcare Organizations. National Patient Safety Goals. 2018. Available from:https://www.jointcommission.org/standards_information/npsgs.aspx Accessed 9 Jan 2018.
Agency for Healthcare Research and Quality (AHRQ). TeamSTEPPS. 2016; http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/index.html. Accessed 10 Aug 2017.
Parsell G, Bligh J. The development of a questionnaire to assess the readiness of health care students for interprofessional learning (RIPLS). Med Educ. 1999;33(2):95–100.
Binienda J. Critical synthesis package: readiness for interprofessional learning scale (RIPLS). MedEdPORTAL Publications. 2015;11:10274.
McFadyen AK, Webster V, Strachan K, Figgins E, Brown H, McKechnie J. The readiness for Interprofessional learning scale: a possible more stable sub-scale model for the original version of RIPLS. Journal of Interprofessional Care. 2005;19:595–603.
Gopalan C, Fox DJ, Gaebelein CJ. Effect of an individual readiness assurance test on a team readiness assurance test in the team-based learning of physiology. Adv Physiol Educ. 2013;37:61–4.
Oates M, Davidson M. A critical appraisal of instruments to measure outcomes of interprofessional education. Med Educ. 2014;49:386–98.
The authors wish to thank Colby Rowe, Perrilynn Baldelli and Sean Cavanaugh for their support and assistance with the simulation scenarios and standardized patient training; Colby Rowe and Getu Teressa MD, PhD for assistance with faculty debriefing.
This research is supported by the TBLC (Team-Based Learning Collaborative) Research Grant. The funding body did not play a role in the design of the study and collection, analysis, and interpretation of the data and in writing the manuscript.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly to protect participant confidentiality but are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
The Institutional Review Board at Stony Brook University Hospital, NY deemed this study as exempt as an educational initiative. Therefore written consent was formally waived. IRB acknowledgement # 748284.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.