A shared point of care ultrasound curriculum for graduate medical education

Abstract

Background

Point of care ultrasound (POCUS) education has grown significantly over the past two decades. Like most curricular items, POCUS education is siloed within individual graduate medical education (GME) programs. The purpose of this study was to evaluate the effectiveness of a shared GME POCUS curriculum between five GME programs at a single institution.

Methods

Post-graduate-year-1 (PGY-1) residents from emergency medicine (EM), family medicine (FM), internal medicine (IM), combined internal medicine-pediatrics (IM-Peds) and combined emergency medicine-pediatrics (EM-Peds) residency programs were enrolled in a core POCUS curriculum. The curriculum included eleven asynchronous online learning modules and ten hands-on training sessions proctored by sonographers and faculty physicians with POCUS expertise. Data was gathered about the curriculum’s effectiveness including participation, pre- and post-curricular surveys, pre- and post-knowledge assessments, and an objective skills assessment.

Results

Of the 85 residents enrolled, 61 (72%) participated in the curriculum. Engagement varied between programs, with attendance at hands-on sessions varying the most (EM 100%, EM-Peds 100%, FM 40%, IM 22%, Med-Peds 11%). Pre- and post-knowledge assessment scores improved for all components of the curriculum. Participants felt significantly more confident with image acquisition, anatomy recognition, interpreting images and incorporating POCUS findings into clinical practice (p < 0.001) after completing the curriculum.

Conclusion

In this shared GME POCUS curriculum, we found significant improvement in POCUS knowledge, attitudes, and psychomotor skills. This shared approach may be a viable way for other institutions to provide POCUS education broadly to their GME programs.

Background

As more medical schools integrate point-of-care ultrasound (POCUS) into their curricula, graduating students entering into graduate medical education (GME) training programs desire continued POCUS training during residency training [1]. POCUS has been a core component of emergency medicine (EM) and general surgery residency training for over a decade and has increasingly become a part of subspecialty training in several other medical specialties, including general internal medicine (IM) [2], family medicine (FM) [1, 3] and general pediatric [4, 5] residency training programs. While training guidelines exist for emergency medicine [6], pediatric emergency medicine [7] and family medicine [3] training programs, consensus training guidelines have yet to be published for internal medicine and general pediatric training.

The American Academy of Family Practice (AAFP) curriculum guidelines for FM residents recommend that all curricula contain some aspect of focused and longitudinal POCUS experiences, and some combination of didactics, hands-on learning, knowledge and skill assessment, and competency evaluation [1]. While these guidelines reflect best practice standards, the resources to provide the necessary didactics, experience and evaluation are challenging. In repeated surveys about providing POCUS training, lack of local expertise and resources are almost universally cited barriers to beginning and/or sustaining a POCUS training program [8,9,10,11,12].

Utilizing national programs, such as the American College of Surgeons’ Ultrasound Essentials for Surgeons Course or the American College of Chest Physicians’ Critical Care Ultrasound Course, allows GME trainees to find extra-curricular opportunities for POCUS training during residency or fellowship. However, these courses are limited by their high cost and the time needed to travel for already busy GME trainees. Similar challenges exist for other desired curricular content where time and local expertise are needed. Models of multidisciplinary delivery across specialty training using shared resources have been described for patient safety and physician leadership [13, 14]. Such multidisciplinary models take advantage of local expertise and scale delivery using workshops, seminars or online platforms that are readily accessible to large groups of learners.

In order to support a more system-wide approach to POCUS education, we sought to use a shared curricular delivery model for POCUS education for all post-graduate-year 1 (PGY-1) residents from five different GME programs at our institution. The primary aim of this study was to evaluate the effectiveness of a novel, cross-specialty GME POCUS curriculum as measured by participants’ attitudes towards POCUS and performance on knowledge assessment inventories comparing before and after the curriculum, as well as their performance on an objective structured clinical exam (OSCE).

Methods

Study design and participants

This was a prospective observational study of PGY-1 resident trainees from EM, FM, IM, combined internal medicine-pediatrics (Med-Peds) and combined EM-pediatrics (EM-Peds) residency training programs at a single institution. Eighty-five PGY-1 trainees were enrolled in the 10-month long curriculum, 61 residents were included in the analysis as they completed at least one learning module and attended at least one hands-on training session. The study went from August 2021 to May 2022. This study was approved by the Indiana University institutional review board with waiver of informed consent (protocol number 12269).

Curriculum

A core POCUS curriculum was developed by institutional POCUS leaders in EM, FM, pediatrics, radiology and obstetrics and gynecology. A smaller version of this curriculum was piloted with 24 residents spanning post-graduate year 1–3 from 4 different residency programs and these methods have been previously published [15]. The curriculum sought to cover POCUS exams with the most relevance to general medical practice and relevant to hospital based GME trainees. Exams for the curriculum were chosen based on consensus of local institutional POCUS leaders and after consultation with residency program directors or assistant directors in each program participating in the shared curriculum. A series of 11 modules (consisting of videos, case-based scenarios, and reading material) were developed and delivered through an institutional online learning management system (LMS) (Canvas by Instructure, Salt Lake City, Utah). The POCUS topics covered included physics, basic machine operation mechanics, concepts of procedural guidance as well as the following POCUS exams: focused assessment with sonography in trauma (FAST), lung, cardiac, soft tissue, early pregnancy, vascular access, renal/bladder, liver/gallbladder, and abdominal aorta. EM and FM trainees completes training in all modules. IM and IM-Peds trainees did not participate in the early pregnancy, procedural guidance, and vascular access training. IM-Peds trainees did not participate in renal or liver/gallbladder training. The asynchronous modules were complemented by 9 hands-on training sessions that occurred over a 10-month period and were proctored by faculty with POCUS experience from multiple departments spanning the GME spectrum.

Assessment tools

Trainee attitudes towards POCUS were measured through an online survey administered before and after the curriculum. POCUS knowledge was assessed using pre and post module quizzes built into each module of the LMS. Attendance at each hands-on training session was tracked. An OSCE with a standardized patient was administered to trainees at the end of the curriculum (see Additional file 1). The OSCE covered elements of the cardiac, aorta, lung, and FAST exam.

Statistical analysis

The Fisher’s Exact test was used to estimate differences in proportions between groups. For continuous variables, the Wilcoxon and Kruskal–Wallis tests were used. Logistic regression was used to estimate OSCE differences between EM and non-EM specialties. Linear regression was used to estimate module quiz score differences between EM and non-EM specialties and was adjusted for time spent in the LMS and event attendance. EM-Peds residents were treated as EM residents in the analysis as they have the same POCUS training requirements for their residency program and the same POCUS education faculty. All statistical analyses were performed using SAS Version 9.4 (SAS Institute, Cary, NC).

Results

Sixty-one of 85 (72%) PGY-1 residents actively participated in the GME POCUS training program by completing at least one module and one hands-on training session. Twenty-three (38%) residents were from IM, 16 (26%) from EM, 2 (3%) from EM-Peds, 13 (21%) from FM and 7 (11%) from IM-Peds. Fifty-one (31/61) percent were female (see Table 1 for demographics). Participants rated their pre-curriculum POCUS training as “very limited” (median score of 2, range 1 to 5) on a 5-point Likert scale (range 1 “no training at all” to 5 “extensive training”). Residents felt that POCUS was essential to learn for their future practice both before (median 4, range 1 to 5) and after the curriculum (median 4, range 2 to 5), p = 0.39.

Table 1 Participant demographics (n = 61)

POCUS Utilization

When comparing pre- to post-curriculum we found an increase in self-reported POCUS utilization for guiding diagnosis and/or patient assessment (p = 0.075) and an increase in POCUS use to guide procedures (p = 0.18). However, these did not reach statistical significance (see Supplementary Table 1). Pre-curriculum, 23 (38%) residents had never used POCUS in the clinical setting and post-curriculum this significantly decreased to 9 (14.7%) p = 0.021.

Barriers and confidence

Participants reported significant improvement to all self-perceived barriers to learning and using POCUS, however, most participants still identified their ability to interpret images as a barrier even after completing the curriculum (Table 2). Residents felt significantly more confident with manipulating the POCUS machine for image acquisition, recognizing anatomy on POCUS images, obtaining and interpreting images, and incorporating POCUS findings into clinical practice (p < 0.001, see Supplementary tables 2 and 3). They reported feeling significantly more comfortable performing every modality taught during the curriculum (p < 0.033) except cardiac standstill evaluation (p = 0.07) and first trimester pregnancy (p = 0.19) (Table 3). Overall confidence with using POCUS after completing the course was high at 4 (range 2 to 5).

Table 2 Comparison of pre- to post-curriculum barriers to POCUS utilization n (%)

Table 3 Participants attitudes towards POCUS use for a given clinical scenario. Median scores reported (min–max)

Pre- to post curriculum knowledge assessment

Median test scores improved significantly across all modalities and specialties except IM-Peds for the lung and cardiac exams, and FM for the soft tissue and ultrasound guided vascular access exams, which both showed improvement but did not reach statistical significance. Ultrasound guided vascular access knowledge assessment remained high for FM residents both before (median 8.5) and after (median 9.5) (Table 4).

Table 4 Knowledge assessment scores pre- and post- modules by specialty and modality. Median scores reported (min–max)

Objective structured clinical exam

An OSCE was performed post-curriculum to assess psychomotor skill and pathology knowledge; 19 (83%) IM, 16 (89%) EM, 7 (54%) FM and 2 (29%) IM-Peds residents participated in this assessment although not all residents completed all parts of the OSCE. Ninety-eight percent (40/41) of residents were able to adjust the POCUS machine settings, 93% (28/41) were able to adequately position the patient, and 76% (31/41) were able to perform positioning adjuncts to improve image quality. Adjuncts were completed accurately by EM and IM-Peds residents 100% of the time (Table 5). While most residents were able to identify anatomic structures including the abdominal aorta, IVC, spine, right and left ventricle, pericardium, diaphragm and aortic outflow tract, no FM residents were able to identify the IVC, and the majority could not identify the right and left ventricle, aortic outflow tract, the left atrium, hepatorenal recess or liver tip. Most EM and FM residents were unable to describe two findings of pneumothorax on ultrasound. All residents were able to describe indications for a cardiac POCUS, and most were able to discuss measurements for a normal abdominal aorta, describe the difference between the IVC and aorta, and name three pathologic findings on an eFAST examination.

Table 5 Performance on OSCE by specialty as measured by Fisher’s Exact test

When comparing psychomotor skill and identification of anatomy on POCUS between EM and non-EM specialties, non-EM residents were significantly less likely to measure the abdominal aorta correctly (p = 0.015), describe normal aorta measurements (p = 0.019), obtain a transverse (p = 0.031) and sagittal (p = 0.012) aorta view on the aorta exam. They were also less likely to obtain a liver tip view (p = 0.002) on a FAST exam. The other assessments found similar results between EM and non-EM residents.

Participation

Attendance at hands-on POCUS training sessions was significantly higher for EM and EM-Peds residents with residents attending 100% of the sessions, followed by FM (40%), IM (22%) and IM-Peds (11%), p < 0.001. Forty-three percent of residents completed < 50% of the LMS modules, while 57% completed ≥ 50% of the modules, see Supplemental Table 4. Residents who completed ≥ 50% of the modules we significantly more confident in their ability to perform POCUS than those who did not (4.0 vs 3.0, p < 0.001). LMS module participation did not impact OSCE performance apart from residents who completed ≥ 50% of the modules being significantly more likely to accurately measure the abdominal aorta (p = 0.011), identify the hepatorenal recess (p = 0.03), and obtain an image of anterior lung sliding (p < 0.001). When comparing LMS module completion to OSCE performance by specialty, only identifying the right ventricle (p = 0.04), identifying the left ventricle (p = 0.04) and identifying the aortic outflow were statistically significant and only for IM residents, see Supplemental Table 5.

Discussion

Although POCUS use continues to grow among various medical specialties, obstacles remain for creating adequate POCUS curricula within both undergraduate and GME programs [3, 8, 16]. Faculty with POCUS expertise, access to equipment and curricular time within the training program remain as core challenges. GME programs have used a variety of methods to navigate these challenges including, utilizing national courses [16], development of dedicated tracks with asynchronous and self-directed learning [3, 16], peer to peer teaching [16], and leveraging the expertise and resources of other GME programs [17]. To overcome these obstacles at our institution, we leveraged the resources used to create our undergraduate medical education POCUS curriculum to create a common core POCUS GME curriculum. This included POCUS experts across a variety of specialties, sonographers with experience teaching POCUS, hand-held POCUS equipment and lab space. To our knowledge, this approach, including our shared POCUS curriculum development by a multi-disciplinary group of POCUS experts with embedded knowledge and skills-based assessments is the first to be described and was piloted on a smaller scale prior to this initial launch [15].

In our study, PGY-1 residents from a variety of GME training programs experienced a significant change in attitudes and behaviors towards POCUS and improved their POCUS knowledge and skill set across a wide variety of POCUS exams. As expected, with experience and training, we found the number of residents with self-perceived barriers of using POCUS decreased, as has been demonstrated in other studies of specialty specific POCUS training programs [2, 4]. This resulted in increased self-reported use in their clinical practice and increased comfort in using POCUS in a wide variety of clinical scenarios. While most residents felt that machine operation and image acquisition were not a barrier post-curriculum completion, the majority continued to feel that image interpretation was still a significant barrier. This finding is not surprising, as exposure to a range of normal and pathologic exams is needed to improve confidence in image interpretation and is in keeping with a similar study of IM residents who were exposed to a novel POCUS curriculum during the Covid 19 pandemic [18].

Of particular interest, our data showed no increase in self-reported likelihood of using POCUS in a variety of clinical scenarios. In fact, we found that residents were significantly less likely to use POCUS for both a DVT or a screening abdominal aorta exam. We did not assess the reasons for this lack of increase. However, we believe this may be due to several factors, including: 1) extremely limited access to any POCUS equipment in the hospital ward or outpatient clinic setting, 2) time needed to complete the exam among other competing priorities and 3) lack of mentorship and supervision by more senior physicians with POCUS experience at the bedside. Interestingly, FM residents, who spend the most time in the outpatient clinic setting compared to the other participants and would more likely have an opportunity to perform screening abdominal aorta exams on high-risk patients, performed significantly worse than their peers on the OSCE in almost all domains relating to this exam. Based on these results, FM residents need more training and experience before clinical use.

While all PGY-1 residents from each of the 5 residency programs had access to the curriculum, not all participated equally. Each residency program was free to implement hands-on training that best fit their existing curriculum and clinical schedules. For FM, hands-on training occurred monthly during didactic conference. For IM and IM-Peds this occurred at varied times throughout the month. For EM and EM-Peds this occurred as part of a dedicated POCUS rotation already built into their existing curriculum. As a result, those specialties who built hands-on training into their established residency curriculum had much higher participation rates. As a result of this data, at the conclusion of the curriculum, the IM program built in mandatory hands-on sessions into their quarterly skills training sessions for interns. Despite the challenges in attending hands-on sessions, IM and IM-Peds residents performed well on the OSCE. However, EM and EM-Peds residents significantly outperformed the other groups in several OSCE domains. This is likely a reflection of their existing curriculum with a dedicated POCUS rotation and many trained faculty using POCUS clinically.

This study demonstrates the feasibility of a shared POCUS curriculum implemented across multiple GME training programs and its impact on trainees. Future iterations of this curriculum will be expanded to include surgical specialties, anesthesia, and critical care training programs. With a more structured approach to the curriculum and enhanced participation, the impact of such a curriculum can be further studied. We hope that this shared POCUS curriculum increases communication across medical specialties and will lead to improved patient care. Furthermore, the impact of such a curriculum on future, independent practice post-graduation from GME training is unknown.

Limitations

There were multiple limitations identified in this study. First, expectations were varied among the various GME programs. While EM has set POCUS requirements for graduation, this is not true for other specialties. This requirement alone would encourage increased dedication to learn and perform these exams among the EM cohort. Additionally, self-directed scanning outside of hands-on sessions for each resident was not monitored, tracked, or studied. Undoubtedly, the availability of ultrasound equipment also affected participation and practice. Multiple cart-based ultrasounds are readily available in the emergency department. This availability is either reduced or completely absent on hospital wards or in outpatient clinics. This increases the dependence of IM, IM-Peds and FM on performing practice examinations in the educational environment and allows less opportunity for use in the clinical environment. This likely negatively affects residents' comfort and confidence in the clinical use and integration of POCUS. The curriculum was supplemented with more clinical bedside teaching for EM and EM-Peds residents as part of their required POCUS rotation. This variation in curriculum could affect participation, knowledge and skill acquisition, and outcome of these cohorts, and is seen in the OSCE results.

Portions of this curriculum and study were performed during the Covid 19 pandemic. Due to the effect of this pandemic on clinical care, social distancing precautions, and potential risk aversion among resident learners' participation in in-person training sessions and ability to perform practice scans in the clinical environment may also have been reduced. This renders the results of this study subject to a degree of selection bias.

Conclusion

A shared, common core, multi-specialty longitudinal GME POCUS curriculum for PGY-1 residents resulted in improved attitudes towards POCUS across a variety of exams. For those who participated in the program, POCUS knowledge and comfort performing exams increased significantly. However, participants continued to lack confidence in their ability to interpret POCUS images. While clinical use increased, beliefs about clinical utility remained unchanged for most exam types. A shared approach may be a feasible way for other institutions to provide POCUS education to their GME programs.