Comparison of In-person VS remote learning modalities for ultrasound-guided knee arthrocentesis training using formalin-embalmed cadavers

Abstract

Objectives

This study aims to compare the efficacy of remote versus in-person training strategies to teach ultrasound guided knee arthrocentesis using formalin embalmed cadavers.

Methods

30 first-year medical student participants were randomly assigned to remote or in-person training groups. Pre- and post- training surveys were used to evaluate participant’s self-confidence in their ability to perform the procedure. Participants were asked to watch a 30-minute training video and then attend a skills training workshop. The workshops consisted of 20 min of hands-on instruction followed by a skills assessment.

Results

Following training, participant self-confidence increased significantly across all survey items in both groups (p = 0.0001). No significant changes in participant self-confidence were detected between the groups. Skills and knowledge-related metrics did not differ significantly between the groups with the exception of the “knowledge of instruments” variable.

Conclusions

Our data suggests that remote ultrasound-guided procedure training, although logistically complex, is a viable alternative to traditional in-person learning techniques even for a notoriously hands on skill like ultrasound guided knee arthrocentesis. Novice first-year medical student operators in the remote-training group were able to significantly increase their confidence and demonstrate competency in a manner statistically indistinguishable from those trained in-person. These results support the pedagogical validity of using remote training to teach ultrasound guided procedures which could have implications in rural and global health initiatives where educational resources are more limited.

Background

Arthrocentesis is essential for the evaluation of patients presenting with acute atraumatic joint pain and remains the gold standard for the diagnosis of septic arthritis. The use of ultrasound (US) to aid with procedural guidance has been shown to decrease complications, decrease length of procedures, decrease patient reported pain, improve overall patient satisfaction, and improve procedural success rates [1,2,3,4,5]. Recently, US has been adopted when performing arthrocentesis based on several advantages over the traditional landmark technique including: increased sensitivity for the detection of joint effusions, improved success rates, higher volumes of aspirated synovial fluid, improved needle placement accuracy, decreased patient reported pain, and higher levels of procedural confidence among operators [6,7,8,9].

Given US guidance is becoming the standard of care when performing invasive procedures, expert consensus recommends competency in US-guided procedures as one of the core milestones for graduating medical students [10]. Currently, arthrocentesis training, like many other procedures, primarily takes place during residency with learners training on either live patients or high-fidelity models. As a result, graduating students often lack adequate exposure to invasive procedures including arthrocentesis. Students without access to procedural training in medical school are less likely to feel prepared to perform these procedures upon entering residency highlighting the need for procedural education in undergraduate medical education [11].

Previous studies have illustrated that brief, educational interventions using cadaver models significantly increases students’ confidence and the technical skills required to perform US guided procedures [12,13,14,15]. However, in-person workshop such as these are not accessible to many students and physicians, particularly those in resource limited settings. Additionally, the recent COVID-19 pandemic dramatically changed the delivery of medical education based on the limited ability to conduct in person educational sessions and the abrupt transition to remote learning [16]. It highlighted the necessity to develop viable methods of remote learning and the delivery of educational materials. Several studies have illustrated remote US training programs provide a safe, feasible and effective alternative to in-person teaching [17,18,19]. More specifically, remote simulation using videoconferencing has been shown to be an effective modality for teaching US guided procedures to physicians [20, 21].

While prior studies have illustrated the ability of remote, procedural US simulation to improve the knowledge and technical skills of study participants, to our knowledge, there has been no direct comparison between in-person and remote training. Additionally, there is a paucity of data evaluating the effectiveness of remote simulation training for medical students. Our primary objective is to directly compare the effectiveness of in-person and remote procedural US educational curriculums teaching US guided arthrocentesis by assessing participant knowledge and procedural competency post-intervention. Secondary objectives are to compare participants’ satisfaction with the educational experience, participants’ confidence regarding their hands-on performance, and identify specific strengths, weaknesses, and challenges with teaching US-guided arthrocentesis remotely.

Methods

Setting and population

This study was conducted in anatomy laboratory facilities provided by the Geisel School of Medicine at Dartmouth College. Study participants were first-year medical student-volunteers who signed a consent form prior to study enrolment. IRB approval to conduct this exempt study was provided by the Dartmouth Committee for the Protection of Human Subjects (#00032582). Permission to use body donors in this project was provided by the Geisel Anatomical Gifts Program.

Participant recruitment

A class of 92 first-year medical students were emailed an announcement to volunteer to participate in this study on a first-come, first-serve basis. Thirty participants with no experience performing knee arthrocentesis enrolled after providing written informed consent. Participants were randomly assigned to an “in-person” or “remote” training group using a random number generator.

Study design

This randomized, prospective single center study was designed to train medical students to perform knee arthrocentesis on formalin-embalmed cadavers using either remote or in-person learning modalities, and then compare participants’ objective and self-perceived proficiency.

Equipment

Mindray MX7 ultrasound systems equipped with L12-3RC linear transducers (Mindray Global, Shenzhen China) were used to guide 22 g 1.5 inch hypodermic needles during knee arthrocentesis. While larger gauge needles (18–20 g) are traditionally used for knee arthrocentesis in practice, 22 g needles were used in order to protect against synovial fluid drainage and damage to the cadavers. Transducers were protected from chemical exposure by Sheathes3D non-sterile probe covers (Morgan Hill, CA U.S.A). Aquasonic transducer gel (Parker Labs, Fairfield NJ U.S.A.) was utilized in all image acquisitions. Video feeds of ultrasound screens and the hand motions of instructors and participants were captured using standard phone and laptop webcams and a single ceiling mounted 12X SDI HDMI camera (PTZ Optics, Downingtown PA U.S.A).

Cadaver ultrasound imaging and selection

Forty-four knees from anatomical donors were screened with ultrasound for suprapatellar fluid accumulation. Knees were considered optimal for training if a large fluid pocket was located directly superior to the patella with minimal medial or lateral displacement. Of the 44 knees screened, 10 were logged as usable for the study. Of these 10, knees with smaller fluid accumulations were injected with approximately 20–30 mls of normal saline into the suprapatellar bursa to create effusions of relatively equal size in each donor used in the study (Fig. 1, Panel B).

Fig. 1

Remote Learning Laboratory Strategy .Panel A depicts the contents of one of the monitors that were viewed by both the instructor and the participant. The two images on the left side of panel A showed a live feed of the instructor’s workstation and ultrasound screen for participant viewing. The two images on the right side of panel A displayed a live feed of the workstation and ultrasound screen of the participant for instructor viewing. Panel B shows successful participant access of a cadaveric suprapatellar effusion. Panel C shows a study participant using the remote learning station to perform arthrocentesis

Skills training activities

Both remote and in-person groups watched a 20-minute orientation video within the 48 h prior to their training workshop. The orientation video outlined ultrasound probe characteristics and maneuvering; it also provided a tutorial on identifying relevant anatomical knee structures on US, locating suprapatellar fluid accumulation on long- and short-axis views, and using a long-axis or in-plane technique to guide a needle in the aspiration of fluid. All cadavers used for the study were positioned supine on a gurney and covered with an opaque plastic sheet; a hole was cut in the plastic to expose the procedural field. Both remote and in-person groups received 25 min of one-on-one instruction by an ultrasound fellowship-trained emergency medicine physician. Prior to the study, the instructors underwent a brief orientation to the training curriculum and objective skills assessment to ensure standardization in teaching technique. Participants were guided by the instructor to aspirate fluid from the suprapatellar bursa as many times as they could during the training period. Since cadaver knees were re-used for multiple training sessions, participants re-injected aspirated fluid exactly from where it was drawn before removing the needle from the donor paying close attention to not reinject air.

Remote training group

All participants in the remote training group received instruction from the same ultrasound fellowship-trained emergency physician for a 25-minute period. The instructor and trainees were located in separate rooms with their own cadaver procedure station. Upon arrival, participants were briefly oriented to the lab surroundings and technical setup by a member of the study staff. Training was then provided via a video conference call (Zoom Video Communications, San Jose CA U.S.A). In the instructor’s room, live video feed was provided by two stationary cameras positioned to visualize the instructor’s hands and ultrasound machine monitor respectively. In the participants’ room, one stationary camera was positioned to visualize the participants’ US monitor, and a mobile camera operated by a member of the study staff was used to visualize participants’ hand motions. In the instructor’s room a laptop equipped with a webcam and a cellphone camera were set on tables near the cadaver station oriented towards the instructor’s hands and ultrasound screen respectively. In both rooms a monitor was used to allow the occupant to see the hands and ultrasound screen of the person in the other room. Both the instructor and participant wore a pair of mic’d headphones to transmit audio between rooms. This setup (Fig. 1) allowed participants and the instructor to visualize each other’s movements, provide/receive real-time feedback, and ask questions as needed.

In-person training

Participants in the in-person group were randomly assigned to train for 25-minutes with one of three ultrasound fellowship-trained emergency medicine physicians. Instructors used verbal instruction and hand-over-hand, physical demonstration to teach participants to perform the procedure. Each instructor used the same cadaver knee to teach all their trainees.

Assessments

Participants were emailed a pre-training survey to assess their previous experience level. The survey also asked participants to use a 5-point Likert scale to indicate their subjective confidence in their ability to perform various aspects of the procedure before training (Table 1). The same survey was taken by participants following training without demographic-based questions and with the addition of an open-ended opportunity to submit comments about their experiences.

Table 1 Participant Demographics and Previous Experience with Ultrasound A table reporting the number of participants with different levels of previous ultrasound experience by training group

Following skills training, a 5-minute objective assessment was performed. Procedure competency for both groups was assessed by the same instructor that they learned from physically present using the same cadaver on which they were taught. Participants were evaluated using a previously validated global ratings scale rubric [22] examining pre-specified outcomes, such as procedure completion, duration, number of attempts, and anticipated complications (Appendix 1).

Statistical analysis

Descriptive statistics were calculated and presented as frequencies and percentages for categorical values. This was done for the full dataset and by the learning modalities groups (In-Person vs. Remote). Likert scales were used in their numeric form (a scale from 1 to 5) where pre- and post- assessments and their difference (deltas) were evaluated using non-parametric Kruskal-Wallis tests. Learning modalities were evaluated in the same way. The variable “Time to aspiration” was evaluated using a t-test. All analysis were performed using SAS/STAT v.9.4 (SAS Institute Inc. Cary, NC) using the NPAR1WAY procedure for all non-parametric test and TTEST procedure for parametric tests. Effect sizes (E²) were estimated for non-parametric tests following guidelines by Tomczak and Tomczak [23]. For parametric test R² was used to express effect size. Both E² and R² are expressed in a range from 0 (indicating no relationship) to 1 (indicating perfect relationship). Statistical significances were declared at P ≤ 0.05.

Results

Participant demographics

Participant demographic and information about all prior ultrasound-related experiences are displayed in Table 1. No participants reported that they had performed knee arthrocentesis before participation in this study. Six of the 26 participants had used ultrasound to guide a needle, and 21 reported they had never used an ultrasound machine before.

Pre-and post-training self-confidence survey

The results of the pre- and post-training self-confidence survey are depicted in Fig. 2; Table 2. Table 3 shows the exact text of items deployed in the self-confidence survey. Following training, participant self-confidence increased significantly across all survey items in both groups. In the remote group, 5 of the 7 items were declared significant at p < 0.0001. The remaining 2 survey items were significant to a p-value of < 0.003 (Table 2). In the in-person group, the increase in confidence from pre to post was significant across all items (p < 0.0001). When comparing the overall change in confidence between the training groups, no significant difference in confidence was identified in any of the survey items (Table 2).

Table 2 Numerical Self-Confidence Changes by Training Group The means, standard errors and p-values related to a subjective self-confidence survey (table 3) as reported by participants in their ability to perform various aspects of the knee arthrocentesis procedure

Table 3 Skills Assessment self-Confidence survey a table reporting the text of survey items deployed to participants before and after ultrasound-guided knee arthrocentesis skills training

Fig. 2

Participant Self-Confidence Changes by Training Group. A pair of bar graphs depicting the relative change in participant self-confidence by training group. Panel C depicts the change in confidence on each survey item between the training groups

Skills training assessments

The outcomes of the objective skills assessment for both groups are shown in Table 4. Three participants failed the skills test (2 from the in-person group and 1 from the remote group). Two participants from the in-person group and 5 from the remote group had complications while performing the procedure. Participants in both training groups took an average of 100 s to aspirate 1mL of fluid from their cadaver knee during the skills assessment. On average, participants in both groups were able to successfully aspirate fluid 2–3 times during their training period. Types of complications are listed in Appendix 1.

Table 4 A table reporting the outcomes of the post-training objective skills assessment. Panel A shows the frequency and percentage of participants by group that passed, failed, and experienced complications during the skills assessment. Panel B reports the performance of participants on 10 measured skills and knowledge related variables associated with the skills assessment

Out of the 10 skills and knowledge-related metrics assessed by the instructors, “knowledge of instruments” was the only variable found to differ significantly between the groups. There was no significant difference between the in-person and remote groups in any of the other measured variables including; the number of successful aspirations performed during the training period, the time needed to aspirate 1 mL of fluid during testing, or the number of attempts needed to withdraw 1 mL. Analyses were also performed to look for significant correlations between each individual variable reported in Table 3, Panel B within each training group. The results of this analysis are reported in Appendix 2.

Discussion

Procedural-based simulation is an important aspect of medical training which has been shown to increase trainees’ confidence, enhance technical skills, and improve procedural success rates on live-patients [12,13,14,15, 24]. The benefits of using US to guide procedures are well documented and, in many cases, US-guidance has become standard of care [1,2,3,4,5, 25]. As portable ultrasound devices have become more accessible, there has been a rapid expansion in the number of specialties performing US-guided procedures, including but not limited to: anesthesia, radiology, surgery, internal medicine, primary care, and emergency medicine [26]. As result, becoming proficient in US-guided procedures through simulation training has become an important aspect of medical training and a recommended competency for graduating medical students [10]. The technical skills acquired while learning US-guided arthrocentesis can be applied to all types of US-guided procedures.

Traditional, in-person simulation, while proven to be an effective method for procedural education, may not be feasible in many settings given limited resources or the inability to conduct in-person events. While prior studies have shown that remote procedural US training programs offer an effective alternative to in-person teaching, to the best of our knowledge, this study is the first to directly compare the effectiveness of in-person and remote procedural US training programs [17,18,19]. Data from this study suggests that remote-training is a viable alternative strategy, as there were no significant differences in knowledge or skills-related outcomes (with one exception) between the in-person and remote learning groups examined in this study.

Both the in-person and remote-training curricula significantly improved participants’ confidence across all surveyed items with no significant differences between groups identified (Table 2). Post-training, both in-person and remote training groups were significantly more confident differentiating the sonographic appearance of the various tissue types (muscle, bone, tendons) and recognizing important anatomic structures (patella, femur, suprapatellar recess). Both groups were also more confident in their ability to use US to identify effusions and perform the actual procedure on live-patients. As mentioned previously, the sonographic knowledge and tactile skills required for US-guided arthrocentesis can be applied to the majority of other US-guided procedures.

“Knowledge of Instruments” was the only objectively measured technical skill that was significantly different between the in-person and remote groups (4.73 vs. 3.36, p = 0.0208). The true cause of this difference is unknown as the training curriculum was standardized across both groups. Additionally, the complication rates were slightly better in the in-person group than remote (n = 2 vs. n = 5). The remaining nine objectively measured variables including those evaluating probe/needle handling, time to aspiration, number of attempts needed, and overall performance were similar between groups. These results provide further support for remote procedural US training as a viable alternative to in-person teaching for learning US-guided knee arthrocentesis.

The mean time to aspiration of 1 ml of synovial for both groups (in-person: 106 s, remote: 93 s) was longer than similar, previous studies using first year medical students. Clason et al., found first year medical students were able to successfully aspirate 1 ml of synovial in an average time of 41 s on the first attempt. Differences in study educational curricula and potentially difference in fluid pocket size likely account for the differences between these times. In their study, students were able to choose from a static, in-plane (long-axis) or dynamic, out-of-plane (short-axis) approach while our participants were only taught and were required to use an in-plane approach. Previous studies directly comparing out-of-plane vs. in-plane approaches to US-guided procedures have shown the out-of-plane approach is faster, particularly with novice US users [27, 28]. Utilizing only the in-plane approach in our study likely accounts for the increased “time to aspiration” for our participants.

The feedback from students was overall very positive in both groups with the vast majority describing the training sessions as enjoyable and efficacious. Students from the remote group frequently commented on being surprised at the effectiveness of the training session after initially being skeptical regarding its efficacy. One student even felt their learning of the procedure may have been improved by remote learning as they were forced to make adjustments on their own with only verbal direction. Frequently described strengths included having two-way communication between the instructor and student and having cameras on both the instructor’s and student’s hands. Described weaknesses included the inability of the instructor to physically guide a student’s hands and challenges seeing the instructors US screen well. All comments are included in Appendix 3.

Limitations

This study has several limitations. Our small sample sizes (26 total participants; 11 in the remote group and 15 in the in-person group) prevented our ability to perform sub-group analyses or comparison of outcomes based on instructor and the cadaver used by participants and may have resulted in selection bias in the sample. While the instructors were all ultrasound fellowship-trained and underwent an orientation for both the training curriculum and objective skills assessment to control for differences in teaching or inter-observer variability, our small sample size did not allow for a direct comparison. The same instructor involved in training also conducted the objective skills assessment for each participant, which introduces the possibility of bias. Future studies would benefit from having an additional, independent evaluator for each participant and analyzing correlations between evaluator scoring. Three different cadavers were used for the training sessions and objective assessments. Anatomic variations in the location of synovial fluid pockets between each cadaver may have resulted in unaccounted differences with regards to the ease of which the procedure was performed. The authors also acknowledge that many institutions across the world don’t have access to human cadaver laboratories. Additionally, a steep and unforeseen learning curve on the part of the remote-group instructor resulted in the loss of the first two remote participants after substantial logistical and technical challenges arose which dramatically altered the participants’ experience. Future studies would benefit from a power analysis to determine the number of participants required to show statistical significance across the primary and secondary outcomes and a “walk through” of the remote-training session procedure prior to the first study participant.

The skills training assessments were performed immediately after the training session, therefore, only the short-term effect of the training sessions could be assessed. Future studies would benefit from performing a repeat skills assessment 4–6 weeks later to assess for any differences in long-term retention. Lastly, since in-person learning is considered the gold standard in teaching, non-inferiority assessments should be implemented.

Conclusion

Our data suggests that remote, US-guided knee arthrocentesis training is a viable alternative to traditional in-person training techniques. Novice first year medical student operators in the remote training groups were able to significantly increase their confidence and demonstrate competency in a manner statistically indistinguishable from those in the in-person training groups (excluding the “knowledge of instruments” category). Although logistically complex and challenging, with the use of simple and commonly available video conferencing equipment, US-guided procedures like knee arthrocentesis can be successfully and effectively taught remotely. These findings could have relevance in the fields of global health and rural medicine as they could be employed to bring high-quality training in US-guided procedures to resource-limited settings that lack trained faculty to teach these important clinical skills.