- Research
- Open Access
- Published:
First year internal medicine residents’ self-report point-of-care ultrasound knowledge and skills: what (Little) difference three years make
BMC Medical Education volume 21, Article number: 476 (2021)
Abstract
Background
With increasing availability of point-of-care ultrasound (POCUS) education in medical schools, it is unclear whether or not learning needs of junior medical residents have evolved over time.
Methods
We invited all postgraduate year (PGY)-1 residents at three Canadian internal medicine residency training programs in 2019 to complete a survey previously completed by 47 Canadian Internal Medicine PGY-1 s in 2016. Using a five-point Likert scale, participants rated perceived applicability of POCUS to the practice of internal medicine and self-reported skills in 15 diagnostic POCUS applications and 9 procedures.
Results
Of the 97 invited residents, 58 (60 %) completed the survey in 2019. Participants reported high applicability but low skills across all POCUS applications and procedures. The 2019 cohort reported higher skills in assessing pulmonary B lines than the 2016 cohort (2.3 ± SD 1.0 vs. 1.5 ± SD 0.7, adjusted p-value = 0.01). No other differences were noted.
Conclusions
POCUS educational needs continue to be high in Canadian internal medicine learners. The results of this needs assessment study support ongoing inclusion of basic POCUS elements in the current internal medicine residency curriculum.
Background
The use of point-of-care ultrasound (POCUS) in the assessment of medical patients is increasingly recognized. Support and endorsement statements of its use by several national and international internal medicine organizations importantly highlight the need for appropriate training [1,2,3,4]. In an attempt to better define an appropriate scope of use, the Canadian Internal Medicine Ultrasound (CIMUS) group identified four diagnostic applications (inferior vena cava, pulmonary B lines, pleural effusion, and abdominal free fluid) and three procedures (thoracentesis, paracentesis, and central venous catheterization) that should be included in the curriculum for core internal medicine residency programs [Postgraduate year (PGY)-1 to PGY-3] across Canada [5]. Foundational to these skills are multiple additional basic POCUS competencies such as knobology (knowledge and use of knobs and controls on the ultrasound device), ultrasound physics, Doppler, and safety principles [6]. To help optimize and monitor the delivery of high quality education, the CIMUS group also developed education indicators [7]. However, despite the presence of these guiding principles and documents, a consistent and significant barrier to POCUS curriculum development and implementation efforts has been the limited number of POCUS-trained faculty currently available [8]. In the face of limited available expertise, it may be preferable if faculty energy could be directed towards teaching a more focused and targeted curriculum.
In a 2014 national survey of Canadian medical schools, approximately half the schools had implemented POCUS education in their medical school curriculum [9]. Since that time, the ultrasound curriculum has been increasingly adopted worldwide [10, 11]. For example, in the United States, approximately one-third of medical schools had an ultrasound curriculum in 2014 [12]. By 2019, over 70 % reported having an ultrasound curriculum [13]. Although specifics within a medical student POCUS curriculum may vary, the basic foundational elements such as transducer selection, knobology, and safety principles of minimizing ultrasound dose delivery, such as the ALARA (as low as reasonably achievable) principle, are consistently agreed upon as critical curricular content [10, 14]. Therefore, if internal medicine residents are now entering residency with basic POCUS skills and knowledge learned at the medical school level, then educationally it may be more efficient to target POCUS training at a higher level. However, it is unclear whether or not residents starting in internal medicine residency have these basic skills. In our needs assessment survey in 2016, PGY-1 learners in internal medicine were largely novices, with minimal baseline POCUS knowledge and skills [15]. These learning needs are similar to needs expressed by internal medicine learners in other countries, including the United States [16,17,18,19], the Netherlands [20], and Saudi Arabia [21, 22].
Given that much time has passed since our initial survey, we hypothesized that learners from more recent cohorts may be starting their residency training with more pre-existing POCUS knowledge and skills than previous cohorts. If more recent learners have higher knowledge and skills, we hope to then refine and update our curriculum accordingly.
Methods
Aim
This study seeks to compare the knowledge and skills of PGY-1 internal medicine residents across three Canadian centers with reported knowledge and skills from a similar cohort in 2016.
Study Design
This multi-center cross-sectional survey study was conducted at three Canadian academic centers: University of Alberta, University of Calgary, and University of Ottawa. All three centers participated in the original 2016 study.
Survey
We used the same survey that we previously developed and published, containing questions regarding learner opinions on 15 diagnostic applications, 9 procedures, and 18 knowledge items, in addition to questions on baseline demographic data [15]. Briefly, content validity of our survey was supported by literature review and expert input, and the draft survey was pretested on 8 non-internal medicine trainees for feedback on survey length, content, and clarity [15]. The final online survey (Survey Monkey Inc. San Mateo, California, USA; www.surveymonkey.com) was originally distributed to the trainees in 2016 and for this present study cohort, from August to October 2019. We sent up to two reminder emails between two and eight weeks to maximize participant response rate. No incentives were used in this study. No repeat pilot-testing of this survey was done for 2019. The results from the 2016 are previously reported [15] and only the data from the three centers included in the present 2019 study are used for comparison purposes. The fourth center was not able to participate in this present study due to competing workload demands precluding the requisite application to their ethics board.
Participants
All postgraduate year (PGY) 1 internal medicine residents in 2019 at the three study institutions were invited to participate. Those who consented and completed the survey were included. Our survey results were then directly compared with the anonymized survey results for 47 PGY-1 learners who completed the identical survey at the same three institutions in 2016 [15].
Outcomes
Perceived applicability of diagnostic applications and procedures to patient care in internal medicine was assessed on a 5-point Likert scale with 1 = not at all applicable, and 5 = very applicable. Self-reported level of skills/knowledge was also assessed on a 5-point Likert scale, with 1 = very poor, and 5 = very good. Skill gap was defined as the difference between perceived applicability of an application or procedure and self-rated skills/knowledge in that application or procedure [15].
This study was approved by the University of Calgary Conjoint Health Research Ethics Board, the University of Alberta Research Ethics Board, and the Ottawa Health Science Network Research Ethics Board. Informed consent was obtained from all subjects. All subjects were over the age of 18. All methods were performed in accordance with the relevant guidelines and regulations.
Statistical Analysis
Categorical variables between groups were compared with Fisher’s exact tests and chi-square tests. Continuous variables between groups were compared with Student’s two-sample t-tests. In comparing group differences in diagnostic applications and procedures, we applied Bonferroni corrections to p-values to control for the effect of multiple comparisons [23]. All analyses were performed using SAS version 9.2 (SAS Institute Inc., Cary, NC).
Results
Of the 97 participants invited to the study, 58 completed the 2019 survey (response rate = 60 %). This response rate is similar to that in 2016, where 74 PGY-1 residents from the three institutions were invited and 47 completed the survey (64 %, p = 0.62). Baseline characteristics and procedural experience of the participants in both cohorts are outlined in Table 1. In general, compared to learners in the 2016 cohort, learners in 2019 reported lower prior procedural experience in ultrasound-guided paracentesis, thoracentesis, and central venous catheterization (Table 1). Both cohorts reported a similar degree of difficulty in finding procedural preceptors/supervisors (Table 1).
Perceived applicability, self-reported skills, and skill gap in diagnostic applications
In both cohorts, ultrasound was perceived to be highly applicable to the practice of internal medicine in all diagnostic applications (Table 2). Learners continued to report low skills in all applications. No differences in learners’ perceptions were identified over time, except for the assessment of pulmonary B lines, where the 2019 cohort felt the application was more applicable to internal medicine than did the 2016 cohort [4.6 ± standard deviation (SD) 0.7 vs. 3.9 ± SD 1.0, adjusted p-value = 0.03]. The 2019 cohort also reported higher skills in assessing pulmonary B lines than the 2016 cohort (2.3 ± SD 1.0 vs. 1.5 ± SD 0.7, adjusted p-value = 0.01). Reported skill gaps were not significant in any other applications (Fig. 1). In both cohorts, skill gaps continued to be the highest in deep vein thrombosis scanning and cardiac applications (Fig. 1). Although the skill gap was higher in the 2019 cohort compared to the 2016 cohort for internal jugular vein, splenomegaly assessment, and abdominal free fluid, these differences were not statistically significant once adjusted for multiple comparisons.
Perceived applicability, self-reported skills, and skill gap in ultrasound-guided procedures
Participants in both cohorts perceived that ultrasound-guided procedures were moderately-to highly applicable to the practice of internal medicine (Table 1). Both cohorts reported the highest applicability for paracentesis, central venous catheterization, and thoracentesis. Learners continued to report low skills in all ultrasound-guided procedures (Table 1). No significant differences in learners’ skill gaps were identified over time (Fig. 2). The largest perceived skill gap was reported in thoracentesis for both cohorts.
Self-reported knowledge in ultrasound
Participants in both cohorts reported low knowledge on key ultrasound knowledge domains (Table 3), with a mean knowledge score of less than 3 for all items.
Discussion
Our follow-up survey conducted three years after our initial needs assessment survey in 2016 continued to show a significant gap between perceived applicability and self-reported skill and knowledge in many ultrasound diagnostic applications and procedures for PGY-1 Internal Medicine residents at three Canadian centers. This gap was present even for basic foundational POCUS competencies such as basic imaging modes, knobology, and ultrasound safety principles (e.g. ALARA). With the exception of an increase in the perceived applicability of as well as self-reported skills in identifying pulmonary B lines in the 2019 cohort compared to the 2016 cohort, we found no significant differences otherwise noted over time. Specifically, junior learners continued to perceive ultrasound to be highly applicable to the practice of internal medicine, but upon entry into residency, continued to report limited skills across all aspects of POCUS.
Since 2016, concerted efforts had been made at improving internal medicine POCUS knowledge and skills across Canada, including published consensus-based curriculum and education indicators [5, 7]. Concurrently, ultrasound education has also increased at the medical school level in Canada [10, 11]. For example, the University of Calgary introduced ultrasound into physical examination and anatomy teaching in 2012 and 2014, respectively [24]. McGill University started its comprehensive longitudinal POCUS curriculum in 2013 and expanded its simulation-based POCUS training in 2017 [25]. The University of Saskatchewan, since 2018, has published on a number of their innovative medical student POCUS education initiatives [26, 27]. More recently the University of Ottawa published the results of on their pre-clerkship POCUS training [28]. With this increase incorporation of POCUS training in Canadian medical schools, we hypothesized that more recent cohorts of learners would enter residency with higher POCUS knowledge and skills. Contrary to our expectations, learners entering internal medicine residency not only continued to have low perceived POCUS knowledge and skills in applications specific to the practice of internal medicine, but also reported low skills in POCUS basics. Our findings argue against an attempt to eliminate basic POCUS concepts and skills from the internal medicine residency POCUS curriculum. These high learning POCUS needs in our participants are similar to those of other learners in internal medicine [16,17,18,19,20,21,22]. However, to our knowledge, there are currently no follow-up studies that evaluate changes in learning needs over time, thus it remains unknown whether the lack of improvement in reported baseline POCUS knowledge and skills is a finding unique to our three Canadian centers or that this finding is also present elsewhere.
Limitations of our study include the following: first, not all learners responded (response rate of 60 %). However, this response rate is comparable to, if not higher than, prior ultrasound needs assessment studies [16, 20, 29,30,31], and is in keeping with prior studies, where an overall response rate of 30–40 % were noted [32, 33]. Second, our results are based on self-reported learner measures, which are subject to bias and inaccuracies [34]. Learners may in fact know more (or less) than they report. However, even if our participants were under-estimating their self-reported skills and knowledge, the fact that learners feel their skills are limited mandates an educational response from the program, at least for skills where competencies are an expectation. Third, our survey distribution periods were not identical. Specifically, surveys distributed to in 2019 were earlier in the academic year (August – October) than for the 2016 cohort, where many of the surveys were completed towards the end of the learners’ academic year. Thus, the lower level of procedural experience reported by the participants in the 2019 cohort may be a reflection of less clinical exposure. Therefore, these differences should be interpreted with caution. Nonetheless, because the level of experience, knowledge, and skills of learners at the start of the training program, rather than at the end of the year, determines curriculum development efforts, our results are nonetheless helpful. Fourth, the lack of knowledge/skill increase over time may be a result of a number of factors, such as an insufficient or ineffective medical school curriculum, insufficient time for follow-up, or that the medical school POCUS curriculum is more targeted towards certain specialties, such as emergency medicine. While our study is unable to identify which factor(s) is/are responsible, it is highly probable that despite a general increase in medical school POCUS educational efforts, basic foundational concepts may not have been consistently or sufficiently taught. For example, at one site, knobology was noted to be a potential barrier to learning, despite its inclusion in the curriculum [24]. Future studies should evaluate not only the quantity, but the content and quality of POCUS education at the medical school level. Finally, our original 2016 survey had data from four training programs. Due to logistic reasons, this present study only captured data from three of the four sites. Generalizability of our study’s conclusions may therefore be limited.
Conclusions
Our multi-center survey study shows that 2019 PGY-1 Canadian internal medicine residents rate diagnostic and procedural POCUS skills as highly applicable, but report low skill levels, at comparable levels to survey participants from an earlier cohort in 2016. Despite the perceived widespread integration of POCUS in medical schools, there continues to be a large POCUS skills and knowledge gap. Therefore, POCUS educators should not yet assume any significant learner baseline POCUS knowledge or skills when developing their internal medicine POCUS curriculum.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.
Abbreviations
- ALARA:
-
As Low As Reasonably Achievable
- CIMUS:
-
Canadian Internal Medicine Ultrasound
- POCUS:
-
Point-of-care ultrasound
- PGY:
-
Postgraduate year
- SD:
-
Standard deviation
References
Soni NJ, Schnobrich D, Matthews BK, et al. Point-of-care ultrasound for hospitalists: A position statement of the Society of Hospital Medicine. J Hosp Med 2019;14:E1-E6.
American College of Physicians. ACP Statement in Support of Point-of-Care Ultrasound in Internal Medicine. 2018. https://www.acponline.org/meetings-courses/focused-topics/point-of-care-ultrasound/acp-statement-in-support-of-point-of-care-ultrasound-in-internal-medicine. Accessed 22 Jul 2021.
Torres-Macho J, Aro T, Bruckner I, et al. Point-of-care ultrasound in internal medicine: A position paper by the ultrasound working group of the European federation of internal medicine. Eur J Intern Med 2020;73:67–71.
LoPresti CM, Jensen TP, Dversdal RK, Astiz DJ. Point of care ultrasound for internal medicine residency training: A position statement from the Alliance of Academic Internal Medicine. Am J Med. 2019;132(11):1356–1360.
Ma IWY, Arishenkoff S, Wiseman J, et al. Internal medicine point-of-care ultrasound curriculum: Consensus recommendations from the Canadian Internal Medicine Ultrasound (CIMUS) Group. J Gen Intern Med 2017;32:1052–1057.
Brown GM, Otremba M, Devine LA, Gray C, Millington SJ, Ma IWY. Defining competencies for ultrasound-guided bedside procedures: consensus opinions from Canadian physicians. J Ultrasound Med. 2016;35:129–41.
Ambasta A, Balan M, Mayette M, et al. Education indicators for internal medicine point-of-care ultrasound: a consensus report from the Canadian Internal Medicine Ultrasound (CIMUS) Group. J Gen Intern Med 2019;34:2123–2129.
Wong J, Montague S, Wallace P, et al. Barriers to learning and using point-of-care ultrasound: a survey of practicing internists in six North American institutions. Ultrasound J. 2020;12:19.
Steinmetz P, Dobrescu O, Oleskevich S, Lewis J. Bedside ultrasound education in Canadian medical schools: A national survey. Can Med Educ J. 2016;7(1):e78-e86.
Ma IWY, Steinmetz P, Weerdenburg K, et al. The Canadian medical student ultrasound curriculum. J Ultrasound Med. 2020;39(7):1279–1287.
Hoffmann B, Blaivas M, Abramowicz J, et al. Medical student ultrasound education, a WFUMB position paper, part II. A consensus statement of ultrasound societies. Med Ultrason 2020;22:220–229.
Dinh VA, Fu JY, Lu S, Chiem A, Fox JC, Blaivas M. Integration of ultrasound in medical education at United States medical schools: A national survey of directors’ experiences. J Ultrasound Med. 2016;35:413–419.
Nicholas E, Ly AA, Prince AM, Klawitter PF, Gaskin K, Prince LA. The current status of ultrasound education in United States medical schools. J Ultrasound Med. 2021; doi: https://doi.org/10.1002/jum.15633
Baltarowich OH, Di Salvo DN, Scoutt LM, et al. National ultrasound curriculum for medical students. Ultrasound Q. 2014;30:13–19.
Watson K, Lam A, Arishenkoff S, et al. Point of care ultrasound training for internal medicine: a Canadian multi-centre learner needs assessment study. BMC Med Educ. 2018;18:217.
LoPresti CM, Boyd JS, Schott C, et al. A national needs assessment of point-of-care ultrasound training for hospitalists. Mayo Clin Proc. 2019;94:1910–2.
Elhassan M, Gandhi KD, Sandhu C, Hashmi M, Bahl S. Internal medicine residents’ point-of-care ultrasound skills and need assessment and the role of medical school training. Adv Med Educ Pract 2019;10:379–386.
Kessler C, Bhandarkar S. Ultrasound training for medical students and internal medicine residents—A needs assessment. J Clin Ultrasound. 2010;38:401–408.
Anstey JE, Jensen TP, Afshar N. Point-of-care ultrasound needs assessment, curriculum design, and curriculum assessment in a large academic internal medicine residency program. South Med J.2018;111:444–448.
Olgers TJ, ter Maaten JC. Point-of-care ultrasound curriculum for internal medicine residents: what do you desire? A national survey. BMC Med Educ. 2020;20:30.
Mahmood N, Souleymane M, Rajendram R, Ghazi AMT, Kharal M, AlQahtani M. Focused cardiac ultrasound is applicable to internal medicine and critical care but skill gaps currently limit use. J Saudi Heart Assoc 2020;32:464–471.
Souleymane M, Rajendram R, Mahmood N, Ghazi AMT, Kharal YMS, Hussain A. A survey demonstrating that the procedural experience of residents in internal medicine, critical care and emergency medicine is poor: training in ultrasound is required to rectify this. Ultrasound J. 2021;13:20.
Jafari M, Ansari-Pour N. Why, when and how to adjust your P values? Cell J 2019;20:604–607.
Jamniczky HA, McLaughlin K, Kaminska ME, et al. Cognitive load imposed by knobology may adversely affect learners’ perception of utility in using ultrasonography to learn physical examination skills, but not anatomy. Anat Sci Educ 2015;8:197–204.
Le C-K, Lewis J, Steinmetz P, Dyachenko A, Oleskevich S. The use of ultrasound simulators to strengthen scanning skills in medical students: A randomized controlled trial. J Ultrasound Med. 2019;38(5):1249–1257.
Olszynski P, Anderson J, Trinder K, Domes T. Point-of-care ultrasound in undergraduate urology education: A prospective control-intervention study. J Ultrasound Med. 2018;37:2209–2213.
Olszynski P, Russell M, Neufeld A, Malin G. The clinical ultrasonography elective in clerkship (CUSEC): A pilot elective for senior clerkship students at the University of Saskatchewan. Can Med Educ J. 2020;11(1):e144-e146.
Khoury M, Fotsing S, Jalali A, Chagnon N, Malherbe S, Youssef N. Preclerkship point-of-care ultrasound: Image acquisition and clinical transferability. J Med Educ Curric Dev 2020;7:2382120520943615.
Reaume M, Siuba M, Wagner M, Woodwyk A, Melgar TA. Prevalence and scope of point-of‐care ultrasound education in internal medicine, pediatric, and medicine‐pediatric residency programs in the United States. J Ultrasound Med. 2019;38:1433–1439.
Smallwood N, Matsa R, Lawrenson P, Messenger J, Walden A. A UK wide survey on attitudes to point of care ultrasound training amongst clinicians working on the Acute Medical Unit. Acute Med 2015;14:159–164.
Schnobrich DJ, Gladding S, Olson APJ, Duran-Nelson A. Point-of-care ultrasound in internal medicine: a national survey of educational leadership. J Grad Med Educ 2013;5(3):498–502.
Cunningham CT, Quan H, Hemmelgarn B, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol 2015;15:32.
Cook C, Heath F, Thompson RL. A Meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Meas. 2000;60:821–836.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 2006;296:1094–1102.
Acknowledgements
The authors wish to thank all trainees who completed the survey.
Funding
This study was unfunded. Open access publication costs are funded by the John. A. Buchanan Chair of General Internal Medicine. The funder has no role in the study design, data collection, data analysis, data interpretation, writing of the manuscript, or the decision to publish.
Author information
Authors and Affiliations
Contributions
TC contributed to the study design, data acquisition, analysis and interpretation, prepared Figs. 1 and 2, and writing and critical revision of the manuscript. AL and SH contributed to the study design, data acquisition and interpretation, and critical revisions of the manuscript. KW contributed to conception and design, data interpretation, and critical revisions of the manuscript. IWYM contributed to conception and design, data acquisition, analysis and interpretation, and writing and critical revisions of the manuscript. All authors reviewed and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Implied consent was obtained for all participants included in this study. This study was approved by the University of Calgary Conjoint Health Research Ethics Board (REB15-2054), the University of Alberta Research Ethics Board (Pro00091211_REN1), and the Ottawa Health Science Network Research Ethics Board (Protocol 20190406-01 H). Informed consent was obtained from all subjects. All subjects were over the age of 18. All methods were performed in accordance with the relevant guidelines and regulations.
Consent for publication
Not applicable.
Competing interests
IWM is funded as the Chair Holder of the John A. Buchanan Chair of General Internal Medicine, University of Calgary. All other authors confirm no conflicts of interest or competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
About this article
Cite this article
Chahley, T., Lam, A.W., Halman, S. et al. First year internal medicine residents’ self-report point-of-care ultrasound knowledge and skills: what (Little) difference three years make. BMC Med Educ 21, 476 (2021). https://doi.org/10.1186/s12909-021-02915-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12909-021-02915-1
Keywords
- Point-of-care ultrasound
- Education needs assessment
- Curriculum development
- Internal medicine