- Research article
- Open Access
Point of care ultrasound training for internal medicine: a Canadian multi-centre learner needs assessment study
BMC Medical Education volume 18, Article number: 217 (2018)
Significant gaps currently exist in the Canadian internal medicine point-of-care ultrasound (POCUS) curriculum. From a learner’s perspective, it remains unknown what key POCUS skills should be prioritized. This needs assessment study seeks to establish educational priorities for POCUS for internal medicine residents at five Canadian residency training programs.
All internal medicine trainees [postgraduate year (PGY) 1–5] from five internal medicine residency training programs in Canada (n = 598) were invited to complete an online survey on 15 diagnostic POCUS applications, 9 bedside procedures, and 18 POCUS knowledge items. For POCUS applications and procedures, participants were asked how applicable they are to patient care in internal medicine and the participants’ reported skills in those domains. Self-reported knowledge and skills were rated on a 5-point Likert scale, where 1 = very poor and 5 = very good. Applicability was rated, where 1 = not at all applicable and 5 = very applicable.
A total of 253 of 598 residents (42%) participated in our study. Data from one centre (n = 15) was removed because of low response rate (15%) and significant baseline differences between those trainees and the remaining participants. Of the remaining analyzable data from four training programs (n = 238), participants reported highest applicability to internal medicine for the following applications and procedures: identifying ascites/free fluid [mean applicability score of 4.9 ± standard deviation (SD) 0.4]; gross left ventricular function (mean 4.8 ± SD 0.5) and pericardial effusion (mean 4.7 ± SD 0.5); thoracentesis (mean score 4.9 ± SD 0.3), central line insertion (mean 4.9 ± SD 0.3), and paracentesis (mean 4.9 ± SD 0.3), respectively. Overall reported knowledge/skills was low, with skill gaps being the highest for identifying deep vein thrombosis (mean gap 2.7 ± SD 1.1), right ventricular strain (mean 2.7 ± SD 1.1), and gross left ventricular function (mean 2.7 ± SD 1.0).
Many POCUS applications and procedures were felt to be applicable to the practice of internal medicine. Significant skill gaps exist in the four Canadian training programs included in the study. POCUS curriculum development efforts should target training based on these perceived skill gaps.
There is an increasing recognition of the value of point-of-care ultrasound (POCUS) in the practice of internal medicine. Its use in the guidance of procedures, such as central venous catheterization and thoracentesis, has been shown to improve patient safety [1, 2] and is considered to be the standard of care [2,3,4,5,6,7,8,9]. Additionally, the use of ultrasound as an adjunct to the physical examination to clarify clinical findings [10,11,12] and its ability to identify important clinical conditions at the bedside in the acutely ill patient is also increasingly recognized .
Appropriate training is integral for incorporating POCUS into the practice of internal medicine. POCUS involves a complex set of skills, including image acquisition, image interpretation, and integration of findings that require consideration of the patient’s clinical context and pre-test probability [14, 15]. To practice POCUS safely, the trainee must have sufficient medical and sonographic knowledge of possible differential diagnoses , as well as an awareness and insight into the limitations of both POCUS use in general and one’s own skill limitations. Given this complexity, it is not surprising the performance of POCUS is highly operator dependent and that the learning curves differ significantly depending on the application and the learner in question [17,18,19]. In the hands of untrained or inadequately trained POCUS operators, medical errors may pose significant safety concerns for the patient . A growing body of evidence indicates that dedicated training is indeed necessary to attain proficiency in POCUS [5, 18, 21] and that training may mitigate against potential harm from common POCUS pitfalls [22, 23]. Targeting skills and knowledge gaps is therefore critical to POCUS education.
For curriculum development, performing a needs assessment is one of the key recommended first steps . From a curriculum development standpoint, consensus was recently established regarding what POCUS elements should be included in a Canadian internal medicine training curriculum [25, 26]. However, this work stemmed from the viewpoint of the educator. Little is known from the trainees’ perspective, both in terms of what they feel are needed skills for internal medicine and what skill gaps exist. For adult learners, recognition of their perspectives and needs is critical for curriculum success [24, 27].
Literature to date suggests gaps exist in POCUS training. One study involving learners at the University of Illinois at Chicago and Northwestern University shows that learners felt generally incompetent in the use of ultrasound .These results are not surprising, considering that, based on results from a survey administered to internal medicine Program Directors in 2012, only 25% reported having a formal POCUS curriculum . Significant training gaps are similarly present in Canada . Specifically, while 100% of the internal medicine Canadian residency and fellowship program directors believed that POCUS should be taught and used, only 53% had actually integrated it into their residency-training programs .
The aim of this study is to establish the current reported skill levels and perceived POCUS needs of internal medicine trainees at five Canadian teaching sites with regard to POCUS. The results from this study will help prioritize training needs for educators tasked with POCUS curriculum development and implementation.
This multi-center cross-sectional survey study was undertaken at five Canadian universities: the Universities of Calgary, Alberta, British Columbia, Ottawa and Western University. The respective Ethics Board at each of the five universities approved this study.
All Internal Medicine residents (postgraduate year [PGY] 1–4), as well as General Internal Medicine subspecialty trainees (PGY 4–5) during the year 2015–2016 from the five institutions were invited to participate in this study (n = 598). Only those who consented to the survey were included in the study.
To support the content development of our survey, key texts and articles on POCUS were reviewed [31,32,33,34,35]. An initial survey was drafted with input from two researchers (KW and IM) in July 2015, containing a list of 15 diagnostic applications, 10 procedures, and 80 basic knowledge items. For each survey item on diagnostic applications and procedures, two questions were asked: 1) How applicable is the application/procedure to patient care in internal medicine? 2) What is the participant’s skill in that area? For knowledge items, only self-reported level of knowledge was asked.
This survey was then piloted with 8 non-internal medicine residents in order to obtain input on survey length, content, and clarity. Based on feedback from the pilot data, in particular with respect to the length of the initial survey, we substantially revised the survey. For diagnostic applications, items on A-lines and Z-lines were removed, as they were felt to be too specific. The addition of two diagnostic applications was suggested: deep vein thrombosis and hydronephrosis. For procedures, incision and drainage was removed as the skills involved were felt to be redundant with the skills involved in abscess aspiration. Lastly, many of the 80 items on POCUS knowledge  were felt to be too specific, resulting in a survey that was unacceptably long. Ultimately, knowledge items were grouped into broader categories. The final survey included 15 diagnostic applications, 9 procedures, and 18 knowledge items, in addition to questions on baseline demographic data (see Additional file 1).
Using an online survey tool (SurveyMonkey Inc. San Mateo, California, USA; www.surveymonkey.com), the final survey was distributed to the trainees between April and June 2016. Up to two reminder emails were sent between two and 8 weeks to maximize participant response rate. As this study was unfunded, no incentives were used in this study at any study site.
Perceived applicability of diagnostic applications and procedures to the practice of internal medicine was assessed using a 5-point Likert scale, where 1 = not at all applicable and 5 = very applicable. Self-reported skill level and knowledge was assessed using a 5-point Likert scale, where 1 = very poor and 5 = very good. We defined skill gap as the difference between perceived applicability and self-reported skill level.
Data were analyzed using standard descriptive statistical techniques. Comparisons of continuous variables between groups were performed using Student’s t-tests. Categorical variables were compared with the use of Fisher’s exact tests and chi-square tests. All analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA).
A total of 253 of 598 residents participated in our study for a cumulative response rate across all centers of 42%. We excluded data from 15 participants in a single center, as this center (Center E) did not achieve a response rate of > 40% (achieved only 12%). Further, baseline demographics of survey respondents from Center E differed significantly from respondents from Centers A to D in terms of procedural experience and in reported access to ultrasound preceptors (Table 1). The 4 centers included in this study had a mean response rate of 51 ± 10%, with a total of 238 participants included in our final analyses.
Participants felt the following three diagnostic uses of ultrasound were most applicable to patient care in internal medicine: identifying ascites/free fluid [mean applicability score of 4.9 ± standard deviation (SD) 0.4]; gross left ventricular function (mean 4.8 ± SD 0.5) and pericardial effusion (mean 4.7 ± SD 0.5, Fig. 1). Participants reported lowest skill levels in identifying deep vein thrombosis (mean skill level 1.7 ± SD 0.8), hydronephrosis (mean 1.7 ± SD 0.8), and pulmonary interstitial syndrome (mean 1.8 ± SD 0.9). Skill gaps (difference between perceived applicability and self-reported skill level) were highest for identifying deep vein thrombosis (mean gap 2.7 ± SD 1.1), right ventricular strain (mean 2.7 ± SD 1.1), and gross left ventricular function (mean 2.7 ± SD 1.0).
Residents identified the following ultrasound-guided procedures as most applicable to internal medicine: thoracentesis (mean applicability score 4.9 ± SD 0.3), central line insertion (mean 4.9 ± SD 0.3), and paracentesis (mean 4.9 ± SD 0.3, Fig. 2). Participants reported lowest skill levels in using ultrasound for: peripherally inserted central catheter (mean skill level 1.6 ± SD 0.8), lumbar puncture (mean 1.8 ± 3.9), and superficial abscess aspiration (mean 2.1 ± 0.9). Skill gaps were highest for lumbar puncture (mean gap 2.1 ± SD 1.4), joint aspiration (mean 2.0 ± 1.3), and peripherally inserted central catheter (mean 1.8 ± SD 1.3).
Participants reported highest knowledge levels in: sterile transducer techniques (mean skill level 3.6 ± SD 1.3), transducer selection (mean 3.3 ± SD 1.2), and ability to interpret pulmonary findings (mean 2.5 ± SD 1.9, Fig. 3). Participants reported lowest knowledge levels in: power Doppler imaging (mean 1.6 ± SD 0.7), continuous wave spectral Doppler imaging (mean 1.6 ± SD 0.7), and pulsed wave spectral Doppler imaging (mean 1.6 ± SD 0.8).
This study demonstrates the perceived applicability to the practice of internal medicine, self-reported skill/knowledge level, and skill gap of 15 POCUS diagnostic applications, 9 ultrasound-guided procedures, and 18 POCUS knowledge items for Canadian internal medicine residents at four training sites. Our results suggest that learners felt using ultrasound to identify ascites/abdominal free fluid, gross left ventricular function and pericardial effusion were most applicable to the practice of internal medicine while least skillful in assessing for deep vein thrombosis and focused cardiac ultrasound assessments. POCUS was felt to be most applicable for the guidance for thoracentesis, central line insertion and paracentesis. Self-reported proficiency was lowest in peripherally inserted central catheter, lumbar puncture, and superficial abscess drainage. Lastly, learners reported overall low level of knowledge of POCUS, especially with respect to Doppler imaging.
Our results mirror those from the University of Illinois where learners reported low overall competence in POCUS  and the University of Toronto where only 21% of learners reported comfort in using POCUS for procedures . Our present study adds to existing literature by providing additional granularity on gaps in specific ultrasound diagnostic skills, procedural skills and areas of knowledge. Time and costs of training are cited barriers to implementing a POCUS curriculum for internal medicine [29, 30, 36]. These results can help internal medicine educators who are tasked with implementing a POCUS curriculum focus their efforts on applications, procedures, or knowledge that are perceived to be the most applicable, where the learners report the least skills/knowledge, and/or where skill gaps are the highest, which takes into account both applicability and reported levels of skills and knowledge.
Our study has a few limitations. First, our results are based on learner self-report and the accuracy of self-assessed competence in skills and knowledge is questionable [37, 38]. Therefore, curriculum efforts should not necessarily ignore applications, procedures, or knowledge in which learners report high competence. Further, our definition of skill gap is only a surrogate measure, based on the difference between reported applicability and skills. Inaccurate assessment of either (or both) of these variables may render the interpretability of the skill gap measure questionable. Second, although this study provides the learner’s perspective on POCUS, curriculum implementation should take into account other factors, such as which POCUS applications can be mastered within constraints of each local curriculum setting (e.g. time requirement for various skill acquisition and availability of trained preceptors). Time requirements and trained preceptors are important considerations in curriculum design. We recommend that our results be triangulated with educators’ perspectives, which were sought based on clinical and educational needs and evidence, education feasibility, as well as consideration of patient safety issues . These principles may not have been considered by the learners during survey response. Third, contrary to our expectations, our participants felt that the identification of interstitial syndrome  has low applicability to internal medicine. Existing data suggest that lung ultrasound is in fact highly applicable to internal medicine as it improves diagnostic accuracies and narrowing of differential diagnoses [40, 41]. In retrospect, it is likely that the term “interstitial syndrome,” while concordant with international recommendations , may be unfamiliar to our survey participants. The reported low skill level in identifying interstitial syndrome supports our hypothesis that our survey participants may not be familiar with the concept. As such, we would not recommend deferring the training of lung ultrasound to internal medicine residents based on our survey results. Fourth, our surveys were administered to Canadian internal medicine trainees at five training sites only and consequently, generalizability may be limited. Lastly, despite our best efforts, our overall response rate was only 42%, while not entirely out of keeping with physician survey response rate in general [42, 43], is lower than we would like.
Despite these limitations, to our knowledge, this study reports results from the largest multicenter internal medicine learners. As such, their collective experience and opinions regarding skill gaps are important to consider in our efforts to integrate POCUS safely into the practice of internal medicine .
Our multi-center survey results from Canadian internal medicine residents suggest that learners found POCUS highly applicable to the practice of internal medicine, especially for identifying ascites and cardiac findings and for guiding central line insertion, paracentesis, and thoracentesis. Significant gaps were reported in skills and knowledge. Development of POCUS education should take these results into consideration when deciding where to focus curriculum efforts.
Arienti V, Camaggi V. Clinical applications of bedside ultrasonography in internal and emergency medicine. Intern Emerg Med. 2010;6:195–201.
Randolph AG, Cook DJ, Gonzales CA, Pribble CG. Ultrasound guidance for placement of central venous catheters- A meta-analysis of the literature. Crit Care Med. 1996;24:2053–8.
Havelock T, Teoh R, Laws D, Gleeson F. Pleural procedures and thoracic ultrasound: British Thoracic Society Pleural Disease Guideline 2010. Thorax. 2010;65(Suppl 2):61–76.
Rupp SM, Apfelbaum JL, Blitt C, Caplan RA, Connis RT, Domino KB, et al. Practice guidelines for central venous access: a report by the American Society of Anesthesiologists Task Force on Central Venous Access. Anesthesiology. 2012;116:539–73.
American College of Surgeons. Revised statement on recommendations for use of real-time ultrasound guidance for placement of central venous catheters. 2011. https://www.facs.org/about-acs/statements/60-real-time-ultrasound. Assessed 3 Jan 2018.
National Institute for Health and Care Excellence. Guidance on the use of ultrasound locating devices for placing central venous catheters. 2002. https://www.nice.org.uk/guidance/ta49. Assessed 3 Jan 2018.
Feller-Kopman D. Ultrasound-guided central venous catheter placement: the new standard of care? Crit Care Med. 2005;33:1875–7.
Troianos CA, Hartman GS, Glas KE, Skubas NJ, Eberhardt RT, Walker JD, Reeves ST. Guidelines for performing ultrasound guided vascular cannulation: recommendations of the American Society of Echocardiography and the Society Of Cardiovascular Anesthesiologists. J Am Soc Echocardiogr. 2011;114:46–72.
Jensen TP, Soni NJ, Tierney DM, Lucas BP. Hospital privileging practices for bedside procedures: a survey of hospitalist experts. J Hosp Med. 2017;10:836–9.
Low D, Vlasschaert M, Novak K, Chee A, Ma IWY. An argument for using additional bedside tools, such as bedside ultrasound, for volume status assessment in hospitalized medical patients: a needs assessment survey. J Hosp Med. 2014;9:727–30.
Clay RD, Lee EC, Kurtzman MF, Dversdal RK. Teaching the internist to see: effectiveness of a 1-day workshop in bedside ultrasound for internal medicine residents. Crit Ultrasound J. 2016;8:11.
Cessford T, Meneilly GS, Arishenkoff S, Eddy C, Chen LYC, Kim DJ, Ma IWY. Comparing physical examination with sonographic versions of the same examination techniques for splenomegaly. J Ultrasound Med. 2017. https://doi.org/10.1002/jum.14506.
Bhagra A, Tierney DM, Sekiguchi H, Soni NJ. Point-of-care ultrasonography for primary care physicians and general internists. Mayo Clinic Proc. 2016;91:1811–27.
Herrle SR, Corbett EC Jr, Fagan MJ, Moore CG, Elnicki DM. Bayes’ theorem and the physical examination: probability assessment and diagnostic decision-making. Acad Med. 2011;86:618–27.
Goodman SN. Toward evidence-based medical statistics. 2: The Bayes factor. Ann Intern Med. 1999;130:1005–13.
Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede S. The causes of errors in clinical reasoning: cognitive biases, knowledge deficits, and dual process thinking. Acad Med. 2017;92:23–30.
Canadian Association of Radiologists. Position statement on the use of point of care ultrasound. 2013. https://car.ca/wp-content/uploads/CAR-Position-Statement-on-the-Use-of-Point-of-Care-Ultrasound.pdf. Assessed 3 Jan 2018.
Blehar DJ, Barton B, Gaspari RJ. Learning curves in emergency ultrasound education. Acad Emerg Med. 2015;22:574–82.
Gaspari RJ, Dickman E, Blehar D. Learning curve of bedside ultrasound of the gallbladder. J Emerg Med. 2009;37:51–6.
Pinto A, Pinto F, Faggian A, Rubini G, Caranci F, Macarini L, Genovese EA, Brunese L. Sources of error in emergency ultrasonography. Crit Ultrasound J. 2013;5(Supple 1):S1.
Beaulieu Y, Laprise R, Drolet P, Thivierge RL, Serri K, Albert M, Lamontagne A, Belliveau M, Denault AY, Patenaude JV. Bedside ultrasound training using web-based e-learning and simulation early in the curriculum of residents. Crit Ultrasound J. 2015;7:1.
Blanco P, Volpicelli G. Common pitfalls in point-of-care ultrasound: a practical guide for emergency and critical care physicians. Crit Ultrasound J. 2016;8:15.
Orde S, Slama M, Hilton A, Yastrebov K, McLean A. Pearls and pitfalls in comprehensive critical care echocardiography. Crit Care. 2017;21:279.
Kern DE, Thomas PA, Hughes MT. Curriculum Development for Medical Education. A Six-Step Approach. 2nd ed. Baltimore, MD: The Johns Hopkins University Press; 2009.
Ma IWY, Arishenkoff S, Wiseman J, Desy J, Ailon J, Martin L, Otremba M, Halman S, Willemot P, Blouw M. Internal medicine point-of-care ultrasound curriculum: consensus recommendations from the Canadian Internal Medicine Ultrasound (CIMUS) Group. J Gen Intern Med. 2017;32:1052–7.
Brown GM, Otremba M, Devine LA, Gray C, Millington SJ, Ma IWY. Defining competencies for ultrasound-guided bedside procedures: consensus opinions from Canadian physicians. J Ultrasound Med. 2016;35:129–41.
Knowles MS, Swanson RA, Holton EF. The Adult Learner. In: The Definitive Classic in Adult Education and Human Resource Development. 7th ed. Oxford: Elsevier Inc.; 2011.
Kessler C, Bhandarkar S. Ultrasound training for medical students and internal medicine residents— a needs assessment. J Clin Ultrasound. 2010;38:401–8.
Schnobrich DJ, Gladding S, Olson APJ, Duran-Nelson A. Point-of-care ultrasound in internal medicine: a national survey of educational leadership. J Grad Med Educ. 2013;5:498–502.
Ailon J, Nadjafi M, Mourad O, Cavalcanti R. Point-of-care ultrasound as a competency for general internists: a survey of internal medicine training programs in Canada. Can Med Educ J. 2016;7:e51–69.
Noble VE, Lamhaut L, Capp R, Bosson N, Liteplo A, Marx J-S, Carli P. Evaluation of a thoracic ultrasound training module for the detection of pneumothorax and pulmonary edema by prehospital physician care providers. BMC Med Educ. 2009;9:3.
Levitov A, Dallas A, Slonim A. Bedside ultrasonography in clinical medicine. New York: The McGraw-Hill Companies; 2011.
Moore CL, Copel JA. Point-of-care ultrasonography. N Engl J Med. 2011;364:749–57.
Carmody KA, Moore CL, Feller-Kopman D. Handbook of critical care and emergency ultrasound. New York: The McGraw-Hill Companies, Inc.; 2011.
Soni NJ, Arntfield R, Kory P. Point-of-care ultrasound. IPhiladelphia, PA: Saunders; 2015.
Smallwood N, Matsa R, Lawrenson P, Messenger J, Walden A. A UK wide survey on attitudes to point of care ultrasound training amongst clinicians working on the Acute Medical Unit. Acute Med. 2015;14:159–64.
Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296:1094–102.
Colthart I, Bagnall G, Evans A, Allbutt H, Haig A, Illing J, McKinstry B. The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10. Med Teach. 2008;30:124–45.
Volpicelli G, Elbarbary M, Blaivas M, Lichtenstein D, Mathis G, Kirkpatrick A, Melniker L, Gargani L, Noble V, Via G, et al. International evidence-based recommendations for point-of-care lung ultrasound. Intensive Care Med. 2012;38:577–91.
Perrone T, Maggi A, Sgarlata C, Palumbo I, Mossolani E, Ferrari S, Melloul A, Mussinelli R, Boldrini M, Raimondi A, et al. Lung ultrasound in internal medicine: A bedside help to increase accuracy in the diagnosis of dyspnea. Eur J Intern Med. 2017;46:61–5.
Filopei J, Siedenburg H, Rattner P, Fukaya E, Kory P. Impact of pocket ultrasound use by internal medicine housestaff in the diagnosis of dyspnea. J Hosp Med. 2014;9:594–7.
Flanigan TS, McFarlane E, Cook S. Conducting survey research among physicians and other medical professionals: a review of current literature. In: Proceedings of the Survey Research Methods Section, American Statistical Association; 2008. p. 4136–47.
Wiebe ER, Kaczorowski J, MacKay J. Why are response rates in clinician surveys declining? Can Fam Physician. 2012;58:e225–8.
The authors wish to thank all trainees who completed the survey and Dr. Mireille Sayegh for her assistance with the drafting and distribution of the surveys for the University of Ottawa.
This study was unfunded.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available but are available from the corresponding author on reasonable request.
Ethics approval and consent to participate
Implied consent was obtained for all participants included in this study. This study was approved by the University of Calgary Conjoint Health Research Ethics Board (Ethics ID number: REB 15–2054), University of Alberta Research Ethics Office (ID: Pro00058881), University of British Columbia Behavioural Research Ethics Board (H15–01955), Ottawa Health Science Network Research Ethics Board (Protocol #20160025-O1H), and Western University Office of Research Ethics (HSREB 107616).
Consent for publication
SA has received stock options for an advisory role in Clarius, outside the submitted work. The remaining authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Watson, K., Lam, A., Arishenkoff, S. et al. Point of care ultrasound training for internal medicine: a Canadian multi-centre learner needs assessment study. BMC Med Educ 18, 217 (2018). https://doi.org/10.1186/s12909-018-1326-8
- Point-of-care ultrasound
- Education needs assessment
- Curriculum development
- Internal medicine