Skip to main content

An educational intervention to facilitate appropriate subspecialty referrals: a study assessing resident communication skills



Our goal was to improve pediatric residents' advanced communication skills in the setting of referral to address the entrustable professional activity of subspecialty referral identified by the American Board of Pediatrics. To accomplish this aim, we created a referral and consultation curriculum to teach and assess core communication skills in subspecialty referral involving an adolescent with syncope, an anxiety-provoking symptom that is rarely associated with serious pathology.


We utilized blended multimodal educational interventions to improve resident communication skills in referral of patients. Trainees participated in 1) an interactive online module on syncope focusing on “red-flag” symptoms that would warrant a subspecialty cardiology referral and 2) a 4-h intervention with Standardized Parents (SPs), focusing on the case-based application of communication skills. Communication skills were assessed by two pre- and post- Objective Structured Clinical Examination encounters of patients with syncope, with an SP evaluation using a 20-item checklist. Analysis was performed with Sign test and McNemar’s test. Trainees provided feedback on a Critical Incident Questionnaire, which was analyzed qualitatively.


Sixty-four residents participated. There was an overall improvement in communication skills based on SP scores (82.7 ± 10.9% to 91.7 ± 5.0%, p < 0.001), and 13/20 items demonstrated significant improvement post-intervention. Residents’ improved performance enabled them to address patient/family emotions, explain referral logistics, and clarify concerns to agree on a plan.


By participating in this curriculum, residents’ communication skills improved immediately post-intervention. Further research is needed to assess if this intervention improves patient care by providing residents with enduring skills to judiciously manage the referral process.

Peer Review reports


Focused subspecialty referral and appropriate consultation are critical skills in providing high-quality care to patients requiring further expertise for addressing potentially serious concerns [1, 2]. Clinicians must recognize red-flag symptoms requiring consultation and subspecialty referral. As importantly, they must demonstrate listening and communication skills to identify the underlying concerns around these issues to address these fears and reassure when appropriate. Primary care pediatricians are often the first to assess children presenting with anxiety-provoking complaints such as syncope. Most children with syncope have no red-flag symptoms meriting diagnostic evaluation or referral, and only 2% have an underlying cardiac etiology [3, 4]. Substantiating this low yield of further cardiac evaluation, 60% of children who were referred to Cardiology for benign simple vasovagal syncope had no medical red-flag criteria for referral [5]. Inappropriate referral can result in unnecessary testing, unjustified use of available resources, and unwarranted financial costs, some of which can be attributed to parental or patient anxiety or demand [6, 7]. Thus, residency training curricula on advanced communication skills are needed to enhance future pediatricians’ skills to manage subspecialty referrals optimally and appropriately.

The American Board of Pediatrics (ABP) recognize referral of patients who require subspecialty consultation as an entrustable professional activity (EPA) and an essential component of a pediatric residency curriculum [8]. To improve pediatric resident confidence and performance in syncope referral, the Children’s National Hospital (CNH) Syncope Education Project was created [9]. During the pilot project, participating pediatric residents on their cardiology rotation completed a 90-min mid-rotation educational workshop focusing on red-flag criteria for cardiology referral and practiced with standardized parents (SPs). Residents exhibited significant improvements in self-efficacy as well as SP rating of their communication [9]. In the current study, we expanded the curriculum to focus on advanced communication skills; for example, offering reassurance to overly concerned parents or negotiating with parents who are not convinced of the need for urgent referral or exercise restriction. Setting the expectation for residents to justify the decision for referral or no referral was an intentional design choice in order to assess educational outcomes on a higher level on traditional taxonomies of clinical assessment. Principles behind our educational intervention include Bloom’s Taxonomy, Miller’s Triangle, and Kirkpatrick’s Four Levels of Training Evaluation [10,11,12].

Our study hypothesis was that a blended, multimodal interactive educational intervention involving an online learning module followed by a workshop utilizing SP encounters will improve pediatric resident subspecialty referral communication skills in counseling a concerned parent. Our aim was to improve residents' advanced communication skills in the setting of referral assessed by SP ratings and create a referral curriculum that is applicable to a broad range of clinical scenarios.


Settings and participants

We implemented a blended multimodal educational intervention to improve pediatric resident subspecialty referral skills. This intervention included an interactive, online module and an in-person, half-day workshop with SPs (Fig. 1). Residents rotating on electives were recruited to participate during academic years 2015 to 2019, when we conducted the study. Since those residents did not have a primary role in covering patients, no cross coverage for patient care was needed. Participation was voluntary, results were kept private and not shared with the program director, and performance on the activity did not impact standing in the program. The study was conducted at a tertiary pediatric center with 120 pediatric residents, with 40 residents in each of three training years.

Fig. 1
figure 1

Educational Curriculum. OSCE: Objective Structured Clinical Examination, SP: Standardized parent

Resources: involvement of standardized parents and faculty

Two SPs were hired for a 4-h timeframe to work with residents as well as for a 2–4 h curriculum-specific training session; over the course of the study, five SPs were utilized in total. Each session could accommodate up to 4 residents, with one half-day session per month. The costs associated with this program included $2250 for consultation, curriculum development support, staff time, and use of the CNH Simulation Center. Additionally, costs of SPs were ~ $25/hour. A faculty member (author ASH) spent 3% FTE each month supervising these sessions and facilitating the workshop.

Training of standardized parents

SPs were trained by an SP Educator (author KL) from the Clinical Learning & Simulation Skills (CLASS) Center at The George Washington School of Medicine and Health Sciences. This training included learning scenarios that addressed the need for referral. As part of these scripts, authors KL and ASH coached the SPs in how to behave emotionally within the context of each scenario and how to respond to residents’ potential comments and questions. To pilot the study and determine if there were gaps that needed addressing before beginning interactions with residents, a practice session was coordinated with a senior Cardiology fellow simulating a resident to train the SPs and fine-tune their responses. The fellow and SPs practiced and discussed responses until SPs had standardized responses. Training design and implementation followed the Association of Standardized Patient Educators (ASPE) Standards of Best Practice (SOBP) [13]. The SPs at the CLASS Center routinely participate in high stakes sessions with George Washington University medical students in addition to curricula such as ours; SPs routinely roleplay and then provide feedback in each of these scenarios.

Specifics of educational curriculum

The interactive, online module (which can be viewed at provided education on the medical concern of syncope, particularly about “red-flag” symptoms that would point towards a cardiac etiology of syncope (i.e. exertional syncope, history of sensorineural hearing loss, family history of sudden death, etc.) [5, 14]. The module was created using Articulate Storyline in collaboration with instructional design expertise at the Children’s National Office of Medical Education. Additionally, this module reviewed how to educate patients and families on benign vasovagal syncopal episodes and what steps to take for the future if the patient feels faint, such as counter-pressure exercises [15]. This module required answering questions before progressing, such as indicating what you would ask a patient on initial presentation or matching symptoms with a particular type of syncope. Residents were tasked with completing the 20-min interactive online module the night prior to the in-person communication skills workshop.

For the half-day in-person workshop, trainees participated in two pre-intervention Objective Structured Clinical Examination (OSCE) cases, one requiring subspecialty cardiology referral and the other not requiring referral, as well as two post-intervention OSCEs with the same themes. The OSCE was designed by authors ASH, LG, and KL. Regardless of the need for referral, the SPs related concerns that the trainee was tasked with resolving during the encounter. For example, the SPs requiring referral had reasons why it might be hard to attend the subspecialty appointment or to limit exercise (i.e. their child had an upcoming athletic competition that could result in a college scholarship), requiring a thoughtful response to persuade them of the severity of the presenting problem and urgency for referral. On the other hand, for the case in which referral was unnecessary, the SPs were instructed to ask challenging questions, such as: “I just read an article about sudden death in an athlete—what are the chances my child will be the next one to die on the field?” Using a 20-question checklist, the SPs assessed the residents in all OSCE encounters, with some questions requiring the SPs to rate participants on a Likert scale from strongly disagree to strongly agree and other questions calling for yes/no responses. The SP checklist was modified from a previously validated SP checklist developed using the Set the stage, Elicit information, Give information, Understand the patient's perspective, and End the encounter (SEGUE) framework, which provides a guideline to evaluation for specific communication tasks in a medical encounter with the goal to achieve high inter-rater reliability between SPs [9, 16]. Utilizing the framework, along with comprehensive training with thoughtful fine-tuning, are both required to attain optimal reliability [16]. In creation of the checklist for this study, there was a focus on using “discrete items” and “clear coding rules” in making the checklist for SP evaluation of residents and referral-specific items were in nominal format [9, 16]. Each SP was trained how to use the SP checklist and given time to ask clarifying questions. As trainees did not obtain a history or perform a physical examination, the door chart of the encounter for the trainee included the history, physical exam findings, diagnosis, and whether or not referral was indicated. Trainees were also provided with a list of the emotions displayed by the family on the door chart, such as “fear” and “surprise” for the patient not requiring referral. The trainees were tasked specifically with counseling the SP within the 10 min encounter about the diagnosis and advised treatment plan.

The interactive workshop also included: (1) review of the principles of referral, (2) a refresher on “red-flag” symptoms of syncope, (3) an interactive communication skills intervention on addressing underlying emotions and concerns and on managing referral logistics, and (4) practice with SPs in a group setting (Fig. 1). During practice with SPs in a group setting, residents watched one of their peers practice the challenging components of the referral conversation with an SP in front of the group and received immediate feedback.


After residents completed the pre-test OSCEs, the SPs provided individual and group feedback about performance, specifically remarking on positive behaviors that were observed. The faculty used these comments to make teaching points. After completion of all OSCEs, SPs and faculty provided feedback to any resident interested in further feedback.

Trainees provided feedback on the curriculum by completing the Brookfield Critical Incident Questionnaire at the end of their SP interaction to characterize which factors enhanced or hindered learning [17]. These qualitative comments were categorized, enumerated, and summarized independently by three co-authors (EAS, LG, and DA) to identify key themes. The themes with the highest number of representative comments were included in Table 4.

Institutional review board approval

The Children’s National Hospital Institutional Review Board approved this study with written informed consent obtained from all participants.

Statistical analysis

Statistical tests for paired data were utilized for all analyses. Analysis of the SP checklist results was performed with the Sign test and McNemar’s test, with the Sign test utilized for questions that were graded by SPs on a Likert scale from “strongly agree” to “strongly disagree”. Additionally, Likert scores were converted from 1–5 to 20–100 with 1 corresponding to 20, 2 corresponding to 40, and so on, in order to report data out of 100%. Since they were paired discrete variables with limited variability, the Sign test, a non-parametric alternative to the paired t-test, was utilized. McNemar’s test was used for items graded by SPs on a binary scale (yes/no) to evaluate for pre-post differences. A total percent score was calculated as a composite of the average of all SP items where binary variables were converted to 0/100. This total score was analyzed by paired t-test as it was normally distributed. A Cronbach alpha score was also calculated for the SP checklist to assess reliability. Demographics were analyzed by simple linear regression analysis to determine if any were associated with the difference between pre- and post-intervention total scores.


Sixty-four trainees volunteered to participate in the study; 54 participants completed a demographic questionnaire, of whom 74% were women (Table 1).

Table 1 Participant demographics

Significant improvement was seen in the SPs’ rating of resident communication skills pre-intervention compared to post-intervention (82.7 ± 10.9% versus 91.7 ± 5.0%, p < 0.001) (Table 2). A Cronbach alpha score was calculated to evaluate checklist reliability (Cronbach alpha = 0.71). Significant improvement was noted in 13 of 20 items on the SP checklist, including but not limited to the following checklist items: “The resident asked me what was the most concerning factor for us today”, and “The resident noted that I seemed sad/mad/distressed/worried”. The trainees improved in 4 of 7 referral-specific metrics, including: “the resident explained what you could expect when you see the Cardiologist,” which increased from 51.9% to 75% post-intervention (p = 0.01). For OSCEs aimed to reassure the family that referral was not indicated, a significant improvement was noted for this checklist item: “The resident successfully helped you to feel reassured that this is not a life-threatening condition” (pre-post scores of 82.6% and 97.8%, respectively, p = 0.008).

Table 2 Pre- and post-intervention results for standardized parent Objective Structured Clinical Examination (OSCE) by checklist item

Analysis of demographic variables to assess for possible associations with difference in score pre- and post-intervention showed no association of year of workshop, sex, age, level of training, prior communication workshop in residency or medical school, or experience with patients presenting with syncope (Table 3).

Table 3 Associations of demographic variables with difference between pre- and post-intervention total scores

Qualitative review of the feedback on the Critical Incident Questionnaire revealed six themes of factors that improved learning, with the three highest including: 1. Timely feedback on performance; 2. Group session observing peers interacting with SPs; and 3. Case discussion (Table 4). Additional areas of value included interactively working with SPs as well as specific teaching on communication skills, including general principles and skills specific to referral. As to activities trainees cited as hindering learning, the main themes were the structure of the session (repetition of cases, filling out forms, and downtime) and learner passivity.

Table 4 Qualitative analysis of feedback on critical incident questionnaire


Participation in this blended, multimodal interactive curriculum improved the immediate, post-intervention communication skills of residents in the setting of subspecialty referral. Residents showed a measurable, significant immediate improvement in most communication areas, including the majority of those focused on recognizing and addressing parent emotions and concerns as well as sharing and managing referral logistics. Of the Kirkpatrick levels of education, this curriculum focuses on level 2C, knowledge, specifically skills acquisition [12, 18]. Additionally, our curriculum meets the level of “evaluate” per Bloom’s Taxonomy and “Shows” in Miller’s Triangle [10, 11]. Residents improved in recognizing and managing logistics of referral, clarifying the plan with the family, ensuring conceptual understanding and agreement on logistics. The authors suggest that the curriculum we developed for our study is generalizable to other clinical scenarios requiring referral and justify broadening this intervention to include other patient scenarios.

It is important that residents were able to identify parental anxiety about an emotionally charged diagnosis and develop competence in communicating with a simulated parent about this issue. Forrest et al. reported 16.7% of pediatric subspecialty referrals were influenced by parental or patient anxiety; moreover, Harahsheh et al. found 14% of low yield echocardiograms were obtained for the same reason [6, 7]. A trusting relationship between the physician and in this case parents is paramount to understanding the need to refer or not and can be conveyed in an evidence-based manner as in our study. Whereas the focus of this study was around the diagnosis of syncope with the goal to improve resource utilization and decrease inappropriate referrals, there are many diagnoses in pediatrics and other specialties in medicine that are similarly concerning to patients and their families in which the curriculum we describe could be utilized to train practitioners.

The strengths of this intervention include demonstration of successful application of learned communication skills, utilization of SPs, and the ability for trainees to participate during elective time. Tasked with counseling SPs about whether or not their child required subspecialty referral, residents were highly successful in explaining this challenging, high stakes topic to families. These valuable skills may be generalizable to other potentially serious medical concerns. Trainee responses indicated what they learned from the experience; for example, one participant shared, “I was most engaged while getting active feedback during the practice scenarios. I felt this was helpful to point ways to improve in connecting with the family to both address their concerns and make sure they understand my recommendations”. Participants also valued review of specific language they could use with families, especially for “approaching patients and family who were more difficult to convince of the utility of going through a particular therapeutic plan”.

Another strength of this intervention is that no association was found between SP scores and the year of the workshop, suggesting consistency of our methodology and results over time. An interesting finding was that participants’ prior communication skills workshop experiences and exposure to patients with the presenting symptom of syncope were not associated with improvement in performance. Trainees at all levels can benefit from this intervention and residency programs may benefit from implementing similar curricula to improve residents learning surrounding subspecialty referrals, a skill endorsed by the ABP.

These results build on our prior work with pediatric trainees in improving skills of subspecialty referral, particularly in providing advanced communication tools to counsel a concerned or anxious parent [9]. The improvement in performance in our intervention parallels the improvement reported by Guse et al., in which diagnostic work-up decreased in patients with low-risk pediatric syncope in the pediatric emergency department after a quality improvement intervention introduced a new syncope guideline to providers [3].

The next steps for the project include refining the curriculum based on resident feedback and adding a component to assess for long-term gains by the residents or communities in which they practice, rather than focusing on the short-term achievements. To decrease resource utilization and address resident feedback about the intervention seeming “repetitive” by the end of the day, the curriculum could be changed to eliminate the pre/post OSCE portions. The residents will still be required to do the online portion of the project a day earlier and emerge into the interventional workshop on the day of the curriculum. To evaluate residents long-term, we propose assessing residents performance with real patients in their continuity clinics, perhaps through a quality improvement project similar to prior work showing a decrease in low-probability chest pain referrals to cardiologists, which would assess residents at Kirkpatrick level 4 (results) [1, 18]. Lastly, one could assess change in residents performance by querying their clinic preceptors or subspecialists receiving their referrals for appropriateness of referrals, also evaluating Kirkpatrick level 4 [18].

Residency programs have found innovative ways to improve skills of subspecialty referral; for example, the University of Chicago Internal Medicine Department created a dedicated training program for interns to improve their comfort and skills in requesting subspecialty referrals within the hospital [19]. Additionally, after a self-assessment by residents indicating that they would value further training in subspecialty referral, a team at Children’s National and George Washington University created tools to do so in the primary care and subspecialty arenas, respectively: a referral feedback form and a subspecialty learning prompt [20]. These tools led to resident reflection, during which 57% of residents reported they would change their future practice with the understanding of “new medical knowledge including red flags for referral”, including 17% of residents specifically sharing that they would pursue further management in primary care before referral [20].

SPs have been successfully utilized in education of trainees in various contexts, including assessing Accreditation Council for Graduate Medical Education (ACGME) milestones of residents and medical students. Many studies have also shown short-term improvement in the ability of trainees to share challenging news after working with SPs, as evidenced by a meta-analysis of 17 studies on this topic [21]. Additionally, trainee confidence improves after communication skills interventions with SPs [9, 22,23,24].

Limitations we identified include: 1) Ten residents did not complete the demographic questionnaire; 2) Study constraints limited our participant numbers; 3) This was a single institution study that may not be generalizable to other training programs; 4) The authors did not study long-term and patient-centered outcome assessments, which logically would follow this report as part of a multi-institutional study, and 5) The potential of smaller residency programs emulating this study might be limited based on resources available. Additionally, the one group pretest–posttest design, while making the study more feasible from a resource standpoint, has disadvantages; to reduce challenges of this design, the study conditions were controlled, participants remained on site for the 4-h time window, and there was a limited time interval between the pretest and posttest [25]. Another limitation includes that the same SPs were utilized in pre- and post-test sessions, playing different patients/families. To counter this potential conflict, the majority of questions SPs were given to evaluate residents were yes/no and all questions were made to be as objective as possible with adaptations of the SEGUE framework and intensive SP training to reduce bias in assessment and optimize inter-rater reliability.


By participating in this blended, multimodal learning curriculum on the referral and consultation process, residents’ short-term communication skills notably improved. This simulation-based learning is a valuable supplement to resident training to improve specific skills involved in subspecialty referral by enhancing the communication needed for a successful referral despite challenging family circumstances. Further research is needed to assess if this intervention improves the patient experience and patient care by providing residents with enduring skills to judiciously manage the referral process.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.


  1. Harahsheh AS, Hamburger EK, Saleh L, Crawford LM, Sepe E, Dubelman A, et al. Promoting Judicious Primary Care Referral of Patients with Chest Pain to Cardiology: A Quality Improvement Initiative. Med Decis Making. 2021;41(5):559–72.

  2. Ray KN, Ashcraft LE, Kahn JM, Mehrotra A, Miller E. Family perspectives on high-quality pediatric subspecialty referrals. Acad Pediatr. 2016;16(6):594–600.

    Article  Google Scholar 

  3. Guse SE, Neuman MI, O’Brien M, Alexander ME, Berry M, Monuteaux MC, Fine AM. Implementing a guideline to improve management of syncope in the emergency department. Pediatrics. 2014;134(5):e1413–21.

    Article  Google Scholar 

  4. Massin MM, Bourguignont A, Coremans C, et al. Syncope in pediatric patients presenting to an emergency department. J Pediatr. 2004;145:223–8.

    Article  Google Scholar 

  5. Tretter JT, Kavey RE. Distinguishing cardiac syncope from vasovagal syncope in a referral population. J Pediatr. 2013;163(6):1618-1623.e1. 2013 Aug 27 PMID: 23992679.

    Article  Google Scholar 

  6. Forrest CB, Glade GB, Baker AE, Bocian AB, Kang M, Starfield B. The pediatric primary-specialty care interface: how pediatricians refer children and adolescents to specialty care. Arch Pediatr Adolesc Med. 1999;153(7):705–14. 10401803.

    Article  Google Scholar 

  7. Harahsheh AS, O’Byrne ML, Pastor B, Graham DA, Fulton DR. Pediatric chest pain-low-probability referral: a multi-institutional analysis from Standardized Clinical Assessment and Management Plans (SCAMPs®), the pediatric health information systems database, and the national ambulatory medical care survey. Clin Pediatr (Phila). 2017;56(13):1201–8. 2017 Jan 12. PMID: 28081617; PMCID: PMC6388765.

    Article  Google Scholar 

  8. Hamburger EK, Lane JL, Agrawal D, Boogaard C, Hanson JL, Weisz J, Ottolini M. The referral and consultation entrustable professional activity: defining the components in order to develop a curriculum for pediatric residents. Acad Pediatr. 2015;15(1):5–8.

    Article  Google Scholar 

  9. Harahsheh AS, Ottolini M, Lewis K, Blatt B, Mitchell S, Greenberg L. An innovative pilot curriculum training pediatric residents in referral and communication skills on a cardiology rotation. Acad Pediatr. 2016;16(7):700–2. 2016 Jun 7. PMID: 27287272.

    Article  Google Scholar 

  10. Armstrong, P. (2010). Bloom’s Taxonomy. Vanderbilt University Center for Teaching. Retrieved [2/15/22] from

  11. Ramani S, Leinster S. AMEE Guide no. 34: teaching in the clinical environment. Med Teach. 2008;30(4):347–64.

    Article  Google Scholar 

  12. May W, Park JH, Lee JP. A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996–2005. Med Teach. 2009;31(6):487–92.

    Article  Google Scholar 

  13. Lewis KL, Bohnert CA, Gammon WL, Holzer H, Lyman L, Smith C, Thompson TM, Wallace A, Gliva-McConvey G. The Association of Standardized Patient Educators (ASPE) Standards of Best Practice (SOBP). Adv Simul. 2017;2(10).

  14. Lewis DA, Dhala A, Syncope in the pediatric patient. The cardiologist’s perspective. Pediatr Clin North Am. 1999;46(2):205–19. 10218070.

    Article  Google Scholar 

  15. Benditt DG, Nguyen JT. Syncope: therapeutic approaches. J Am Coll Cardiol. 2009;53:1741–51.

    Article  Google Scholar 

  16. Makoul G. The SEGUE Framework for teaching and assessing communication skills. Patient Educ Couns. 2001;45(1):23–34.

    Article  Google Scholar 

  17. Branch WT. Use of critical incident reports in medical education. J Gen Intern Med. 2005;20(11):1063–7.

    Article  Google Scholar 

  18. Kurt, S. "Kirkpatrick Model: Four Levels of Learning Evaluation," in Educational Technology, October 24, 2016. Retrieved from

  19. Martin SK, Carter K, Hellermann N, Glick LR, Ngooi S, Kachman M, Farnan JM, Arora VM. The consultation observed simulated clinical experience: training, assessment, and feedback for incoming interns on requesting consultations. Acad Med. 2018;93(12):1814–20.

    Article  Google Scholar 

  20. Muradian S, Widge A, Hanson JL, Lane JL, Boogaard C, Agrawal D, Ottolini M, Hamburger EK. Tools for learning about the referral and consultation process for pediatric residents. Acad Pediatr. 2018;18(3):357–9.

    Article  Google Scholar 

  21. Johnson J, Panagioti M. Interventions to improve the breaking of bad or difficult news by physicians, medical students, and interns/residents: a systematic review and meta-analysis. Acad Med. 2018;93(9):1400–12.

    Article  Google Scholar 

  22. Koller SE, Moore RF, Goldberg MB, Zhang J, Yu D, Conklin CB, Milner RE, Goldberg AJ. An informed consent program enhances surgery resident education. J Surg Educ. 2017;74(5):906–13.

    Article  Google Scholar 

  23. Greenberg LW, Ochsenschlager D, O’Donnell R, Mastruserio J, Cohen GJ. Communicating bad news: a pediatric department’s evaluation of a simulated intervention. Pediatrics. 1999;103(6):1210–7.

    Article  Google Scholar 

  24. Tender JA, Cuzzi S, Kind T, Simmens SJ, Blatt B, Greenberg L. Educating pediatric residents about breastfeeding: Evaluation of 3 time-efficient teaching strategies. J Hum Lact. 2014;30(4):458–65.

    Article  Google Scholar 

  25. Campbell DT, Stanley JC. Experimental and quasi-experimental designs for research on teaching. In: Gage NL, editor. Handbook of research on teaching. Chicago, IL: Rand McNally; 1963. p. 171–246.

    Google Scholar 

Download references


The authors thank the 2015-2019 trainees at Children’s National Hospital, Lindsay Attaway, Jeff Sestokas, the Office of Medical Education, Children’s National Board of Visitors, the staff of the Children’s National Simulation Learning Center, and the Research, Education, Advocacy, and Child Health Care (REACH) program within the Children’s National Hospital Pediatric Residency Program.


The project was funded in part by the Board of Visitors Simulation Learning Center grant at Children’s National Hospital. Dr. Harahsheh is supported by a Sub-agreement from the Johns Hopkins University with funds provided by Grant No. R61HD105591 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development and the Office of the Director, National Institute of Health (OD). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Eunice Kennedy Shriver National Institute of Child Health & Human Development, the Office of the Director, National Institute of Health (OD), or the Johns Hopkins University. The funding sources had no involvement in study design, the collection, data analysis and interpretation, report writing or the decision to submit the article for publication.

Author information

Authors and Affiliations



E.A.S. was involved in data analysis and drafting the manuscript. L.G. was involved in study design, workshop and tools development, data analysis, and provided critical review of the manuscript. E.H. was involved in study design, workshop, and tools development and provided critical review of the manuscript. M.O. was involved in study design, workshop, and tools development and provided critical review of the manuscript. D.A. was involved in study design, workshop, tools development, data analysis and provided critical review of the manuscript. K.L. was involved in tools development, training the standardized parents and revising the manuscript. J.R.B. was involved in data analysis and revising the manuscript. J.E.B. was involved in data analysis and revising the manuscript. A.S.H. was involved in study design, workshop and tools development, conducting the workshop, data gathering, analysis of data and providing critical review of the manuscript. This report was submitted with the full knowledge and approval of all listed coauthors. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Ashraf S. Harahsheh.

Ethics declarations

Ethics approval and consent to participate

Research was performed in accordance with the Declaration of Helsinki and was approved by the Children’s National Hospital Institutional Review Board, with informed consent obtained in writing from all participants (ID Pro00004226).

Consent for publication


Competing interests

Non-financial interest: Dr. Harahsheh serves as a scientific advisory board member of OP2 DRUGS (“OP2”). This advisory position has no relevant disclosures for this publication.

There are no other conflicts of interest for the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Dr. Ottolini was the Vice Chair of Medical Education at Children’s National Hospital at the time of conducting the study.

From the Department of Pediatrics, Children’s National Hospital (Drs. Agrawal, Bost, Greenberg, Hamburger, Harahsheh, and Stave), Division of Cardiology (Dr. Harahsheh), Medical Education (Dr. Agrawal), Biostatistics (John Barber, Dr. Bost), George Washington University School of Medicine and Health Sciences Clinical Learning and Simulation Skills Center (Dr. Lewis), George Washington University School of Medicine and Health Sciences (Drs. Agrawal, Bost, Greenberg, Hamburger and Harahsheh), and Maine Medical Center (Dr. Ottolini).

Supplementary Information

Additional file 1.

All names listed in supplementary material were assigned to Standardized Parents and no names listed represent the names of actors or participants in the study.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stave, E.A., Greenberg, L., Hamburger, E. et al. An educational intervention to facilitate appropriate subspecialty referrals: a study assessing resident communication skills. BMC Med Educ 22, 533 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: