- Research article
- Open Access
An outpatient telehealth elective for displaced clinical learners during the COVID-19 pandemic
BMC Medical Education volume 21, Article number: 174 (2021)
In response to the COVID-19 pandemic, medical schools suspended clinical rotations. This displacement of medical students from wards has limited experiential learning. Concurrently, outpatient practices are experiencing reduced volumes of in-person visits and are shifting towards virtual healthcare, a transition that comes with its own logistical challenges. This article describes a workflow that enabled medical students to engage in meaningful clinical education while helping an institution’s outpatient practices implement remote telemedicine visits.
A 4-week virtual elective was designed to allow clinical learners to participate in virtual telemedicine patient encounters. Students were prepared with EMR training and introduced to a novel workflow that supported healthcare providers in the outpatient setting. Patients were consented to telehealth services before encounters with medical students. All collected clinical information was documented in the EMR, after which students transitioned patients to a virtual Doxy.me video appointment. Surveys were used to evaluate clinical and educational outcomes of students’ participation. Elective evaluations and student reflections were also collected.
Survey results showed students felt well-prepared to initiate patient encounters. They expressed comfort while engaging with patients virtually during telemedicine appointments. Students identified clinical educational value, citing opportunities to develop patient management plans consistent with in-person experiences. A significant healthcare burden was also alleviated by student involvement. Over 1000 total scheduled appointments were serviced by students who transitioned more than 80 % of patients into virtual attending provider waiting rooms.
After piloting this elective with fourth-year students, pre-clerkship students were also recruited to act in a role normally associated with clinical learners (e.g., elicit patient histories, conduct a review of systems, etc.). Furthermore, additional telemedicine electives are being designed so medical students can contribute to patient care without risk of exposure to COVID-19. These efforts will allow students to continue with their clinical education during the pandemic. Medical educators can adopt a similar workflow to suit evolving remote learning needs.
On March 17th, 2020, the Association of American Medical Colleges (AAMC) issued strong guidance that medical schools suspend clinical rotations during the coronavirus pandemic, effective immediately . A temporary and unified suspension of clinical teaching by all institutions would ensure medical student safety as new information about COVID-19 continued to emerge. Medical schools following this guidance developed innovative curricular modifications to deliver uncompromised education during the ongoing pandemic. This article reports the successful implementation of a new telehealth elective course designed to introduce fourth-year medical students to the use of technologies for telehealth at the Rutgers Robert Wood Johnson Medical School (RWJMS) in response to the crisis.
The elective course was designed not only to provide students with a unique educational opportunity, but also to help patients of the Robert Wood Johnson Medical Group (RWJMG) transition rapidly from in-person to remote consultations. The Commonwealth Fund estimated outpatient practices scheduled less than half their regular volume of in-person visits throughout March to mitigate virus transmission . For the numerous practices that struggle to care for their patient population, the ability to extend patient-provider relationships beyond traditional in-person visits is invaluable. Under normal circumstances, the RWJMG outpatient practices care for approximately 15,000 patients each week. Total patient volume decreased nearly 33 % in the last two weeks of March 2020 before remote visits were possible. By the end of April, the outpatient practices scheduled 6800 telemedicine visits per week. This transition was not without challenges, as provider illness and post-exposure quarantine requirements exacerbated an already stressed system during the pandemic’s initial wave.
The 2015–2016 Liaison Committee on Medical Education (LCME) Annual Medical School Questionnaire reported heterogenous use of telemedicine in undergraduate medical education by one-half of medical schools surveyed . A 2020 review of telemedicine in undergraduate medical education revealed varied integration of telemedicine into medical school curricula. 71 % of respondents discussed the basics of telemedicine during didactics. Telemedicine featured more practically during standardized patient encounters (59 %), patient encounters (53 %), interprofessional training (40 %), and scholarly projects (29 %) . The pandemic has stimulated a surge in telehealth integration for undergraduate medical education, providing quality education while maintaining social distancing policies . A telehealth workflow was developed in coordination with RWJMG practice managers to facilitate efficient diagnosis, treatment, management, and education virtually. Students participating in an elective course were instrumental during this transition, reducing the workforce burden while gaining valuable exposure to a modern healthcare intervention.
The aim of this elective was to provide students with opportunities to utilize telemedicine technologies for authentic patient encounters and enable them to understand telehealth applications and factors affecting the ability of patients to connect remotely.
Setting and objectives
Fourth-year medical students were recruited throughout March and April 2020 to pilot the four-week service elective. To ensure satisfaction of curricular competencies, several educational objectives were defined for students to: (1) understand the broad applications of telehealth interventions in the outpatient setting, (2) develop fluency with telehealth technologies to support virtual appointments, (3) learn from clinicians and implement techniques to interact with patients during virtual appointments, (4) use telehealth modalities for assessment of clinical status and risk of decompensation, (5) develop a plan of care and counsel patients throughout virtual appointments, and (6) communicate with patients about health literacy and other social determinants of health affecting virtual appointment success. These educational objectives aligned with competencies subsequently published by the AAMC in their Competencies Across the Learning Continuum Series that addresses telehealth in September 2020 .
Students attended one mandatory training session – conducted via Zoom video conference – where an overview of the teleconsultation workflow (Fig. 1), the electronic medical record (EMR) was demonstrated, and the Doximity and Doxy.me applications were introduced. Shift schedule logistics, use of interpretation services for non-English speaking patients, and common technology and connectivity issues were also discussed. All training sessions were recorded for students to review later.
For attending physicians participating in the elective, a Microsoft Excel workbook was populated with their contact information and deidentified appointment times in half-day increments each time they requested student support. Automated emails were generated throughout each week to alert students as the Excel workbook populates with new attending physicians and their respective appointment times. Students could indicate availability to support attending physicians via a hyperlink provided in these emails. Students were required to contact each respective attending physician and confirm their desire for student support before servicing any patients. Attending physicians indicated whether students could observe or participate in patient encounters – as permitted by time and technology constraints – through subsequent correspondences.
Patients scheduling a telemedicine appointment were told to expect a phone call from the office in lieu of standard check-in procedures. It was made clear by office staff that a smartphone or computer was required to complete video visits. On the day of patients’ scheduled appointments, students initiated contact by phone call approximately 30 min before their scheduled appointment time. The dialer feature of the Doximity application was used to protect personal phone numbers, and the subsequent patient encounter was documented in the EMR via remote access . A new document type was created in the EMR for telehealth patient encounters to satisfy documentation requirements. This telehealth note requires information about the visit type (phone call or video call), the method used to connect the patient (phone, Doxy.me, or other), the patient’s location (state), and the attending physician’s location (clinical office, location other than clinical office, home or other non-affiliated location). Before collecting any clinical information, the patient was provided with information about the risks and benefits of telehealth services. Their verbal consent to participate in telehealth services was obtained and students proceeded with the patient encounter thereafter. Vital signs were then obtained from personal home health monitoring equipment if available (e.g., scales, thermometers, blood pressure monitors, etc.). All clinical information including history of current illness, review of systems, medication reconciliation, allergies, and any interval surgical, medical, or social history was entered into the telemedicine note where applicable.
Next, students transitioned the patient from the initial phone call to a Doxy.me video call. This HIPAA-compliant telehealth system provides a cloud-based video conferencing platform with a virtual waiting room feature for providers to triage and connect with patients at their appointment time . Students used skills attained from the training sessions to troubleshoot technological difficulties and other issues that may arise during the transition. Students held their preliminary note in the EMR for use by the attending physician once the transition to Doxy.me was successful. Attending physician schedules influenced whether a student joined the subsequent patient-attending consultation or instead exited the patient encounter. Students could also participate via telephone to preserve bandwidth needed for a three-person Doxy.me video conference. In rare cases, the patient-attending consultation was conducted through an alternate HIPAA-compliant video conference platform that also allowed student participation. Debrief sessions were held by attending physicians either after certain patient encounters or after a completed shift to provide feedback and education.
To be eligible for course credit, students had to complete a survey each time they logged contributed hours. The survey assessed barriers to successful telehealth utilization. Various appointment and workflow data were collected to enable retrospective analysis and real-time process improvement. One week of elective course credit was granted per 20 h of support contributed. Students wrote a one-page reflection on their experience with telehealth and virtual patient care and completed an elective evaluation form.
Over a four-week period (April 13 to May 8), 64 fourth-year medical students (38.1 % of the graduating class) completed the mandatory training session and logged contributed hours within the program. 29 (45.3 %) students contributed enough hours to qualify for elective credit, with an average of 35.4 h logged (median of 26.8 h). Three students logged over 80 h.
Surveys identified many barriers to patient encounter initiation that were used by practice managers for process improvement. Students learned to troubleshoot with patients when appropriate and provided detailed solutions thereafter to ensure future telehealth appointments were successful. Social determinants of health were frequently recognized as barriers to successful telehealth delivery for underserved patients. Technological illiteracy was often cited in association with elderly patients. Despite these challenges, collected evaluation forms revealed students gained substantial experience with telehealth technologies. Many students received practical instruction about various aspects of telehealth and felt they were able to explore virtual patient encounter capabilities simultaneously. Students also felt they could connect with patients, understand patient perspectives, and determine patients’ understanding of their conditions (Fig. 2).
Students’ reflections provided narrative affirmation of objective data collected on the evaluation forms. Confidence was a recurrent motif, as students honed their abilities to elicit a history and counsel patients via video conference or telephone. Other topics discussed in students’ reflections include navigating technology, heightened awareness of barriers to telehealth use, and telehealth in medical education. Excerpts from students’ reflections are provided in Table 1 to highlight some of their experiences with telehealth. These excerpts are representative of the overwhelmingly positive response to the elective. For balance, some negative remarks were extracted from what otherwise were positive reflections.
Reflections also lent insight to the educational values of the elective. Interviewing patients, collecting the history, and documenting the encounter all provided significant educational value for students. Students who supported attending physicians with fewer scheduled patients were able to develop patient management plans and participate extensively during the Doxy.me patient encounter. Of the 34 students for whom reflections were received, 11 (32.4 %) mention debrief sessions with an attending physician after observation of and/or participation in patient-attending encounters. Unfortunately, this statistic does not capture the true frequency of student observation and participation – nor debrief sessions – as the majority of students chose to reflect upon other aspects of their experience.
Given the large selection of participating specialty and subspecialty attending physicians at RWJMG, students were able to serve a broad spectrum of patients. During the elective, 58 attending physicians from 3 specialties (internal medicine, pediatrics, neurology; inclusive of 17 total subspecialties) participated. The 64 fourth-year medical students serviced 1411 total scheduled appointments. The number of student-initiated patient encounters was high from the outset and increased significantly in subsequent weeks, with the percentage of patients successfully transitioned to a Doxy.me virtual encounter trending higher over time (Fig. 3).
Several barriers (Fig. 4a, b) to successful telehealth utilization were encountered during the pilot period. Together, 197 (13.9 %) of 1411 total scheduled appointments were unable to be initiated for two primary reasons: patients either asked to reschedule (80, 41.5 %) or simply did not answer the phone (79, 40.9 %). Of the 1214 patient encounters that students were able to initiate, 992 (81.7 %) were successfully transitioned to a Doxy.me video call. Most patients from the remaining 222 encounters specifically requested transition to a traditional phone call (101, 46.5 %) or reported inability to access a compatible smartphone or computer at the time (71, 32.7 %). Despite assistance from students, 40 (18.4 %) patients were not adept enough with technology to enter the Doxy.me virtual waiting room. Student surveys contained valuable workflow feedback, which allowed clinic teams to modify the workflow where necessary to better meet patient needs.
The pilot employed a workflow enabling students to meaningfully participate in patient care within the telehealth setting. Students expressed increased confidence and comfort with telemedicine technologies and patient interviewing. Many reflections identified barriers to telehealth implementation and access (Table 1).
There were variations in encounter quality and teaching opportunities. Attending physicians were able to personalize their requests for student documentation and participation, with some encouraging student participation during the actual patient encounters or including subspecialty-specific review of systems. Although all attending physicians were asked to conduct debrief sessions when possible, time constraints and schedule conflicts reduced the number of attending physician-conducted educational moments. Most debrief sessions focused on the medical knowledge displayed through each student’s documentation.
Since the pilot, efforts expanded to include 100 second-year medical students. With these additional pre-clinical participants, the telemedicine elective has serviced thousands of patients from the RWJMG outpatient practices. This expansion demonstrates how the workflow can be adapted to accommodate large volumes of students. Another project has adapted the workflow to identify social determinants that prevent the successful adoption of telehealth. Student volunteers perform similar calls with a greater focus on addressing patients’ needs across multiple clinic sites.
As patient volumes stabilize in the outpatient clinics, the elective will increasingly focus on education. As in-person appointments become possible with social distancing protocols, fewer telemedicine visits per attending physician will promote uniformity among students’ experiences and increase educational moments. To ensure student exposure to various patient populations and pathophysiology, several specialty clinics adopted the presented workflow to create specialty-specific telehealth elective opportunities. The Pediatrics Clerkship has also integrated a modified outpatient to supplement clinical experiences.
After a successful four-week outpatient pilot, an inpatient telemedicine initiative was developed based on the Yale iCollaborative resources . This initiative will allow medical students to contribute to inpatient care for patients under isolation. The primary objective of this initiative is to introduce students to telemedicine encounters with patients where appropriate without risk of exposure to COVID-19, while additional objectives require students to participate in rounds and practice case presentations, contribute to patient documentation, and act as patient liaisons after discharge to ensure patients successfully connect with the next phases of their care. Successful integration of this inpatient clinical curriculum will improve the capacity necessary for displaced students to engage with inpatients, albeit through a different medium.
The American Medical Association (AMA) recently published a playbook of telehealth workflows in support of recent national policy changes to reduce in-person visits and concomitant exposure risks to the coronavirus . The AAMC further recognizes the ability of telehealth to support longitudinal patient care models across the continuum of practice . Together, these AMA and AAMC recommendations establish a framework from which medical schools can build educational telehealth opportunities into their curricula. These objectives ultimately reflect the evolving demand for non-traditional digital healthcare services in periods of both crises and normalcy. Telemedicine workflows like the pilot’s can increase flexibility for clinical educators within this new framework.
With the development and implementation of these new telehealth initiatives, clinical learning experiences can be expanded in virtual settings. These novel educational experiences have the potential to make the clinical curriculum more resilient during this global pandemic and beyond.
Elective evaluations were only completed by those students who requested elective credit. Of the 64 fourth-year medical students, only 29 completed enough sessions to receive credit. Many of the other students who participated significantly did not require credit and did not submit an evaluation.
Among the measured clinical outcomes, data collected about barriers to telehealth implementation are likely skewed. The pilot was conducted while the RWJMG outpatient practices existed in an unprecedented state of flux without their usual automated scheduling systems in place. Instances where students were unable to initiate the patient encounter via phone call were increasingly corrected towards the end of the pilot as these RWJMG systems were reimplemented. Unfortunately, it was not possible to follow up with these patients who were rescheduled to ensure they eventually received care.
During these unprecedented times, medical schools are developing innovative methods of remote educational instruction. The pilot study provides a workflow for displaced clinical learners to engage with patients via telehealth technologies, allowing them to remain clinically active through authentic patient encounters. The workflow was well-received by students and allowed a large number of students to support the outpatient practices of the medical school. With support from governing bodies, telehealth endeavors can remain a permanent feature in medical school curricula.
Availability of data and materials
The datasets used and/or analyzed in this study are available from the corresponding author on reasonable request.
Association of American Medical Colleges
Rutgers Robert Wood Johnson Medical School
Robert Wood Johnson Medical Group
Liaison Committee on Medical Education
Electronic medical record
American Medical Association
Association of American Medical Colleges. Guidance on Medical Students’ Participation in Direct Patient Contact Activities. https://www.aamc.org/system/files/2020-04/meded-April-14-Guidance-on-Medical-Students-Participation-in-Direct-Patient-Contact-Activities.pdf. Revised April 14, 2020. Accessed 19 July 2020
Ateev, Mehrotra, et al. The Impact of the COVID-19 Pandemic on Outpatient Visits: A Rebound Emerges. To the Point, Commonwealth Fund (blog). May 19, 2020. https://doi.org/10.26099/ds9e-jm36. Accessed 19 July 2020.
Waseh S, Dicker AP. Telemedicine Training in Undergraduate Medical Education: Mixed-Methods Review. JMIR Med Educ. 2019;5(1):e12515. doi:https://doi.org/10.2196/12515. Published 2019 Apr 8.
Iancu AM, Kemp MT, Alam HB. Unmuting Medical Students’ Education: Utilizing Telemedicine During the COVID-19 Pandemic and Beyond. J Med Internet Res. 2020;22(7):e19667. doi:https://doi.org/10.2196/19667. Published 2020 Jul 20.
New and Emerging Areas in Medicine Series: Telehealth Competencies. Association of American Medical Colleges website.https://www.aamc.org/system/files/2020-09/hca-telehealthcollection-telehealth-competencies.pdf. Accessed 15 Sept 2020.
Doximity, Inc. Doximity Mobile for iPhone and Android. http://www.doximity.com/app. Accessed 19 July 2020.
Doxy.me, LLC. The Simple, Free, and Secure Telehealth Solution. https://doxy.me/en. Accessed 19 July 2020.
Amanda Zhou et al. Internal Medicine Inpatient Telehealth Elective Design.https://icollaborative.aamc.org/resource/5098/. iCollaborative. April 22, 2020. Accessed 19 July 2020.
American Medical Association. Digital Health Implementation Playbook.https://www.ama-assn.org/system/files/2018-12/digital-health-implementation-playbook.pdf. Published 2018. Accessed 19 July 2020.
Association of American Medical Colleges. Core Entrustable Professional Activities for Entering Residency: Curriculum Developer’s Guide.https://store.aamc.org/downloadable/download/sample/sample_id/63. Published 2014. Accessed 19 July 2019.
We thank Daniel Levin and Valeriya Gershteyn for facilitating remote EMR access and workflow optimization, without whom such quick adoption would not have been possible. Thanks also to Drs. Amy Tyberg and Frank Sonnenberg for their support, and to Julie Lawson for her flexibility in helping integrate the workflow into our many practices.
The authors did not receive funding for this study.
Ethical approval and consent to participate
This study was approved by the Rutgers Institutional Review Board (committee reference number PRO2020001884).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Weber, A.M., Dua, A., Chang, K. et al. An outpatient telehealth elective for displaced clinical learners during the COVID-19 pandemic. BMC Med Educ 21, 174 (2021). https://doi.org/10.1186/s12909-021-02604-z
- Telemedicine elective
- Remote learning
- Outpatient clinic
- Coronavirus pandemic