Using electronic patient records: defining learning outcomes for undergraduate education
BMC Medical Education volume 19, Article number: 30 (2019)
Healthcare professionals are required to access, interpret and generate patient data in the digital environment, and use this information to deliver and optimise patient care. Healthcare students are rarely exposed to the technology, or given the opportunity to use this during their training, which can impact on the digital competence of the graduating workforce. In this study we set out to develop and define domains of competence and associated learning outcomes needed by healthcare graduates to commence working in a digital healthcare environment.
A National Working Group was established in the UK to integrate Electronic Patient Records (EPRs) into undergraduate education for healthcare students studying medicine, pharmacy, nursing and midwifery. The working group, comprising 12 academic institutions and representatives from NHS England, NHS Digital and EPR system providers, met to discuss and document key learning outcomes required for using EPRs in the healthcare environment. Outcomes were grouped into six key domains and refined by the group prior to external review by experts working in medical education or with EPRs.
Six key domains of competence and associated learning outcomes were identified and defined. External expert review provided iterative refinement and amendment. The agreed domains were: 1) Digital Health: work as a practitioner in the digital healthcare environment; 2) Accessing Data: access and interpret patient data to inform clinical decision-making; 3) Communication: communicate effectively with healthcare professionals and patients in the digital environment; 4) Generating data: generate data for and about patients within the EPR; 5) Multidisciplinary working: work with healthcare professionals with and alongside EPRs; and 6) Monitoring and audit: monitor and improve the quality and safety of healthcare.
The six domains of competence and associated learning outcomes can be used by academics to guide the integration of EPRs into undergraduate healthcare programmes. This is key to ensuring that the future healthcare workforce can work with and alongside EPRs.
The patient record is fast becoming digitised. Electronic Patient Records (EPRs) enable real-time sharing of information within and across the interface of care. Digital interventions within the EPR such as electronic prescribing (ePrescribing), or Computerised Physician Order Entry (CPOE), can reduce the risk of error [1,2,3,4]. A reduction in risk translates to improved patient safety and potential cost savings for the National Health Service (NHS), with interoperability of systems offering further reductions in expenditure . Owing to the proven benefits of the technology, NHS England (NHSE) has committed to making all patient care records digital, real-time and interoperable by 2023 through the creation and integration of EPRs .
The effective utilisation of EPRs relies on information technology (IT) skills, user familiarity, competence, and a knowledge of data within systems to effectively inform clinical decision-making. Training is essential for the successful implementation and on-going use of the technology [7, 8]; sub-optimal use can increase the risk of clinical and procedural errors [9,10,11]. Lack of training, education and staff development in this area have been identified as major barriers to innovation . It could be argued that such training of healthcare professionals has not kept pace with digital challenges to date . NHS England is working to improve the digital competence of the workforce through the introduction of digital academies and clinical information officers . This will transform care by encouraging staff to embrace new technology, however the training does not include the future workforce—healthcare students. Students are increasingly exposed to EPR systems, and so need to be given the opportunity to develop the competencies to “access, discriminate, analyse, apply knowledge and master large flows” of information from these . Importantly, they require robust training within an environment made safe for learning. This research aimed to develop and define competencies needed by healthcare graduates to commence working in a digital healthcare environment. Learning outcomes were selected to be identified since these are used throughout undergraduate and postgraduate healthcare training in the United Kingdom (UK) [15, 16]. As such, medical educators are familiar with how the roles defined within a competency can be translated into outcomes as knowledge, attitudes and skills and used to monitor the progress of students and trainees [17, 18].
A National Working Group of academics in the UK was established to integrate EPRs into the undergraduate healthcare education they oversee. The group comprised academics working in medicine, pharmacy, nursing, midwifery and health informatics programmes across 12 different institutions along with a medical and pharmacy student. The academics were joined by representatives from NHS England, Health Education England and EPR system providers (Table 1).
The group were asked to discuss and document competencies and associated learning outcomes that they considered to be needed by their healthcare students, as they learned with and alongside EPRs in modern healthcare settings. A competency was defined to the group as “an observable behaviour in the context of the role of the healthcare professional”. The group were asked to consider the learning outcomes needed by graduates to meet the needs of patients and other healthcare professionals in the safe and effective delivery of care . The group were aware that no such learning outcomes currently existed in undergraduate healthcare programmes, although many professional bodies were beginning to include their use in professional standards (e.g. Royal Pharmaceutical Society professional standards for hospital pharmacy services include a section on ‘Digital technology and informatics to support medicines use’) . Upon completion of a first draft, the learning outcomes were grouped into six overarching domains of competence by two academics (SP, KW) and presented back to the group for review and refinement. The group agreed upon the wording for each domain of competence and allocation and wording of learning outcome to create a final draft (Additional file 1). This was written up and disseminated via email to the group for any final comments for refinement prior to external review.
Experts working in medical education or as healthcare professionals working with, or researching EPRs, were invited to participate in a two-round eDelphi to independently review the domains of competence and associated learning outcomes to gain consensus. Ten experts were identified through recommendations from members of the working group. The experts were asked to review the domains and outcomes developed by the working group and to make suggestions for refinement, addition, amendment or removal through electronic return of a standardised pro-forma. Following completion of the first round, competencies and learning outcomes were amended in line with feedback and sent back to participants for further review, with any other comments and suggestions included in the final document for publication (Additional file 2).
The working group agreed upon six domains of competence and 29 learning outcomes related to the training of undergraduate healthcare students in the context of EPRs (Additional file 1). The final list was emailed to eight experts who agreed to participate in the eDelphi process (Table 2).
In Round 1 the eDelphi process, participants made recommendations to remove two learning outcomes, add four, amend 19 and move five (See Fig. 1). Suggestions were made to remove learning outcomes that were not considered necessary, were unclear or were covered elsewhere. For example ‘Describe the digitisation in the NHS’ was suggested to be removed as this was not considered important for the domain of competence relating to the effective use of EPRs. Amendments were suggested to make the description of competencies and learning outcomes clearer. For example, Domain 1 ‘Digital Literacy: proficiency in the use of EPRs and adaptive to changes in this technology’, was suggested to be changed to ‘Digital Health: working as a practitioner in the digital NHS’. Digital literacy was considered to refer more to the terminology used in the context of EPRs, rather than the use of the technology. The responses from Round 1 were discussed by two academics (SP, KW), who worked to resolve any conflicting opinions and amended the domains of competence and associated learning outcomes accordingly for disseminating in Round 2.
All eight participants completed Round 2 of the process. In this round, suggestions were made to add two learning outcomes, remove one and amend 8 eight (see Fig. 1 and Additional file 3). For example, in this round one participant suggested that the reporting of adverse events should be included in ‘Monitoring and audit’ (Domain 6). Again, responses from Round 2 were discussed by two academics (SP, KW). Following two rounds of review (Fig. 1), the six domains of competence and 32 learning outcomes were agreed (see Table 3). The domains agreed were: 1) Digital health; 2) Accessing data; 3) Communication; 4) Generating data; 5) Multidisciplinary working; and 6) Monitoring and audit.
Training is a key for the successful implementation of technology in the healthcare environment , and insufficient training can lead to sub-optimal use. In this study we set out to develop and define competencies and associated learning outcomes needed by healthcare graduates to commence working in a digital healthcare environment. A number of professionals were involved in the iterative development of six domains of competence and 32 learning outcomes, which have been identified as integral to the training of undergraduate healthcare students to work with and alongside EPRs. These provide the baseline knowledge for the use of EPR technology in healthcare, as well as the essential skills and professional attitude to work alongside technology to provide patient care.
The first domain of competence, ‘Digital health’, ensures that students have an understanding of the technologies available, the impact they can have on clinical care, and how to work safely with and alongside the systems. Healthcare students may be aware, for example, that EPRs such as CPOE can reduce the risk of medication errors , but may be less cognisant of how to respond to clinical decision support or the complex functionality of systems to optimise workflow and coordinate care. Learning outcomes relating to ‘Accessing data’ ensure the healthcare student can demonstrate effective access and interpretation of patient data to inform clinical decision-making. Information held within EPRs can be fragmented when compared to paper-based records; this fragmentation may have an impact on clinical reasoning . It is important that healthcare staff can access relevant data in order to build a patient story and make clinical decisions about patient care .
It is important that staff consider the integrity of data in systems. Incorrect information within systems, through wrong data entry or miscommunication, can lead to medical error . Healthcare students need the awareness, skills and experience to deal with issues novel to the electronic environment when generating data. Although technology such as CPOE can reduce the risk of certain error types and cost-savings, research has also shown that digitising the patient record can change communication, coordination of work and workflow patterns [26,27,28,29,30,31]. Medication errors caused by staff interacting and generating data within electronic prescribing systems can introduce new risks to patient safety [9, 31, 33]. For example, new error types such as those created from selecting the wrong patient or wrong drug [32, 34].
The delivery of patient care is dependent on effective communication between healthcare staff and between staff and patients. Failures in the process of communication are one of the leading causes of adverse events in healthcare [7, 35]. Where communication has traditionally been undertaken through verbal and written forms, the digital environment now offers new and exciting ways to augment these processes. It is important that students know how to communicate effectively within the EPR and know when to adjust their modality of communication according to the situation . Training related to EPR communication has also been shown to improve history taking skills and empathetic engagement in patient care .
Within modern healthcare, multidisciplinary teamwork is the norm. This means that access and contribution to the EPR is relevant to all. It is important that different professions across the healthcare team understand each other’s requirements from the system. This collaborative approach needs to extend beyond respect and communication. Users of the EPR across professions must contribute to the generation and review of data, and embrace the coordination of care and sharing of decisions.
Finally, the increasing use of digital records means large volumes of digital data are being generated , which can be used to drive quality improvement in healthcare . The learning outcomes outlined for ‘Monitoring and audit’ ensure that students can effectively capture and interpret data, and can demonstrate respect for the ethical considerations in relation to this type of research.
The domains of competence and associated learning outcomes developed provide an overview of the knowledge, skills, and attitudes needed by healthcare graduates to commence working in a digital healthcare environment. In the same way that healthcare is dynamic and non-linear, many of these learning outcomes will relate to each other, and may not be measurable in isolation. For example, under Communication, ‘Document information relating to the management of patients’ overlaps with learning outcomes in Domain 4 for ‘Generating data’. Similarly, under Communication, ‘Communicate effectively with other healthcare professionals in the electronic environment’ overlaps with ‘Demonstrate effective coordination of care within and across healthcare teams’.
The six domains of competence and associated learning outcomes outlined from this study are designed for use by academics to guide the integration of EPRs into undergraduate healthcare programmes. It has been argued that changes to healthcare curricular (particularly in medical education) can follows “fads” and that changes need to be appropriately evidenced for inclusion since many are already working at capacity . The evidence presented here clearly shows that education with and alongside EPRs is fundamental to the future practice of healthcare professionals and the safe and effective delivery of care in the twenty-first Century. However, the integration of the technology into teaching can be gradual process. Kushniruk et al. (2009) describe two approaches to this: the first is “loose coupling”, where the EPR is demonstrated to students and assignments involve the EPR outside of the classroom, and the second is “tight coupling” where the EPR is fully integrated into teaching, assignments and assessment . The “continuum” as it is described demonstrates how integration of the EPR into education can be varied and on-going process of development and refinement. Importantly, the learning outcomes defined can be used to guide the development of educational initiatives along this continuum.
The implementation of the learning outcomes into curricular requires the technological resource to facilitate delivery—that is the EPR technology for educators to implement for students to interact with. This is likely to be a barrier for many academic institutions. The researchers have been working with an EPR system provider to create a University simulation EPR  so that the education may be delivered through didactic, simulated, experiential and reflective pedagogy with and alongside the EPR.
Strengths and limitations
A number of experts were involved with the development of the domains of competence and associated learning outcomes, who worked in a range of settings including medical education, informatics or healthcare, or a combination of these roles. In addition, two students were present to ensure all the various stakeholders were represented. This was important to ensure that the learning outcomes developed were relevant and could be successfully implemented into undergraduate curricula. The participants for the eDelphi were purposively selected based on their experience working in medical education and/or medication safety/electronic patient records. Although this had the potential to introduce bias and affect the quality of data generated, the researchers (SP and KW) compiled all the feedback and the final version of the domains of competence and learning outcomes were shared with the working group for comment.
Since the subject area was new, with no existing curricula to review, the researchers selected a methodology that would ensure iterative development and refinement of the learning outcomes, to ensure that the final list of would meet the needs of healthcare and meet the requirements of medical educators. However, all participants from both the working group and the eDelphi were UK-based, and therefore the results may not be entirely transferable to other countries.
Finally, as for any outcome-based educational approach, the competencies defined here will require regular review to ensure relevance to practice.
The move to digitise patient records introduces new challenges for healthcare professionals and healthcare students who interact with patients. They also introduce new challenges and opportunities for the academics that provide healthcare teaching. Domains of competence and associated learning outcomes have been identified to guide the teaching of students to work with and alongside EPRs. These are important for ensuring that healthcare students gain structured experience to promote the safe, effective and optimal use of EPRs from an undergraduate level.
Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA. 1998;280(15):1311–6. https://doi.org/10.1001/jama.280.15.1311.
Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, Boyle D, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc. 1999;6(4):313–21.
Radley DC, Wasserman MC, Oisho LE, Shoemaker SJ, Spranca MD, Bradshaw B. Reduction in medication errors in hospitals due to adoption of CPOE. JAMIA. 2013;20(3):470–6. https://doi.org/10.1136/amiajnl-2012-001241.
Nuckols TK, Smith-Spangler C, Morton SC, Asch SM, Patel VM, Anderson LJ, et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Systematic Reviews. 2014;3:56.
PricewaterhouseCoopers LLP (2013). A review of the potential benefits from the better use of information and technology in Health and Social Care. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/217012/DoH-Review-of-Information-and-Technology-Use-Final-Report-V2.pdf. Accessed 4 Feb 2018.
National Information Board. Personalised Health and Care 2020. Using Data and Technology to Transform Outcomes for Patients and Citizens. https://www.gov.uk/government/publications/personalised-health-and-care-2020. Accessed 4 April 2018.
Joint Commission. Sentinel event alert, issue 42: safely implementing health information and converging technologies. December 11, 2008.
Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc. 2013;20(e1):e9–e13.
Redwood S, Rajakumar A, Hodson J, Coleman JJ. Does the implementation of an electronic prescribing system create unintended medication errors? A study of the sociotechnical context through the analysis of reported medication incidents. BMC Medical Informatics and Decision Making. 2011;11(1):29.
Baysari MT, Reckmann MH, Li L, Day RO, Westbrook JI. Failure to utilize functions of an electronic prescribing system and the subsequent generation of ‘technically preventable’ computerized alerts. J Am Med Inform Assoc. 2012;19(6):1003–10. https://doi.org/10.1136/amiajnl-2011-000730.
Mozaffar H, Cresswell KM, Williams R, Bates DW, Sheikh A. Exploring the roots of unintended safety threats associated with the introduction of hospital ePrescribing systems and candidate avoidance and/or mitigation strategies: a qualitative study. BMJ Quality & Safety. 2017;26:722–33. https://doi.org/10.1136/bmjqs-2016-005879.
Health Education England. Framework 15. Health Education England Strategic Framework 2014–2029. https://www.hee.nhs.uk/our-work/planning-commissioning/strategic-framework. Accessed 4 April 2018.
Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010:1923–58. https://doi.org/10.1016/S0140-6736(10)61854-5.
Health Education England. Building a digital ready workforce. https://hee.nhs.uk/our-work/building-digital-ready-workforce. Accessed 5 May 2018.
General Medical Council. Outcomes for graduates. Tomorrow’s doctors. 2015. https://www.gmc-uk.org/-/media/documents/outcomes-for-graduates-jul-15-1216_pdf-61408029.pdf. Accessed 29 Dec 2018.
Ross S, Maxwell S. Prescribing and the core curriculum for tomorrow's doctors: BPS curriculum in clinical pharmacology and prescribing for medical students. Br J Clin Pharmacol. 2012;74(4):644–61. https://doi.org/10.1111/j.1365-2125.2012.04186.x.
Leung WC, Diwakar V. Learning in practice. Competency based medical training: review. BMJ. 2002;325(7366):693–6. https://doi.org/10.1136/bmj.325.7366.693.
Harden RM. Outcome-based education: the future is today. Medical teacher. 2007;29(7):625–9.
Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Medical teacher. 2007;29(7):642–7.
Royal Pharmaceutical Society (2017) Professional Standards for Hospital Pharmacy Services: For providers of pharmacy services in or to acute hospital, mental health, private, community service, prison, hospice and ambulance settings. https://www.rpharms.com/resources/professional-standards. Accessed 12 November 2018.
Cresswell KM, Bates DW, Williams R, Morrison Z, Slee A, Coleman J, et al. Evaluation of medium-term consequences of implementing commercial computerized physician order entry and clinical decision support prescribing systems in two ‘early adopter’ hospitals. J Am Med Inform Assoc. 2014;21(e2):e194–202.
Knight AM, Kravet SJ, Harper GM, Leff B. The effect of computerized provider order entry on medical student clerkship experiences. J Am Med Inform Assoc. 2005;12(5):554–60.
Varpio L, Day K, Elliot-Miller P, King JW, Kuziemsky C, Parush A, et al. The impact of adopting EHRs: how losing connectivity affects clinical reasoning. Med Educ. 2015;49(5):476–86. https://doi.org/10.1111/medu.12665.
Varpio L, Rashotte J, Day K, King J, Kuziemsky C, Parush A. The EHR and building the patient’s story: a qualitative investigation of how EHR use obstructs a vital clinical activity. Int J Med Inform. 2015;84(12):1019–28. https://doi.org/10.1016/j.ijmedinf.2015.09.004.
Ruder DB. Malpractice claims analysis confirms risks in EHRs. Pat Saf Qual Healthc. 2014;11(1):20–3.
Beuscart-Zéphir MC(1), Pelayo S, Anceaux F, Meaux JJ, Degroisse M, Degoulet P. Impact of CPOE on doctor-nurse cooperation for the medication ordering and administration process. Int J Med Inform 2005;74(7–8):629–641.
Pirnejad H, Niazkhani Z, van der Sijs H, Berg M, Bal R. Impact of a computerized physician order entry system on nurse-physician collaboration in the medication process. Int J Med Inform. 2008;77(11):735–44. https://doi.org/10.1016/j.ijmedinf.2008.04.001.
Niazkhani Z, Pirnejad H, de Bont A, Aarts J. Evaluating inter-professional work support by a computerized physician order entry (CPOE) system. Studies in Health Technology and Informatics. 2008;136:321–6.
Pirnejad H, Niazkhani Z, van der Sijs H, Berg M, Bal R. Evaluation of the impact of a CPOE system on nurse-physician communication- a mixed method study. Methods Inf Med. 2009;48(4):350–60. https://doi.org/10.3414/ME0572.
Niazkhani Z, Pirnejad H, Berg M, Aarts J. The impact of computerized provider order entry systems on inpatient clinical workflow: a literature review. J Am Med Inform Assoc. 2009;16(4):539–49.
Niazkhani Z, Pirnejad H, van der Sijs H, de Bont A, Aarts J. Computerized provider order entry system- does it support the inter-professional medication process? Lessons from a Dutch academic hospital. Methods Inf Med. 2010;49(1):20–7. https://doi.org/10.3414/ME0631.
Brown CL, Mulcaster HL, Triffitt KL, Sittig DF, Ash JS, Reygate K, et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc. 2017;24(2):432–40. https://doi.org/10.1093/jamia/ocw119.
Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc. 2013;20(6):1159–67. https://doi.org/10.1136/amiajnl-2013-001745.
Castro GMBL, Hafner JM. The contribution of sociotechnical factors to health information technology-related sentinel events. Joint Commission Journal on Quality & Patient Safety. 2016;42(2):70–6. https://doi.org/10.1016/S1553-7250(16)42008-8.
Dayton E, Henriksen K. Communication failure: basic components, contributing factors and the call for structure. Jt Comm J Qual Patient Saf. 2007;33(1):34–47.
Pontefract SK, Hodson J, Marriott JF, Redwood S, Coleman JJ. Pharmacist-physician communications in a highly-computerised hospital: sign-off and action of electronic review messages. PLoS One. 2016;11:e0160075. https://doi.org/10.1371/journal.pone.0160075.
LoSasso AA, Lamberton CE, Sammon M, Berg KT, Caruso JW, Cass J, et al. Enhancing student empathetic engagement, history-taking, and communication skills during electronic medical record use in patient care. Acad Med. 2017;92(7):1022–7. https://doi.org/10.1097/ACM.0000000000001476.
Luo J, Wu M, Gopukumar D, Zhao Y. Big data application in biomedical research and health care: a literature review. Biomedical Informatics Insights. 2016;8:1–10.
Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010;363(6):501–4. https://doi.org/10.1056/NEJMp1006114.
Kushniruk A, Borycki EM, Armstrong B, Joe RS, Otto T. Bringing electronic patient records into health professional education: towards an integrative framework. Studies in Health Technology and Informatics. 2009;150:883–7.
Khalil S. Universities trial patient record replicator tool. Digital Health. 2017; https://www.digitalhealth.net/2017/12/universities-trial-patient-record-replicator-tool/. Accessed 12 Nov 2018.
The authors would like to thanks all members of the working group and expert participants of the eDelphi.
Dr. Sarah Pontefract is a Lecturer in Clinical Pharmacy and Therapeutics at the University of Birmingham. Dr. Kurt Wilson is Senior Lecturer at the University of Manchester Medical School.
Availability of data and materials
All supporting data is included as additional files.
Ethics approval and consent to participate
Consent for publication
All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author).
SP and KW want to declare (1) Financial support for the submitted work from the National Institute of Health Research. All authors also declare (2) No financial relationships with commercial entities that might have an interest in the submitted work; (3) No spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; (4) No non-financial interests that may be relevant to the submitted work.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Domains of competence and associated outcomes for undergraduate healthcare students developed during the working group prior to external review. (DOCX 18 kb)
Feedback for amendment and refinement from Round 1 of the eDelphi process. (DOCX 23 kb)
Feedback for amendment and refinement from Round 2 of the eDelphi process. (DOCX 22 kb)
About this article
Cite this article
Pontefract, S.K., Wilson, K. Using electronic patient records: defining learning outcomes for undergraduate education. BMC Med Educ 19, 30 (2019). https://doi.org/10.1186/s12909-019-1466-5