Skip to main content

Delivering clinical tutorials to medical students using the Microsoft HoloLens 2: A mixed-methods evaluation

Abstract

Background

Mixed reality offers potential educational advantages in the delivery of clinical teaching. Holographic artefacts can be rendered within a shared learning environment using devices such as the Microsoft HoloLens 2. In addition to facilitating remote access to clinical events, mixed reality may provide a means of sharing mental models, including the vertical and horizontal integration of curricular elements at the bedside. This study aimed to evaluate the feasibility of delivering clinical tutorials using the Microsoft HoloLens 2 and the learning efficacy achieved.

Methods

Following receipt of institutional ethical approval, tutorials on preoperative anaesthetic history taking and upper airway examination were facilitated by a tutor who wore the HoloLens device. The tutor interacted face to face with a patient and two-way audio-visual interaction was facilitated using the HoloLens 2 and Microsoft Teams with groups of students who were located in a separate tutorial room. Holographic functions were employed by the tutor. The tutor completed the System Usability Scale, the tutor, technical facilitator, patients, and students provided quantitative and qualitative feedback, and three students participated in semi-structured feedback interviews. Students completed pre- and post-tutorial, and end-of-year examinations on the tutorial topics.

Results

Twelve patients and 78 students participated across 12 separate tutorials. Five students did not complete the examinations and were excluded from efficacy calculations. Student feedback contained 90 positive comments, including the technology’s ability to broadcast the tutor’s point-of-vision, and 62 negative comments, where students noted issues with the audio-visual quality, and concerns that the tutorial was not as beneficial as traditional in-person clinical tutorials. The technology and tutorial structure were viewed favourably by the tutor, facilitator and patients. Significant improvement was observed between students’ pre- and post-tutorial MCQ scores (mean 59.2% Vs 84.7%, p < 0.001).

Conclusions

This study demonstrates the feasibility of using the HoloLens 2 to facilitate remote bedside tutorials which incorporate holographic learning artefacts. Students’ examination performance supports substantial learning of the tutorial topics. The tutorial structure was agreeable to students, patients and tutor. Our results support the feasibility of offering effective clinical teaching and learning opportunities using the HoloLens 2. However, the technical limitations and costs of the device are significant, and further research is required to assess the effectiveness of this tutorial format against in-person tutorials before wider roll out of this technology can be recommended as a result of this study

Peer Review reports

Introduction

Clinical tutorials which include encounters with real patients are recognised as integral elements in medical education [1,2,3]. Sir William Osler famously stated that “medicine is learned by the bedside and not in the classroom.” [4] However, many medical schools are facing challenges in delivering clinical education to students in an environment where there are increasing numbers of students, a limited number of patients and tutors, and increased scrutiny regarding the costs and environmental impacts of travel [5,6,7,8]. The COVID-19 pandemic also had a significant impact on in-person medical education in many countries, where students’ access to patients was severely curtailed [9, 10]..

The argument that medical education requires interactive tutorials on actual patients is supported by various educational theories. Bandura’s Social Learning Theory and Social Cognitive Theory propose that students learn via attention, retention, reproduction and motivation [11, 12]. This supports the need for direct observation and modelling of relevant clinical role-models participating in doctor-patient interactions [13, 14]..

The Constructivist theory is based on the premise that the act of learning is based on a process which connects new knowledge to pre-existing knowledge [15, 16]. Vertical Integration in medical education involves the integration of aspects of the curriculum across time, namely the integration of basic sciences and clinical sciences [17,18,19]..

Providing medical education within these frameworks, prioritising student exposure to direct interactions with clinicians and patients, and vertical integration of curriculum material, in situations where physical access to patients may be limited by numbers, logistics or infection control concerns poses a significant challenge to medical schools around the world. Utilising technology to facilitate the delivery of clinical education remotely may present a solution to these issues.

The broadcast of bedside tutorials to a remote location can be delivered using a “third-person” perspective, via a fixed or mobile broadcasting device, or using a first-person perspective, via a device mounted on the tutor. Devices which provide a first-person perspective are typically head-mounted-display devices (HMDs). The capabilities of these devices range widely, from basic two-way communication with a remote location, to devices with Augmented Reality (AR) and Mixed Reality (MR) functions which allow the integration of holographic artefacts into tutorials.

Augmented reality (AR) is a virtual environment that allows the user to view both their physical environment and virtual elements in real-time. Mixed Reality (MR) is an extension of AR which allows the real and holographic elements to interact [20, 21]..

The use of AR and MR are expanding in many industries including healthcare, education, engineering, and manufacturing [22,23,24]. MR investigated in a variety of settings pertaining to medical education. Many early studies focused on teaching relevant anatomy, and more recently studies have evaluated the use of MR in procedural training, and its use in streaming of clinical ward-rounds to medical students [25,26,27,28,29,30,31,32,33]..

Head-mounted-display devices which offer MR experiences are growing in number and capability [34].The Microsoft HoloLens2 is one such device which enables the creation of an immersive Mixed Reality environment and can superimpose holographic images onto the user’s surroundings.

The HoloLens 2 has a number of specific capabilities which can be utilised in the virtual delivery of in-person clinical tutorials.The device can facilitate educationally effective, three-way communication between students, tutors and patients, as well as facilitating the incorporation of mixed reality elements into tutorials. The MR capabilities may provide a means of sharing holographic artefacts such as images and diagrams, which can allow the vertical and horizontal integration of curricular elements at the bedside.Utilisation of the MR capabilities of the device may improve student experiences and learning, in particular through instructional scaffolding (e.g rendering cell, organ or system pathways proximate to a patient) [35] Given the device’s connectivity capabilities, students can be in a separate geographical location to the patient and tutor. This has the potential to decrease student travel requirements and enables the delivery of tutorials to students in multiple different locations simultaneously [36]. The tutorial can also be delivered to a greater number of students than would be practical in a traditional bedside clinical tutorial environment. This can decrease the burden on both tutors and patients in comparison to multiple smaller group sessions. Finally, infection control risks are reduced as only the tutor enters patients’ environments.

Study goals

There is little published research to date which robustly evaluates the use of the HoloLens in replicating bedside tutorials while also incorporating mixed reality elements into the tutorials. The aims of this study are to evaluate the use of the Microsoft HoloLens 2 device to deliver a tutorial on preoperative anaesthetic history and upper airway examination to medical students in a remote location, while incorporating MR holograms in the tutorial delivery. Specific objectives include evaluating the feasibility of delivering tutorials with the HoloLens device, assessing the learning efficacy of these tutorials, and assessing student, tutor, facilitator, and patient perspectives of the tutorials.

Methods

This study was approved by the Clinical Research Ethic Committee of the Cork Teaching Hospitals, and the University College Cork Research and Postgraduate Affairs Committee. All participants including students, patients, tutor and technical facilitator provided written informed consent prior to inclusion in the study.

Study population

University College Cork medical students from two cohorts, third year Graduate-Entry and fourth year Direct-Entry medical students attending a tertiary referral teaching hospital for a clinical attachment with the Department of Anaesthesia and Intensive Care Medicine were invited to participate in the study. Both groups are in their second-last year of medical training, and thus have completed modules and examinations in basic medical sciences and clinical practice in the preceding years, with a maximum of 1 week experience in the field of anaesthesia [37,38,39]. Patients attending Cork University Hospital for scheduled surgery were selected and approached for consent by tutors according to clinical relevance. All participants were 18 years or over and were deemed capable of providing consent. Each student provided information on their age, gender and previous third-level qualifications.

Tutorial Sturcture

A one-hour tutorial focusing on completing a preoperative history and focused assessment of the upper airway was developed by MC (adjunct clinical lecturer), GI (Senior Clinical Lecturer) and GS (Professor) in line with the University curriculum’s learning objectives. (Fig. 1) Tutorials were delivered on a weekly basis to groups of third year Graduate Entry and fourth year Direct Entry medical students across the 2021–2022 academic year.

Fig. 1
figure 1

Preoperative Anaesthetic History and Focused Preoperative Assessment of the upper airway tutorial structure

All tutorials were delivered by one tutor (MC) and assisted by a technical facilitator (NOB), both males aged in their thirties, who enabled the connection between the site of the clinical encounter and nearby tutorial room. The tutor had no prior experience with the HoloLens 2 device or other AR HMDs prior to participation in this study; the facilitator had significant experience in its use. The tutor was given a period of familiarisation with the device which included using the Microsoft “HoloLens Tips” app, which provides a structured tutorial on the various hand gestures used to control the device, as well as a number of practice calls in order to test the network and audiovisual equipment in the tutorial room [40]. This familiarisation period totalled approximately 3 hours.

During the tutorial, the tutor (MC) interacted with a patient (face to face) in the pre-or postoperative units and remotely with a small group of [6,7,8,9,10] students in a nearby tutorial room. The remote interaction occurred via Hololens 2 worn by the tutor, institutional Wi-Fi (Eduroam), and Microsoft Teams.He demonstrated and explained the techniques of preoperative history taking and preoperative upper airway assessment.

Throughout the patient assessment the tutor interacted both with the patient and with the students as if conducting an in-person tutorial, providing additional information, asking the students pertinent questions, and expanding on the findings of the patient’s history and physical examination. Students communicated with the patient by asking questions via the tutor.

Resources employed

Resources necessary to provide the tutorials via the HoloLens included capital costs of the HoloLens device (€3500) and microphone (€88) as well as annual licence costs of €275 per user (n = 4). Human resources employed in developing the tutorials and trialling equipment included approximately 20 hours of training, remote assistance (Microsoft) and collaboration between the tutor (MC), Professor (GS) and facilitator (NOB), as well as 5 hours input from the Senior Clinical Lecturer (GI).

Internet connectivity

An internet connection of at least 1.5mpbs of bandwidth is recommended by Microsoft for best audio, visual and content sharing experience [41]. Secure, password protected wireless internet access via the University institutional network (Eduroam) was utilised by both tutor and students.

Hardware

In most tutorials, broadcasts were hosted by an MSI running the Windows 10 operating system, audio was amplified using a Bose SoundLink Mini portable speaker and video was screened via a HDMI cable to a 36″ monitor. In one tutorial students accessed the tutorial via their personal smartphones or laptops. In order to bypass the noise cancellation technology within the HoloLens an external microphone (Saramonic SmartMic+UC L/weight Smartphone Mic USB-C) and 3.5 mm earphone were used.

Software

Dynamics 365 Remote Assist application was used, in-tandem with Microsoft Teams, to host each video call. This connection allowed the students to see the tutors field of vision and hear both the tutor and patient. Hand gestures including the “hand-ray”, “air-tap”, “air-tap and hold” and “start-gesture” were used to control the HMD and manipulate the holographic artefacts. Relevant holographic artefacts were superimposed during the tutorial. This included the insertion of diagramatic representaions of the Mallampati scoring system and Thyromental Distance during the airway assessment portion of the tutorial [Fig. 2 (a) and (b)]. The holographic pointer and “drawing” functions were used by the tutor to highlight relevant upper airway structures and emphasise information on the holographic diagrams [Fig. 2 (c) and (d)].

Fig. 2
figure 2

a Assessment of Mallampati Score. b Assessment of Thyromental Distance. c Identification of thyroid cartilage using holographic pointer. d Illustration of holographic “drawing” function

Assessment of tutor perceptions

Immediately after completion of the first tutorial, the tutor completed a System Usability Scale assessment and on completion of the last tutorial, the tutor and facilitator summarised their perceptions of using the HMD.

Assessment of student perceptions

Immediately after completion of the tutorial, students completed a modified Evaluation of Technology-Enhanced Learning Materials: Learner Perceptions (ETELM-LP) questionnaire in order to assess their perceptions of the tutorial, which incorporated a seven-point Likert Scale and open questions [42]. Cronbach’s Alpha was calculated after exclusion of question 1 and reverse scoring of questions 13 and 15.

Three students also took part in semi-stuctured interviews via Microsoft Teams. Researchers undertook this study from an interpretive approach [43]. The interviews were conducted by JV, and followed a template of questions and corresponding probes from which the interviewer expanded as appropriate [Additional file 1]. The template served as a foundation from which the interviewer expanded as appropriate. The interviews were recorded and transcribed. Analysis of the interview transcripts and questionnare responses was performed using Dedoose Qualitative Research Software Version 4.3.Qualitative data from interviews and feedback questionnaires were coded thematically in alignment with Clarke and Braun’s suggestions for qualitative analysis [44]. Following the initial thematic coding, researchers conducted a content analysis to strengthen the interpretation of results. Illustrative quotes were chosen based on the representativeness of the theme or subtheme and the clarity of their intrinsic interpretation. In alignment with current literature, the quotes selected were determined to be illustrative of the point, reflective of patterns observed, and relatively succinct [45]..

Assessment of patient perceptions

On completion of the tutorials, patients were also asked to complete a mixed quantitative and qualitative questionnaire in order to assess their perceptions of the tutorial.

Assessment of learning efficacy

We carried out a prospective non-comparative study of tutorial efficacy. Students completed a pre-tutorial Multiple Choice Question (MCQ) examination to assess baseline knowledge [Additional file 2], and a post-tutorial MCQ two to 3 days later [Additional file 3]. Students then completed an end-of-year assessment two to 5 months later consisting of a data interpretation exam and an Objective Structured Clinical Examination (OSCE) which focused on preoperative history taking and preoperative assessment of the upper airway respectively [Additional files 4 and 5]. These examinations were written by an investigator and the University Senior Clinical Lecturer in line with University standards. Examination results were converted to percentages and the data interpretation and OSCE results were combined to give a total End-of-Year result.

The Chi-Squared test was used to compare direct-entry and graduate-entry student demographics. Welch’s two-sample t-Test assuming unequal variances was used to compare student group ages. The Shapiro Wilk and Kolmogorov-Smirnov Tests were used to assess to normality of distribution of student assessment scores for data sets less than 50 and greater than 50 respectively. The Mann-Whitney U Test was performed to compare group performance in assessments and overall student performance between the pre- and post-tutorial examinations, and between the post-tutorial and End-of-Year scores. Cohen’s d was calculated for the pre and post-tutorial MCQ scores to assess effect size.

Results

Twelve tutorials were completed involving 12 separate patients and 78 students. Four students did not complete the post-tutorial MCQ and one did not complete the End-of-Year assessments due to illness related absences. These students were excluded from efficacy calculations. Baseline characteristics of the student participants are summarised in Table 1. As expected the graduate-entry students was a significantly older cohort (graduate-entry median age 26 vs direct-entry mean of 22). Mean age of patient participants was 43.25, with an SD of 16.48, and a range of 18–64.

Table 1 Baseline characteristics of student participants

Feasibility

We found that it was feasible to use the HoloLens2 to facilitate weekly bedside tutorials on live patients in a busy, tertiary referral teaching hospital. No tutorials were cancelled or postponed due to technology-related issues. Of note, in order to improve the audio quality of the patient’s voice, it was neccessary to add the USB microphone, which is not routinely supplied with the HoloLens 2. The tutorials were also dependent on secure Wi-Fi access for both tutor and students, the presence of a tutorial facilitator to control the equipment at the student end, and access to a quiet space to examine the patient.

Tutor feedback

The sole tutor (MC) completed the System Usability Scale score, which was 72.5 (a score > 68 is deemed above average). The tutor (MC) stated that the HoloLens 2 was found to be comfortable to wear, the visor was unobtrusive and did not interfere with interaction with the patient or impede visualisation of clinical signs. The interaction with the device via hand gestures was relatively smooth and intuitive after the intial familiarisation period and the MR functions including the insertion of holographic diagrams, pointing, drawing and highlighting were useful. The holographic artefacts were visible throughout the tutorials at a “brightness” setting of seven out of 10.

Occasionally when talking to the students via the HMD, it was not clear to the patient if the tutor was talking to the patient or to the students. Utilising a structured pattern of speech such as “I am now talking to the students” was found to be useful to overcome this issue.

Facilitator feedback

The technical facilitator (NOB) found that the set-up of the live broadcast to the students was akin to that of a video presentation and that the learning curve for hosting the tutorials was short as the Dynamic 365 Remote Assist application was quite similar to general videoconferencing software. He noted that patient proximity to the tutor was essential to ensure adequate audio quality and referenced an example where a supine patient was farther from the device than normal and that patient responses had to be repeated by the tutor. Backgound noise was noted as a “minor issue and transient in nature”, and the technical facilitator accepted that a certain amount of background noise was unavoidable in an active hospital ward.

Student feedback

Quantitative student feedback via the modified ETELM-LP questionnaire is summarised in Fig. 3. Results are presented as (mean, SD) and refer to a seven-point Likert scale. Students had little experience in MR prior to the tutorial (1.7, 1.29). They found the audio and visual quality was clear and that the MR elements of the tutorial were useful. Most agreed the tutorial approximated a live patient encounter (5.69, 1.26), was more beneficial than a PowerPoint-based tutorial, and were neutral when asked if it was as beneficial as a live clinical encounter (5, 1.69). They did not agree that the tutorial structure required inappropriately high technology skill levels on the part of the students, nor that the MR elements served as a distraction. Most agreed that they would like MR to be incorporated into further tutorials (6.05, 1). Cronbach’s Alpha, excluding question 1 was calculated as 0.86, displaying good internal consistency.

Fig. 3
figure 3

Student Modified ETELM-LP Scores. 7 point Likert scale with 7 as strongly agree and 1 as strongly disagree. Presented as Mean +/− 1 Standard Deviation

Student qualitative feedback results

Analysis of written and verbal feedback from 78 students identified 90 specific positive excerpts and 62 negatives (Table 2). Positive feedback included the technology’s ability to broadcast the tutor’s point-of-vision, the inclusion of holographic artefacts, and the remote nature of the tutorial. Negative feedback included issues with the audio-visual stream quality, the fact that students were not able to individually carry out the practical examination, and 11 students expressed concerns that the tutorial was not as useful as traditional in-person bedside clinical tutorials.

Table 2 Student feedback qualitative results

Three students participated in semi-structured interviews. The limited sense of “presence” and interaction with the patient were identified as limitations to the format by all three interviewees. With respect to the physical examination one student explained he would have preferred to “experience it yourself, and have a look and feel and touch”. Specific mention was made of the value of combining broadcast (patient) and rendered (schematics) images, “The adding of the images … right next to the patient was really, really helpful”. This may indicate the potential to employ this format to support vertical and horizontal integration of curricular elements. All three interviewed students reported either a six or seven (on a verbal scale of 1–7) when asked to recommend this technology for inclusion in the medical curriculum.

Patient feedback

Quantitative feedback data from patient questionnaires is summarised in Fig. 4. Most patients had little experience with MR in the past (mean, SD: 1.75, 1.48) apart from one patient who scored 6. All agreed that the communication with the tutor was clear, that they felt safe, that the experience was enjoyable and that they would participate in a similar session in the future. Six of seven expressed that it was preferable to both small (5 or less) and large group in-person tutorials. Most patients did not agree that the HoloLens served as a distraction or made them uncomfortable.

Fig. 4
figure 4

Patient Feedback Questionnaire Results. 7 point Likert scale with 7 as strongly agree and 1 as strongly disagree. Presented as Mean +/− 1 Standard Deviation

Five patients gave qualitative feedback. Positive comments included that “it is good to see that you are moving on with new technology”, “it was well explained beforehand so I was very comfortable” and “it was fantastic to teach students when they can’t be at the bedside. Very unobtrusive”. One patient commented that “sometimes not sure if he [the tutor] was talking to me or the students” and another commented that “it would be lovely to see who I was talking to [the student group]”.

Learning efficacy

Student examination scores are sumarised in Table 3 and Fig. 5. Student assessment scores were not normally distrubuted. A statistically significant improvement was observed between overall students’ pre and post tutorial MCQ scores (mean 59.2% Vs 84.7%, p < 0.001). Cohen’s d was 0.612, indicating a medium effect size. There was a statistically significant difference in student performance between the post tutorial MCQ and the composite End-of-Year scores (84.7% Vs 82.2%, p < 0.05). There were no statistically significant differences found between the graduate-entry and direct-entry students for any individual examination.

Table 3 Student Assessment Scores. Values are mean percentage score (Standard Deviation)
Fig. 5
figure 5

Boxplot of overall student assessment scores

Discussion

Mixed Reality headsets offer several novel capabilities which can facilitate remote education and vertical and horizontal integration of curriculum elements, particularly when aligned with appropriate educational theories such as Constructivism and Social Cognitive Theory. A large number of studies have focused on applying the technology in surgical and anatomical subject fields [46]. However, there are significant gaps in the evidence base, particularly studies specific to anaesthesiology, clinical exam, and addressing the provision of interactive tutorials to remote locations. Our study has demonstrated that it is feasible and effective to use the Microsoft HoloLens 2, incorporating its Mixed Reality functions to provide a live bedside tutorial on anaesthetic preoperative assessment to students situated in a remote location. Feedback from students, patients and the tutor were generally positive. Quantitative feedback from students regarding the audio-visual quality was mainly positive, however technical issues were noted, and preference for in-person tutorials was expressed by a minority of students.

Mill et al. previously examined the feasibility of the HoloLens 2 in broadcasting medical ward rounds [26]. While papers such as that by Mill et al. demonstrated the feasibility of utilizing the HoloLens 2 HMD to stream educational ward-rounds, they did not utilize the MR functions of the HMD, nor assess the learning efficacy of the device [26]. This study incorporates both quantitative and qualitative feedback from multiple sources, namely students, patients, the tutor, and tutorial facilitator. We believe this demonstrates a robust examination of the perceptions of the relevant stakeholders involved in the provision of clinical tutorials to medical students. Our findings that the tutorials were feasible, agreeable to both patients and students, and that students had occasional audio-visual difficulties are consistent with those of Mill et al. Our study additionally demonstrates that incorporation of holographic artefacts is both feasible and regarded by the tutor and students as useful, and that the tutorials provide effective knowledge acquisition.

Our tutorial format aimed to reproduce some of the educationally relevant components of an in-person tutorial. Other suggested structures advocate streaming video of the physician as opposed to the physician’s point-of-view [47]. The HoloLens 2 device allows the students to view the tutor’s field of vision which we argue is superior, and student feedback reflected this. This viewpoint allows students to appreciate in real time the clinical signs demonstrated during the clinical examination and correlate these with the holographic diagrammatic examples used. The MR environment provides an ideal setting to facilitate vertical integration in real time by displaying holographic artefacts of anatomical, physiological and pathological information, as well as patient specific data such as radiological imaging or lab results while interacting with a patient. Furthermore, delivering tutorials remotely reduces infection-control concerns and allows delivery to greater numbers of students in multiple locations.

Preserving patient confidentiality is essential in medical practice and education. In our study, both the HMD and devices at the student end were connected to secure institutional Wi-Fi and accessed via University accounts. Also, access to the audio-visual stream was controlled by the technical facilitator, and the students were located in a supervised tutorial room. It would be essential to control both access to the tutorial and the environment to which it is broadcast to maintain confidentiality.

Limitations

Our study design has a number of limitations. It is non-comparative, and thus we are unable to draw conclusions regarding the relative learning experience or efficacy associated with tutorials delivered via the HoloLens device and the more traditional in-person bedside tutorials. Additionally, the different assessment methods between the MCQs and end of year examinations make direct measurement of knowledge retention difficult. The number of patients involved in the study was relatively small, and thus interpretation of both quantitative and qualitative data must be viewed in this context, and the generalisability of the data is low. The feedback from the tutor and tutorial facilitator must be viewed in the context that they were study investigators.

There are a number of limitations specific to research involving the HoloLens. Common limitations in studying the learning effects of the HoloLens in tested roles include the absence of validated measures and comprehensive evaluation instruments. Unlike other technologies, there are no benchmarks, datasets, or standard standardized protocols to specifically evaluate augmented reality systems, experiences, and methodologies [48,49,50]. Although the viewpoint offered to the students by the HoloLens allows the students to appreciate what the tutor is demonstrating, one drawback to this is that the focus of attention is primarily controlled by the tutor, and thus it is difficult for the tutorial to challenge the students to select the relevant areas to attend to. Depending on the tutorial topic and structure, an ideal virtual format may provide three perspectives: the tutors view, a third person view of the clinical encounter, and where applicable, an instrument’s view.

Regarding the generalisability of our study to other tutorial topics, the appreciation of clinical signs which would require palpation or auscultation would be beyond the current capabilities of the HoloLens 2 and therefore, careful tutorial design and topic selection is necessary.

Conclusions

Our results demonstrate the feasibility of facilitating remote bedside tutorials on preoperative anaesthetic assessment using the HoloLens 2. The tutorial structure was found to be agreeable to students, patients, and tutors. Provision of tutorials in the format described in this study may be an option for situations where students’ access to live bedside tutorials are limited. However, further research is required to characterise the role, potential and limitations of incorporating Mixed Reality into clinical medical education in a broader context. Poor audio-visual quality and lack of hands-on practice were found to be the most frequent issues identified in our study and may be significant limitations to the use of this technology in wider medical education. There are significant costs involved in developing the infrastructure and expertise necessary to provide tutorials in this format. Prior to this technology being adopted by educational institutions, we recommend the completion studies to compare the learning efficacy of MR facilitated remote tutorials and traditional in-person bedside tutorials.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AR:

Augmented Reality

ETELM-LP:

Evaluation of Technology-Enhanced Learning Materials: Learner Perceptions

HMD:

Head-Mounted Display

IQR:

Interquartile Range

MCQ:

Multiple Choice Question

MR:

Mixed Reality

OSCE:

Objective Structured Clinical Examination

SD:

Standard Deviation

References

  1. Keifenheim KE, Teufel M, Ip J, Speiser N, Leehr EJ, Zipfel S, et al. Teaching history taking to medical students: a systematic review. BMC Med Educ. 2015;15(1):159.

    Article  Google Scholar 

  2. Wessely S. Making doctors: an institutional apprenticeship. BMJ. 1998;316(7132):713.

    Article  Google Scholar 

  3. Dornan T, Tan N, Boshuizen H, Gick R, Isba R, Mann K, et al. How and what do medical students learn in clerkships? Experience based learning (ExBL). Adv Health Sci Educ Theory Pract. 2014;19(5):721–49.

    Article  Google Scholar 

  4. Stone MJ. The wisdom of sir William Osler. Am J Cardiol. 1995;75(4):269–76.

    Article  Google Scholar 

  5. Walsh K. E-learning in medical education: the potential environmental impact. Educ Prim Care. 2018;29(2):104–6.

    Article  Google Scholar 

  6. Sherpa JR, Donahue L, Tsai J, Nguemeni Tiako MJ. The planetary benefit of suspending USMLE step 2 CS: estimating carbon emissions associated with US medical Students' travel to testing centers. Yale J Biol Med. 2023;96(2):185–8.

    Article  Google Scholar 

  7. Sharma D, Rizzo J, Nong Y, et al. Virtual Learning Decreases the Carbon Footprint of Medical Education. Dermatol Ther (Heidelb). 2024;14(4):853–9. https://doi.org/10.1007/s13555-024-01120-4.

  8. Medical Schools Council. The expansion of medical student numbers in the United Kingdom. London: MSC; 2021. Available at https://www.medschools.ac.uk/media/2899/the-expansion-of-medical-student-numbers-in-the-unitedkingdom-msc-position-paper-october-2021.pdf. Accessed April 2024.

  9. Harries AJ, Lee C, Jones L, Rodriguez RM, Davis JA, Boysen-Osborn M, et al. Effects of the COVID-19 pandemic on medical students: a multicenter quantitative study. BMC Med Educ. 2021;21(1):14.

    Article  Google Scholar 

  10. Choi B, Jegatheeswaran L, Minocha A, Alhilani M, Nakhoul M, Mutengesa E. The impact of the COVID-19 pandemic on final year medical students in the United Kingdom: a national survey. BMC Med Educ. 2020;20(1):206.

    Article  Google Scholar 

  11. Bandura A. Social learning theory. Oxford: Prentice-Hall; 1977.

  12. Irby DM. Clinical teaching and the clinical teacher. J Med Educ. 1986;61(9 Pt 2):35–45.

    Google Scholar 

  13. Horsburgh J, Ippolito K. A skill to be worked at: using social learning theory to explore the process of learning from role models in clinical settings. BMC Med Educ. 2018;18(1):156.

    Article  Google Scholar 

  14. Park J, Woodrow SI, Reznick RK, Beales J, MacRae HM. Observation, reflection, and reinforcement: surgery faculty members' and residents' perceptions of how they learned professionalism. Acad Med. 2010;85(1):134–9.

    Article  Google Scholar 

  15. Dennick R. Constructivism: reflections on twenty five years teaching the constructivist approach in medical education. Int J Med Educ. 2016;7:200–5.

    Article  Google Scholar 

  16. Steffe LP, Gale J, editors. Constructivism in education. 1st ed. New York: Routledge; 1995.

    Google Scholar 

  17. Kulasegaram KM, Martimianakis MA, Mylopoulos M, Whitehead CR, Woods NN. Cognition before curriculum: rethinking the integration of basic science and clinical learning. Acad Med. 2013;88(10):1578–85.

    Article  Google Scholar 

  18. Wijnen-Meijer M, van den Broek S, Koens F, ten Cate O. Vertical integration in medical education: the broader perspective. BMC Med Educ. 2020;20(1):509.

    Article  Google Scholar 

  19. Brauer DG, Ferguson KJ. The integrated curriculum in medical education: AMEE guide no. 96. Medical Teacher. 2015;37(4):312–22.

    Article  Google Scholar 

  20. Azuma RT. A survey of augmented reality. Presence: teleoperat virt environ. 1997;6(4):355–85.

    Article  Google Scholar 

  21. Hu HZ, Feng XB, Shao ZW, Xie M, Xu S, Wu XH, et al. Application and Prospect of mixed reality Technology in Medical Field. Curr Med Sci. 2019;39(1):1–6.

    Article  Google Scholar 

  22. Garzón J, Pavón J, Baldiris S. Systematic review and meta-analysis of augmented reality in educational settings. Virt Real. 2019;23(4):447–59.

    Article  Google Scholar 

  23. Hantono BS, Nugroho LE, Santosa PI, editors. Meta-Review of Augmented Reality in Education. 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE); 2018 24–26 July 2018.

  24. Hanna MG, Ahmed I, Nine J, Prajapati S, Pantanowitz L. Augmented Reality Technology Using Microsoft HoloLens in Anatomic Pathology. Arch Pathol Lab Med. 2018;142(5):638-644. https://doi.org/10.5858/arpa.2017-0189-OA.

  25. Eckert M, Volmerg JS, Friedrich CM. Augmented reality in medicine: systematic and bibliographic review. JMIR Mhealth Uhealth. 2019;7(4):e10967.

    Article  Google Scholar 

  26. Mill T, Parikh S, Allen A, Dart G, Lee D, Richardson C, et al. Live streaming ward rounds using wearable technology to teach medical students: a pilot study. BMJ Simul Technol Enhanc Learn. 2021;7(6):494–500.

    Article  Google Scholar 

  27. Lia H, Paulin G, Yi N, Haq H, Emmanuel S, Ludig K, Keri Z, Lasso A, Yeo C, Andrews J, Fichtinger G. HoloLens in suturing training. Proc. SPIE 10576, Medical Imaging 2018: Image-Guided Procedures, Robotic Interventions, and Modeling, 2018;1057628. https://doi.org/10.1117/12.2293934.

  28. Carmigniani J, Furht B, Anisetti M, Ceravolo P, Damiani E, Ivkovic M. Augmented reality technologies, systems and applications. Multimed Tools Appl. 2011;51(1):341–77.

    Article  Google Scholar 

  29. Silvero Isidre A, Friederichs H, Müther M, Gallus M, Stummer W, Holling M. Mixed reality as a teaching tool for medical students in neurosurgery. Med Int. 2023;59(10)

  30. Kolecki R, Pręgowska A, Dąbrowa J, Skuciński J, Pulanecki T, Walecki P, et al. Assessment of the utility of mixed reality in medical education. Translat Res Anatom. 2022;28:100214.

    Article  Google Scholar 

  31. Robinson BL, Mitchell TR, Brenseke BM. Evaluating the use of mixed reality to teach gross and microscopic respiratory anatomy. Med Sci Educ. 2020;30(4):1745–8.

    Article  Google Scholar 

  32. Maniam P, Schnell P, Dan L, Portelli R, Erolin C, Mountain R, et al. Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype. J Vis Commun Med. 2020;43(1):17–26.

    Article  Google Scholar 

  33. Kumar N, Pandey S, Rahman E. A novel three-dimensional interactive virtual face to facilitate facial anatomy teaching using Microsoft HoloLens. Aesth Plast Surg. 2021;45(3):1005–11.

    Article  Google Scholar 

  34. Cipresso P, Giglioli IAC, Raya MA, Riva G. The Past, Present, and Future of Virtual and Augmented Reality Research: A Network and Cluster Analysis of the Literature. Front Psychol. 2018;9:2086. https://doi.org/10.3389/fpsyg.2018.02086.

  35. Quarles J, Lampotang S, Fischler I, Fishwick P, Lok B. Scaffolded learning with mixed reality. Comput Graph. 2009;33(1):34–46.

    Article  Google Scholar 

  36. Palumbo A. Microsoft HoloLens 2 in medical and healthcare context: state of the art and future prospects. Sensors (Basel). 2022;22(20)

  37. University College Cork. Book of Modules 2023. [Internet]. Cork. Cited March 2024. Available from: https://ucc-ie-public.courseleaf.com/modules/.

  38. University College Cork School of Medicine. Medicine (Graduate Entry) (MB, BCh, BAO Degrees) - CK791 Programme Requirements 2023. [Internet]. Cork. Cited March 2024. Available from: https://ucc-iepublic.courseleaf.com/programmes/bmbbg/#programmerequirementstext.

  39. University College Cork School of Medicine. Medicine (MB, BCh, BAO Degrees) - CK701 Programme Requirements 2023. Internet. CorkCited March 2024. Available from: https://ucc-ie-public.courseleaf.com/programmes/bmbb/#programmerequirementstext.

  40. Microsoft Corporation. HoloLens Tips 2023. Internet. [Internet]. Cited March 2024. Available from: https://www.microsoft.com/en-us/p/hololens-tips/9pd4cxkklc47?activetab=pivot:overviewtab.

  41. Microsoft Corporation. Infrastructure Guidelines for HoloLens. Microsoft Learn. [Internet]. Cited October 2022. Available from: https://learn.microsoft.com/en-us/hololens/hololens-commercial-infrastructure.

  42. Cook DA, Ellaway RH. Evaluating technology-enhanced learning: A comprehensive framework. Med Teach. 2015;37(10):961–70.

    Article  Google Scholar 

  43. Thompson Burdine J, Thorne S, Sandhu G. Interpretive description: A flexible qualitative methodology for medical education research. Med Educ. 2021;55(3):336–43.

    Article  Google Scholar 

  44. Clarke V, Braun V. Teaching thematic analysis: overcoming challenges and developing strategies for effective learning. Psychol. 2013;26:120–3.

    Google Scholar 

  45. Lingard L. Beyond the default colon: effective use of quotes in qualitative research. Perspect Med Educ. 2019;8(6):360–4.

    Article  Google Scholar 

  46. Tang YM, Chau KY, Kwok APK, Zhu T, Ma X. A systematic review of immersive technology applications for medical practice and education - trends, application areas, recipients, teaching contents, evaluation methods, and performance. Educ Res Rev. 2022;35:100429.

    Article  Google Scholar 

  47. Pennell CE, Kluckow H, Chen SQ, Wisely KM, Walker BL. Live-streamed ward rounds: a tool for clinical teaching during the COVID-19 pandemic. Med J Aust. 2020;213(7):306–8.e1.

    Article  Google Scholar 

  48. Herron J. Augmented reality in medical education and training. J Electron Resour Med Libr. 2016;13:1–5.

    Google Scholar 

  49. Gerup J, Soerensen CB, Dieckmann P. Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. Int J Med Educ. 2020;11:1–18.

    Article  Google Scholar 

  50. Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc. 2016;30(10):4174–83.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the assistance from members of the UCC College of Medicine and Health, including Dr. Colm O’Tuathaigh, Dr. Gabriella Rizzo, Dr. Pat Henn and Professor Paula O’Leary, as well as Ms. Michelle Donovan in the UCC Centre for Digital Education.

Funding

This study received funding and research support through the UCC Learning Analytics LITE programme, which is funded through the Strategic Alignment of Teaching and Learning Enhancement fund. The UCC Learning Analytics LITE programme provided logistical and research support in study design and funds were used to hire assistance in data interpretation.

This study also received funding from the UCC College of Medicine and Health which was utilised to purchase the HoloLens 2 Device and associated licences.

Author information

Authors and Affiliations

Authors

Contributions

MC lead the design of the study, carried out the tutorials, analysed both quantitative and qualitative data and was the primary author of the manuscript. GI contributed to the design of the study, the student examinations contributed to writing the manuscript. NOB contributed to the technical and logistical design of the study and acted as technical facilitator for the tutorials and contributed to manuscript composition. JV designed, completed and analysed the semi-structured student interviews and contributed to manuscript composition. AOM, PS, AB, LG and supervised and analysed student examination data. KGG analysed student demographic data and student examination data. CS contributed to initial evaluation of the HoloLens device and tutorial design. GS played a central role in study design and completion and was a major contributor in manuscript composition.

Corresponding author

Correspondence to Murray Connolly.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Clinical Research Ethic Committee of the Cork Teaching Hospitals, and the University College Cork Research and Postgraduate Affairs Committee. All methods were carried out in accordance with guidelines and regulations as set out by the ethics and research committees. All participants provided informed consent to participate in the study.

Consent for publication

All participants including students, patients, tutor and technical facilitator provided written informed consent prior to inclusion in the study. Participants who’s identifiable images are included provided informed consent for publication of identifiable information/ images in an open access journal.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Connolly, M., Iohom, G., O’Brien, N. et al. Delivering clinical tutorials to medical students using the Microsoft HoloLens 2: A mixed-methods evaluation. BMC Med Educ 24, 498 (2024). https://doi.org/10.1186/s12909-024-05475-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12909-024-05475-2

Keywords