Innovation in early medical education, no bells or whistles required

Background Despite a paucity of evidence to support a multitude of educational innovations, curricular leaders are pressured to find innovative solutions to better prepare medical students for an evolving twenty-first century health care system. As part of this effort, this study directly compared student-rated effectiveness scores of six different learning modalities. Methods Study participants included 286 medical students enrolled in the second-year rheumatology core at a single academic medical center between 2013 and 2017. Students were surveyed at the end of the core with a 15-item questionnaire, and student perceived effectiveness of six different learning modalities were compared. Results The modality that outperformed all others was Live Patient Encounters (LPE), with significantly higher student-rated effectiveness scores when compared to the referent modality of Problem-Based Learning (PBL). Using a 5-point Likert scale with responses ranging from “not effective” to “highly effective,” LPE received a mean effectiveness score of 4.77 followed by Augenblick (4.21), PBL (4.11), Gout Racer video game (3.49), Rheumatology Remedy e-module (3.49), and simulation knee injection (3.09). Conclusions Technologically advanced novel learning strategies were outperformed in this study by the more traditional active learning modality of LPE. This finding highlights the importance of testing innovative learning strategies at the level of the learner. Three additional conclusions can be drawn from this result. First, conflation of technology with innovation may lead to a myopic view of educational reform. Second, human factors seem to be responsible for the success of LPE and may have far-reaching educational rewards. Third, further applications of LPE should be tested in non-rheumatologic curricula. The relevance of this study is innately tied to the humanities-based application. While a formal qualitative analysis was not performed in this study, preliminary results suggest that live, structured patient interactions in the pre-clinical years of medical education may not only promote the learning of important educational objectives but also foster professional development, empathy, reflection, leadership, agency, and interpersonal skills. This “win-win” scenario (if true) would stand out as a rarity among strategic educational initiatives.


Background
Integrating novel pedagogical techniques can be challenging to medical educators tasked with curricular reform [1]. While it may be tempting to adopt innovative curricular changes, these efforts are often based on a paucity of evidence. Despite the widespread application of novel educational modalities, direct comparison data are rare, and specific curricular prescriptions are non-existent. Although active learning strategies are considered superior to traditional lecture format, strategic implementation of specific active learning components has been less clearly defined [2]. As a result, curricular leaders are often conflicted with how to integrate small groups, e-learning, and traditional lecture to achieve the best possible learning experience [3,4]. Unfortunately, evolving modalities of information delivery can merely add to the confusion [5].
Additional barriers to innovation include cultural inertia and limitations in time, finances, faculty, and technological support. Assuming these practical barriers can be overcome, a sense of uncertainty remains due to the inadequacy of comparison data to make relative value decisions with respect to various innovative modalities [6]. Although this should be the most critical factor driving educational innovation, no study to date has tested multiple innovative educational modalities in head-to-head fashion within an already established medical school curriculum.

Methods
The purpose of this study was to test six active learning modalities and compare learner perceived effectiveness. With the exception of Problem-Based Learning (PBL), traditional lecture had been the mainstay of information delivery in the rheumatology curriculum at the University of Nebraska Medical Center (UNMC) until 2014. From 2014 to 2017, five additional innovative learning strategies were deployed for second year medical students (Table 1).
Our aim was threefold: (1) to implement and test a combination of synchronous and asynchronous active learning components (with purposeful redundancy) in order to enhance the learning experience in the rheumatology curriculum, (2) to integrate these modalities seamlessly, without compromising student satisfaction, and (3) to study how these modalities would be received by students to inform future curricular changes.
Upon completion of the final rheumatology examination, all second year medical students were verbally consented to answer a voluntary and anonymous 15question survey (embedded in a larger post-core questionnaire) regarding their perceptions of the effectiveness of each learning modality experienced during the core (Appendix 1). This survey was administered yearly at a single academic medical center between 2013 and 2017.
Learning modality effectiveness was assessed using a 5-point Likert scale with responses ranging from "not effective" to "highly effective." Mean effectiveness scores were then compared between learning modalities using one-way ANOVA with post-hoc pair-wise comparisons and Scheffe's method to adjust for multiple comparisons. Because PBL had been a staple of active learning within the rheumatology curriculum at UNMC for several years, it was treated as the referent modality in post-hoc comparisons. Statistics were run using Stata SE 14.2 software (Stata Corp, College Station, Texas).
A brief description of each innovative modality is provided below:

1) Live Patient Encounters (LPE)
Ten stations were developed including polarized microscopy for crystal analysis, musculoskeletal ultrasound, and eight patients recruited from clinic with representative rheumatic diseases. Patients consented to having their labs, x-rays, and photographs available when appropriate. The class was divided into ten groups and groups rotated through each station in 15-min intervals over a 3 h time period, including a patient break in the middle. During each station, patients were allowed to lead the group through their history, and students were encouraged to ask questions and examine each patient. Faculty facilitators were present to answer medical questions. LPE occurred after each of the representative diseases had been formally taught in lecture. At the end of the encounters, the students completed a low-stakes quiz.

2) Augenblick
By definition, augenblick means "blink of an eye" or "moment." Thirty pathognomonic rheumatic disease pictorial findings (with two associated high yield questions) were presented to students in a Power Point format. Initially, student-led small groups met to work through the augenblick cases. Rheumatology faculty then supervised each student group at a later date to review answers and facilitate discussion.

3) Problem-Based Learning (PBL):
Students independently met twice (small groups of 10) to review two electronic cases with timed release of information followed by additional questions. Rheumatology faculty then supervised each student group at a later date to demonstrate how a clinician would work through the cases, answer questions, and highlight the learning objectives. At the completion of the cases, students completed a low-stakes quiz and were given a study guide to review important learning points.

4) Gout Racer video game:
Rheumatology faculty teamed with the College of Information Science and Technology at the University of Nebraska Omaha (IST at UNO), to develop a gout-themed video game. Through visually-rich graphics, students were challenged to navigate a dune buggy (the "Gout Racer") through a series of obstacles and differing terrain representing the pathophysiology, clinical presentation, and treatments of gout. Bonus points and hazards were utilized to visually and audibly reinforce important clinical content. Within each terrain, students had to answer multiple choice questions in order to advance to the next level. Students were given immediate feedback with explanations after each question attempt. This video game was voluntary and no formal grade or assessment was linked to student performance. As an incentive to play the game, students were informed that questions from Gout Racer would appear on the final exam.

5) Simulation Knee Injection:
Students were asked to view a preparatory video on joint injection techniques followed by dedicated time in the simulation lab to practice knee injections. No faculty supervision was assigned during the first 2 years of the study. In response to student feedback, this activity was modified in the latter 2 years of the study to include a 20-min lecture with faculty demonstration of injection technique followed by direct faculty supervision of simulated knee injections.

6) Rheumatology Remedy e-module:
The e-learning lab at the IST at UNO supported the development of an interactive, inter-professional, e-learning module that was accessible to students throughout the entirety of the rheumatology core. The module highlighted both pharmacologic and nonpharmacologic therapeutics. This formative assessment included 100 multiple choice questions with immediate feedback. The module was voluntary, and no formal grade or assessment was linked to student performance.

Effectiveness of learning modality
Of the six modalities tested, LPE was associated with the highest student perceived effectiveness ( Fig. 1) with a mean effectiveness score of 4.77 followed by Augenblick (4.21) and PBL (4.11). The three least effective innovative modalities were the Gout Racer video game (3.49), the Rheumatology Remedy e-module (3.49), and the simulation knee injection (3.09). Mean effectiveness scores with standard deviations are summarized in Table 2.
Based on analysis with ANOVA pair-wise comparisons, LPE outperformed PBL in student-rated effectiveness (p < 0.001). While Augenblick had a higher mean effectiveness score, the difference in relation to PBL was not statistically significant.
Of note, 97% percent of students perceived LPE to be highly or moderately effective and LPE was the only modality perceived by a majority of students to be highly effective ( Table 2). A qualitative analysis of survey responses was not performed in this study. Examples of student responses can be found in Appendix 2.

Discussion
As the needs of the twenty-first century health care system continue to evolve, medical students must be trained to meet a multitude of professional demands. Given restraints on time and resources, curricular reform leaders have placed an emphasis on innovation and novel learning strategies [7]. As part of this national effort, UNMC has implemented five innovative learning modalities within the rheumatology core and tested them against PBL-a benchmark of active learning within the prior UNMC curriculum.
Although effectiveness scores varied amongst each of the educational modalities, one clearly stood out from the rest. LPE consistently outperformed the other active learning strategies over the course of 4 years and was well received by greater than 97% of students. The degree to which this occurred is somewhat surprising as most of the active learning literature to date emphasizes the importance of selfdirected study, inquiry-based learning (e.g. PBL), blended learning (with use of asynchronous elearning), simulation, and gamification [8][9][10][11][12]. LPE is not well represented in the literature, has no technological prowess, and only requires 3 h of dedicated student time; yet it was able to outperform the more widely accepted, technologically-advanced, asynchronous, and traditional active learning strategies (including PBL).  Table 2 Summary of mean effectiveness scores as reported by students. Mean effectiveness scores were analyzed with one-way ANOVA using post-hoc adjustments for pair-wise comparisons relative to PBL ("baseline") and included Scheffe's method for multiple comparison adjustment This irony highlights the importance of testing innovative learning modalities at the level of the learner. Interestingly, the utility of LPE has been well documented in rheumatologic curricula for decades but its scope of use pales in comparison to other innovative learning modalities with less supporting evidence but greater technological appeal [13,14]. In an era where technology is often conflated with innovation and evidence is in relatively short supply, curricular leaders are left to "innovate" based on cultural readiness and limited resources. As such, innovations may be adopted more so on the basis of popular trends and consensus rather than merit. Moreover, modalities requiring significant investment in time and energy (like LPE) are unlikely to gain traction without substantial evidentiary support. Ultimately it seems "proof of concept" is not good enough for curricular prescriptions, and head-to-head comparison data is required to make relative value decisions. To our knowledge, we are the first to study LPE in this manner.
Our results may highlight a potential bias within educational reform initiatives that favors technology. Ultimately, technological advances may be a step away from humanism and could compromise the learning of important cross-cutting domains [15] such as professional development, empathy, and agency within the health system sciences. This perspective is important in balancing an otherwise myopic view of educational reform.
It should not come as a surprise that medical students crave patient contact particularly in their preclinical years. Although we hesitate to make claims attributing the entirety of LPE's success to the human factors element, the association is difficult to ignore as this is the main difference between LPE and other innovative educational modalities. Live patients activate the affective domain of learning [16] in ways that other modalities simply cannot. This domain is important in assimilating long-term retention of knowledge and facilitating student identity formation and professional development [17]. The hidden curriculum is now widely recognized as an important factor in undergraduate medical education, and early exposure to real patients is strongly advised as one strategy to accomplish this aim [18]. As such, we suggest that LPE may not only promote the learning of educational objectives but also foster professional development, empathy, reflection, leadership, agency, and interpersonal skills [19][20][21]. This "win-win" scenario would stand out as a rarity among strategic educational initiatives [22].
More research is needed to investigate long term outcomes of LPE, and replication of our findings at external sites is recommended. Additionally, we recommend "proof of concept" trials in non-rheumatologic curricula as the application of this modality may not be generalizable to the learning of representative diseases outside of rheumatology.
Directly testing the acquisition of medical knowledge with LPE would also be an important metric to consider. Assessment of knowledge acquisition was not feasible in this study primarily because the curriculum at UNMC has a competing priority of planned redundancy that precludes attribution of knowledge to any singular modality. For instance, gout was intentionally taught in LPE, PBL, Augenblick, Rheumatology Remedy, and the Gout Racer video game. Any attempt to isolate the effects of one modality would defeat the purpose of having multiple exposures to the same educational content. It should be noted, however, that learner satisfaction has been indirectly linked to knowledge acquisition when evaluating novel educational modalities [23]. Thus, when knowledge cannot be directly assessed, student-rated effectiveness may be an appropriate surrogate metric.
Another limitation of this study is that it may be subject to survey sampling bias. With greater than a 50% response rate, we feel respondents accurately represented students at UNMC although we do not have demographic data to differentiate respondents from nonrespondents. More importantly, students at UNMC may not be representative of all medical students. Similarly, the individual modalities described may not reflect the practices or implementation strategies used at other institutions.
Despite these limitations, we feel the rewards of implementing LPE into rheumatologic curricula outweigh the costs in time, management, and organization (Appendix 3). This innovative modality was well received by students within our institution, and future rewards of LPE may be yet to be seen. As we continue to navigate an evolving health system, educators should strive for innovative learning solutions that foster knowledge acquisition, professional identity formation, and learner satisfaction.

Appendix 1
Survey from 2017 with some parts excluded

Appendix 2
Selected student survey responses regarding Live Patient Encounters "Best thing ever. I could remember the diseases because I could put a face to the disease, and thus the findings. It was inspiring to hear the real patient stories." (M2 from 2014) "This was one of the most effective forms of education I have been presented while at UNMC. What better way is there to learn medicine than to see real patients with real diseases?" (M2 from 2014) "Talking with patients who suffered from the diseases was by far the best small group experience we have ever had in medical school. I almost categorically do not enjoy small group, but this was really worth our time. These patients will probably remain in my memory for the rest of my career." (M2 from 2014) "I will never forget these cases. I also learned how to empathize with patients in a way that I never would learn in a classroom." (M2 from 2014) "Patient small groups were the best thing we have done in medical school." (M2 from 2014) "I LOVED the day the patients came. It brought back the humanity of medical school, and I will always remember those patients." (M2 from 2014) "The real patient interaction was phenomenal, and permanently impacted the way I will view rheumatology cases for the rest of my career." (M2 from 2014) "I am very grateful for these patients being willing to share their stories with so many students/ strangers. It made the material much more tangible, and was a great opportunity to put a real face on some very difficult diseases. It's easy for us to just memorize the facts and completely miss the human factor in these diseases. Awesome part of this core!" (M2 from 2014) "As I was taking the test, I could picture the specific patients as I was thinking about the disease. SO BENEFICIAL!" (M2 from 2014) "One of the best forms of medical education that I have experienced in the first 2 years. Wish we had this for every core. Keeps our head focused in the right direction. Gives us a chance to put a face & experience to a disease. Very very very helpful and highly recommended to continue (and advise other faculty to do something similar for their cores if possible)." (M2 from 2015) "So far, this was the highlight of my M2 year. It was extremely helpful and I can tell I will remember the things we discussed with the actual patients for a long time. Even during the exam, if there was a lupus question, I would envision the patient with lupus and some of the things that she talked about, and did the same for scleroderma, dermatomyositis, etc." (M2 from 2015) "The patient stations were awesome and this was the most effective way of learning for me.

Core Director Considerations
Recommended timeline and steps required for the successful implementation of Live Patient Encounters (LPE) into a medical school curriculum 1) Identify patients with representative diseases from clinics (4 months prior to encounter) 2) Invite patients to participate via telephone, with immediate email confirmation (4 months prior to encounter) 3) Prepare power-point with images from patient records for student review prior to encounter 4) Prepare quiz for students upon completion of encounter 5) Contact patients to confirm participation and give logistics for the day (2 weeks prior to encounter) 6) Day of encounter a. Administrative assistants and nursing facilitate getting patients from designated meeting point to encounter location b. Faculty transport ultrasound equipment and supplies and microscope (with MSU/CPPD slides) to encounter location c. Faculty and administrative assistants monitor time and alert room changes d. Refreshments are provided to patients and faculty (rolls, coffee, tea) 7) Students sign a thank you card for each patient 8) Send thank you letter from Block Director and Dean with selected student comments, students' signed card and $50 reimbursement (1 month after encounter) Author contact information: Please direct any specific questions to Amy C. Cannella, MD, MS at acannel-la@unmc.edu A summary of administrative, faculty, technological, and financial requirements estimated for each innovative learning modality can be found in Table 3 in Appendix 4.  These are costs incurred in addition to faculty and administrative support for the time required for development, implementation, and ongoing administration of each educational modality