Blended learning approach improves teaching in a problem-based learning environment in orthopedics - a pilot study

In recent years, there has been a growing interest in research on the education and learning progress of students in medicine. Great effort has been put into knowledge transfer via internet-based electronic learning (e-learning) [1].

E-learning has also become an integral part of medical education [2–10]. Various authors have shown that greatest benefit and student satisfaction is achieved when combining e-learning with face-to-face courses, as blended learning[2–4]. Blended learning comprises the systematic integration of online and face-to-face engagement to support and enhance a meaningful interaction between students, teachers and resources [11, 12]. When attempting to successfully facilitate the transfer of knowledge, it is essential that teaching be competent, appealing, and recipient oriented [5]. In this context, e-learning can be achieved e.g. by providing videos [6], podcasts [7] or interactive diagnostic tools [8], leading to a considerable improvement of knowledge transfer capabilities in a mix with face-to-face lessons [9, 10]. However, more studies are still needed to proof the impact of e-learning and blended learning on the enhancement of students’ knowledge and clinical skills in general and in the field of orthopedics and traumatology particularly.

After the launch of e-learning courses in recent years, many studies have focused primarily on evaluating students’ satisfaction – an important factor in acceptance and use of e-learning [13–15]. Others have additionally analyzed e-learning’s influence on the acquisition of knowledge or skills [3, 6, 8, 9, 12, 16].

In traumatology and orthopedics education such studies are comparatively still rare [13, 17, 18], despite the fact that e-learning might substantially improve quality and success of teaching in these highly clinically and practically oriented disciplines. While the number of musculoskeletal diseases and injuries has been steadily increasing over the last years [19], data indicate that medical student education concerning the diagnosis and treatment of musculoskeletal diseases should be enhanced [20]. Here, e-learning could add appeal [21] and promote better knowledge and clinical skills [18] by providing useful multimedia adjuncts (e.g. videos, podcasts, or radiological cases). However, more data is still needed to guide the design of blended learning curricula in these subjects, questioning especially to what extend the use of e-learning might be beneficial and to investigate the effect of different approaches or configurations of e-learning.

The findings should allow a more informed discussion about the aspects that may have to be considered when integrating blended learning approaches into a PBL curriculum of orthopedics and traumatology.

The purpose of this pilot study was to give first impressions of the effect of a blended learning concept in orthopedics and traumatology called NESTOR both on students’ satisfaction and on its contribution to acquisition of knowledge and clinical skills in a problem-based learning curriculum, which already provides an intensely practice-oriented teaching environment. To the best of the author’s knowledge, this is the first study examining the influence of blended learning not only on satisfaction, but also on knowledge and practical clinical skills of students in traumatology and orthopedics, two highly practically oriented medical subjects.

Evaluations of students’ opinion and acceptance can be seen as first step when establishing a new e-learning program [4, 14, 21]. Consistent with the literature, this study revealed a high approval of the participating users for the additionally offered e-learning contents. While a broad acceptance is crucial for successful e-learning implementation [1], it is also important to evaluate its influence on students’ gain of knowledge and skills [9].

Thus, as second step, not only the impact on users’ satisfaction, but also on their knowledge should be demonstrated [6, 21, 22]. For this pilot study we have chosen newly developed written tests to evaluate improvement in theoretical knowledge, which seem to be valid as indicated by the measured Cronbach’s Alpha values. While some studies showed benefits of e-learning on improvement of students’ knowledge [6, 23] others did not – despite of positive evaluation [2, 8, 22]. In this pilot study, we found a significant improvement from pre- to post-tests for both groups. NESTOR users scored better in the written post-tests than non-users and showed further improvement between post-test 1 and 2. A possible interpretation for this success in the group of NESTOR users may be students’ very positive attitude towards e-learning and a high satisfaction with structure and contents. However, these results should be interpreted carefully, especially when referring this effect exclusively to the use of NESTOR. In a recent review, Rowe et al. [12] showed that the existing data to evaluate an improvement of clinical competencies by blended learning can still be regarded as rudimentary. It seemed to be a problem of the study design in a clinical environment to determine the effect of blended learning exclusively. Rowe et al. identified 71 studies dealing with the role of blended learning in the clinical education of healthcare students, but only 7 articles were enrolled for the review due to methodological flaws of the remaining 64.

As potential third step it might be anticipated that even practical clinical skills may be improved through blended learning in this context [9], which has been shown in some studies [9, 16, 24]. The results of the OSCE revealed no differences between users and non-users in this pilot study with high scores in both groups. These findings are consistent with other studies which failed to detect significant benefits on examination performance or other practical clinical skills with e-learning implementation [21, 25, 26]. A possible explanation could be that the pre-existing, highly clinically oriented curriculum made it difficult for any additional e-learning exposure to further improve skills.

In this context, the question may arise what e-learning potentially can achieve [1]. It can be argued that clinical examination skills will always be preferentially based on personal experiences and training rather than on the use of e-learning – unlike acquiring skills in other areas such as radiological diagnosis [16]. In the presented pilot study, e-learning enhanced competencies for gaining theoretical medical knowledge. Further research will be necessary to determine, if it is possible to adjust the components of a blended learning approach in this context to achieve also an improvement of practical skills compared to mere face-to-face teaching. However, as knowledge about diseases is an important basis for developing treatment and examination skills, this and the overall high approval provide good arguments for the continued use of NESTOR in the preparation for the tested subjects. Following suggestions of the non-users, acceptance of the program might be further increased by improving announcements about it. Additionally, it could be made even more appealing with links to e-learning programs of other subjects (e.g. anatomy) of the faculty.

Concerning the willingness to use e-learning offerings voluntarily, an additional inference can be taken from this study en passent: As the use of NESTOR was significantly linked to the use of LMS Blackboard, the likelihood of voluntarily using an e-learning offering may be directly connected to the acceptance and use of the hosting LMS. This would require the need for the entire faculty to join in a combined effort to establish e-learning offerings broadly to increase students’ familiarity with such resources. Thereby, voluntary and perhaps even mandatory use of e-learning components could be increased.

There are some limitations to be noted in this study. A selection bias cannot be excluded due to the voluntary nature of participation and use of NESTOR, also with respect to the significant correlation between this aspect and a pre-existing use of the LMS Blackboard. This can be seen as main shortcoming, which was tolerated because data evaluation was incorporated into an ongoing mandatory course to establish this new blended learning concept. This pilot study design guaranteed a high practical orientation and a sufficient number of participants. However, for final conclusions on the chosen blended learning technique a randomized controlled trial should rather be performed in the future. Furthermore, preexisting experiences in students’ physical examination skills will have to be evaluated in the final implementation study by a pre-test. The possibility that the exposure to or interaction with any additional resources or experiences led to an increase in knowledge (i.e. due to a possibly more in-depth coverage of the topic), cannot be excluded completely with the study design. In a future study, there should also be a tracking function to detect the individual accessed e-learning tools or the time students spend with them. Such a study design would also avoid the present division of users and non-users merely according to their own declaration, as this step does not guarantee a correct assignment of data to the test and control group, respectively.

Finally, no direct correlation between test results and questionnaires was possible due to guaranteed complete anonymity. This could have potentially given more information about individual user’s attitudes towards the program and their concomitant test results.

This pilot study could underline that it is possible to achieve an improvement in theoretical knowledge combined with high acceptance of students with a blended learning program. The results indicate that the blended learning concept might be superior compared to face-to-face teaching alone, even in the setting of a problem-based learning environment where a high level of self-reliant learning has already existed. Future research on the presented concept should assess which blended learning scenarios might support best students’ acquisition of practical examination skills and identify further crucial points in knowledge and competence transfer to support the improvement of teaching in this context. Therefore well designed randomized controlled trials within realistic clinical education scenarios are still needed.