Skip to main content

Development of the EVIBEC Learning Outcomes Framework to support the delivery of evidence-based practice curricula in health care professional programmes: a codesign approach

Abstract

Background

All healthcare professional education programmes must adopt a systematic approach towards ensuring graduates achieve the competencies required to be an evidence-based practitioner. While a list of competencies for evidence-based practice exist, health care educators continue to struggle with effectively integrating the necessary competencies into existing curricula. The purpose of this project was to develop an open access cross-discipline, learning outcomes framework to support educators in integrating the teaching, learning and assessment required to ensure all graduates of health care professional programmes can achieve the necessary evidence-based practice competencies.

Methods

An interdisciplinary team of health care professional educators and a librarian completed a review of the health professions literature on the teaching and assessment of evidence-based practice. The literature, coupled with the teams’ collective experiences in evidence-based education and research, were used to identify relevant teaching, learning and evidence-based competency frameworks to inform the project design. The guide and toolkit for experience-based co-design developed by the National Health Service Institute for Innovation and Improvement was adopted for this study ( Institute for Innovation and Improvement: Experience Based Design: Guide & Tools In. Leeds: NHS; 2009.). A four-step approach involving three online participatory co-design workshops and a national validation workshop was designed. Students (n = 33), faculty (n = 12), and clinical educators (n = 15) participated in formulating and mapping learning outcomes to evidence-based competencies.

Results

Through a rigorous, systematic co-design process the Evidenced-based Education Collaborative (EVIBEC) Learning Outcomes Framework was developed. This framework consists of a series of student-centred learning outcomes, aligned to evidence-based practice competencies, classified according to the 5 As of EBP and mapped to the cognitive levels of Bloom’s taxonomy. Associated learning activities for each step of EBP are suggested.

Conclusions

A consensus-based, student-centred learning outcomes framework aligned to a contemporary set of EBP core competencies has been developed. The freely accessible EVIBEC framework may support entry level health care professional EBP education, by informing EBP curriculum development and offering the potential for interdisciplinary approaches to and sharing of valuable teaching and learning resources. Co-design proved an effective method in creating and refining this framework.

Peer Review reports

Background

Evidence based practice (EBP) is defined as the integration of best research evidence with clinical expertise and patient values [1] It is recognised as fundamental for safe, high-quality healthcare and therefore should be an essential component of all healthcare professional (HCP) education programmes [2,3,4,5]. Despite this recommendation, one of the long-recognised barriers to EBP education is lack of consistency in EBP curricular content [6,7,8]. Recent reports also highlight that effectively embedding EBP throughout HCP curricula remains a particular challenge [9,10,11,12]. While academics are supportive of teaching EBP, barriers such as competing job demands, lack of faculty skills, knowledge, and confidence, or supporting curricula frameworks impede progress [12,13,14,15,16,17]. These academic barriers impact HCP student EBP competency and contribute to deficits that continue to undermine healthcare quality, safety, and patient outcomes [15, 18].

In Europe, in 2005, an international working group of EBP health care educators and developers published the Sicily consensus statement on EBP [19], comprising a five-step, 5As model (Ask, Acquire, Appraise, Apply, Assess) for teaching and conducting EBP, and a description of core competency requirements. Since then, various EBP competency frameworks or statements have been developed independently by most HCP disciplines and their regulatory bodies [12, 20]. Despite this, the lack of uptake by healthcare educators suggests they may not meet the needs of all disciplines or educators [14, 21]. Additionally, where EBP has been integrated, inconsistencies in curricular content and the quality of teaching have been noted [8, 22]. Aiming to standardise and further enhance teaching, Albarqouni and colleagues [22] in 2018 developed an international, consensus-based, set of core EBP competencies for entry-level HCP programmes. This approach offers a universal shared language with clear benchmarks to inform EBP curricular development.

Whilst defining a set of core competencies for EBP education was a necessary first step for the implementation of effective EBP education, such competency frameworks and statements provide a broad overview only. They do not identify in any detail the knowledge, skills, and attitudes (KSAs) required to be an effective evidence-based practitioner or articulate these as actionable learning outcomes. Since the Bologna agreement [23], outcomes-based curriculum design has increasingly been recognised as the appropriate approach for all third level institutions [23, 24]. Learning outcomes are defined as: “statements of what a learner is expected to know, understand and/or be able to do at the end of a period of learning” [25], and place the emphasis on student achievement of learning rather than content delivery [26].

The next logical step towards supporting EBP educators is to disaggregate the competencies of Alberqouni et al. [22] into a series of transparent and explicit achievable learning outcomes. These learning outcomes should encompass multiple cognitive learning levels allowing scope for progression and vertical integration of EBP learning from early to late programme stages. In addition, they should also be expressed from a student perspective in a way that students can clearly relate to, requiring a development process that is most effectively achieved using a co-design process [27,28,29,30]. Given that EBP competencies are required by multiple HCP disciplines, there is huge potential gain from an interdisciplinary approach to curriculum design, with the collaboration of an interprofessional team of HCPs offering additional benefits [31].

The overall aim of this study was to develop the first cross-discipline, vertically integrated, open -access, learning outcomes framework to support EBP curriculum development and delivery across entry-level HCP programmes.

Objectives

  1. 1.

    To articulate student-centred EBP learning outcomes

  2. 2.

    To create an EBP learning outcomes framework.

  3. 3.

    To identify student-friendly learning activities to attain EBP learning outcomes.

Methods

An interdisciplinary (ID) group of academics (8 educators from six HCP programmes and one librarian), from a leading Irish University, involved in the design and delivery of Evidence Based Practice in their respective HCP programmes established the Evidenced-based Education collaborative (EVIBEC) project management team. Ethical approval for the project was obtained from University College Dublin, Human Research Ethics Committee (LS-E-20–166-Redmond). Access and support for the recruitment and involvement of registered students was obtained from the Heads of Schools/Disciplines in each of the participating healthcare programmes.

Co-design approach

A co-design approach was deemed most suitable for the development of this learning outcomes framework. Co-design, within the context of health services research, is the process of bringing together service users, clinical and non-clinical staff, and at times, relevant support, and advocacy groups to work collaboratively to improve or refine elements of the care system, services, or processes [32]. This overall approach lends itself well to application in a healthcare education setting although reports of its use in this context are limited. What is clear from the few studies undertaken is that co-design in health professional education has great potential to provide a structured approach to collaborative working in creating and refining curriculum which is responsive to the needs and interests of students, educators, practitioners, and patients [28, 29, 33].

Co-design team

The co-design team comprised the interdisciplinary EVIBEC project management team and thirty-three undergraduate and postgraduate students, registered to the participating HCP programmes in the University, (see Table 1).

Table 1 Co-design participant demographics

HCP students in the later stages of their programmes, who had undertaken mandatory EBP or research methods training within the last year, were sent an email outlining the study. Students responding to the initial call for expressions of interest were given an information leaflet and consent form for participation in the co-design team.

Supporting meaningful involvement

Open, reciprocal, and democratic dialogue where all participants contribute equally is core to meaningful involvement in co-design [34]. When developing the co-design approach, the development team were mindful of the contextual, environmental, and social enablers to effective and meaningful participation [34, 35]. The team recognised the importance of fostering a positive team atmosphere which is receptive to the contribution of students. The strategies employed to ensure meaningful co-design included the identification of a faculty co-design team member as an ‘involvement champion.’ This champion was a named point of contact for the students if they had any queries or concerns that they wished to address. We also provided pre-workshop information sessions to ensure students felt confident when joining the wider co-design team. At these sessions we described the overall aim of the co-design process, discussed the roles of different co-design team members, reviewed the context and background for EBP education in healthcare professional programmes and introduced all the co-design team members on a first name only basis. Following best practice for co-design [34, 35], we ensured that there was a critical number of students from each of the 7 HCP disciplines recruited (min. 4 students from each discipline). This meant that students outnumbered faculty representation for each discipline and a student was never the sole student voice representing a discipline in any of the breakout groups. The aim of student recruitment and the co-design sessions was the amplification of the student voice relative to the faculty voice as well as balancing the representation across disciplines. Towards this aim, we made an explicit commitment to avoid use of jargon and to explain any technical language as and when it arose in discussions. The collective aim was for a relaxed and informal atmosphere with all team members identified by first name only.

Because COVID-19 public health guidance was in operation at the time of the co-design sessions, all workshops were conducted via Zoom™ (Zoom Inc, CA USA). Students were provided with an ‘icebreaker’ session in advance of the first workshop which ensured access, familiarity and confidence with virtual platforms being used. To accommodate the varying schedules of students across programmes all workshops were conducted in the evening. This facilitated attendance of students who were undertaking clinical practicums with resulting stable participation (bar one student who missed one workshop due to family commitments) throughout the workshops. All students received an honorarium (one-for-all voucher for €30) at the end of the project in recognition of their contribution to the co-design work.

Research design

This study followed the guide and toolkit for experience-based co-design developed by the NHS Institute for Innovation and Improvement [36]. The strategies for meaningful co-design, described above, were embedded into a four-step iterative approach which started with an exploration of educator and student experience (step 1), to understanding these experiences (step 2), to improving the experience (step 3) and finally to assessing the experience (step 4). At the end of each step, data were collated and presented to students prior to the next step. These steps were taken during three online interactive and participatory co-design workshops and one key national stakeholder validation of the emerging framework event. Figure 1 describes each of the four experience-based co-design steps and illustrates the incremental nature of the work where the outcomes from each step provided the starting point for the next step. The online workshops were conducted using Zoom™ web conferencing platform and interactive collaboration was facilitated using Mural™ (Tactivos Inc. Buenos Aires, Argentina), a digital whiteboard and innovative platform for creative collaboration.

Fig. 1
figure 1

The four step co-design approach and target outcomes. Four iterative steps were taken, and a series of target outcomes were proposed for each step of the co-design approach

Step one: capture the experience

Capturing key stakeholder (here educators and students) experiences is identified by the NHS Toolkit as a critical first step in this co-design approach. This guide notes that the approach should start by helping people to tell the story of their experiences.

Step 1a) Educator experience of EBP

The EVIBEC project management team began with a series of brainstorming sessions exploring their experiences of facilitating EBP teaching, followed by a critical review of the literature to identify relevant teaching, learning and research frameworks to inform the project design. Albarqouni et al. [22] set of core EBP competencies for health professionals was deemed the most suitable for this project as these are contemporary and consensus-based and have recently been advocated by the Clinical Effectiveness Unit, Department of Health, Ireland, as relevant for all entry level health professional programmes [37]. Bruner’s spiral curriculum design was identified as appropriate to underpin the draft learning outcomes framework, where knowledge and skills are introduced and applied at increasing levels of complexity [38]. This type of spiral curriculum design supports student-centred outcomes-based education and provides flexibility in implementation. In addition, it was decided that Bloom’s taxonomy would provide the cognitive levels necessary to scaffold the progression of learning outcomes throughout all stages of a programme [39].

Step 1b) Student experience of EBP

The focus of this step was to assist students in telling the story of their own experiences of learning EBP. In the first co-design workshop, students were grouped by discipline in virtual break out rooms. Each room had a member of the project team with experience in facilitating group discussions. The Mural platform contained instructions on each task set, the 5 As and the core competencies. It also displayed a clock, so tasks were completed on time.

The tasks were designed to return students to educational experiences obtained in their previous EBP module (to be situated), facilitating them in processing how they thought it connected with their HCP practice. Task one required students, individually and then in their group, to reflect on what they considered to be the key characteristics of an evidence-based practitioner. This helped them to link back to their experience and facilitated them in discussing and exploring their individual and shared experiences and understandings with colleagues. Task two asked participants to outline the knowledge and skills they considered necessary to become this effective evidence-based practitioner. Finally, the group was asked to reflect on their experiences of EBP education and if, how and why they thought their educational experience had facilitated their acquisition of the knowledge, skills, and attributes that they had identified.

Following the small discipline group discussions, everyone returned to the main virtual room and shared their discipline group’s perspectives with the larger group.

At the end of this workshop all participants had:

  • Developed discipline specific conceptualisation of competencies for EBP

  • Experienced sharing of perspectives and commonalities across disciplines

Step two: Understand the experience

The overall focus of this step was to facilitate co-design participants in integrating their experiences of EBP learning with conceptual frameworks for the five steps of evidence-based practice.

Workshop two aligned with the principles of the world cafe method (https://theworldcafe.com/key-concepts-resources/design-principles/). This method has been used extensively in education to facilitate large group dialogue and has, more recently, been used in co-design health education projects [40]. It is characterised by targeted small group rotating discussions, with each conversation building on the last. Mural proved an ideal platform for virtual hosting this method. Participants in this virtual world café method were randomly assigned to 1 of 5 mixed-discipline groups and initially allocated to an outer ring of one of the 5A steps of EBP of: Ask, Acquire, Appraise, Apply or Assess. They were asked to reflect on, discuss, and using post-its, develop a list of knowledge and skills relevant to the 'A' assigned to that room. They then rotated to the next A and the middle ring to explore and add to the comments of former participants in that outer ring. This sequence was repeated one further time facilitating collaborative dialogue among many students on each task set. The facilitator in each room prompted contribution, listened for patterns and insights, and harvested an overview of the conversations to share with the whole group during the concluding session of the workshop. This overall process ensured the integration of interdisciplinary perspectives and experiences.

Following step 2, the project management team examined and discussed the data obtained. Duplications were removed, data was placed under the correct ‘A’ domain if misplaced, and the content was converted to measurable learning outcomes, framed using an action verb followed by an appropriate object for that verb. The project team also ensured that each learning outcome was appropriately assigned and scaffolded to each cognitive learning level of Bloom's Taxonomy: knowledge, comprehension, application, analysis, synthesis, and evaluation.

Step three: Improve the experience

The step focuses on the facilitation of co-design participants in the creation of ideas for learning activities that support the development of the knowledge and skills identified in step 2.

Following principles from the participatory research method of deliberative dialogue, where the best course of action is determined through discussion [41, 42], in workshop three students were placed in interdisciplinary groups within breakout rooms. In these groups they discussed learning activities that they had previously found useful for learning EBP and proposed additional novel exercises that they thought might aid learning within each of the EBP A domains. They then framed these activities in terms of their position and progression within their programmes (early, middle or late-stages). At the end of the process, students were asked to reach a consensus, identifying one key Interdisciplinary learning activity for each ‘A’ and prepare a ‘sales pitch’ to present to the whole group explaining their choice. Each facilitator`s role was to absorb information, promote reflection, and provide feedback, encouraging prioritising with the intention of identifying and reaching consensus on the intervention components.

Step four: Assess the experience with a ‘critical friend’

In this final step of the co-design process the emergent co-designed framework was presented to an audience of stakeholders and disciplinary experts to allow open discussion and feedback. This workshop was delivered at the Irish Network of Healthcare Educators (INHED) online national conference in Dublin in 2021. Information re the workshop was made available on the conference website, outlining who it would be of interest to, that workshop data would form part of an on-going research project, that ethical permission for the study was obtained from the hosting institution and that informed consent would be gained from all conference participants on registration to the workshop. Participants (n = 15) were also asked to complete a short demographic survey.

During this interactive workshop, participants undertook some of the same exercises that had been performed by students in their workshops. The aim of this was to provide context for the participants and to explore their perceptions of the knowledge, skills and learning strategies necessary to attain the 5 A EBP competencies. In break out rooms participants then explored the developing EVIBEC learning outcomes framework and its related learning activities and were asked to validate or contest its content. All discussions were fed back to the whole group at the end of the session to derive a consensus.

Finally, the participants in this 4th workshop brainstormed immediate and long-term suggestions for implementation and dissemination of the EVIBEC framework under the following headings: clinical practice; module/ course level; programme level—positioning, embedding, acceptability, feasibility, training, evaluation; Institution level—policy and procedures; National level—regulatory bodies, dissemination mechanisms and international level.

Results

Participants

The demographic profile of the student and educator participants is presented in Table 1. A minimum of four students participated from each of the disciplines involved, with some registered to late stages of an undergraduate degree and some registered at postgraduate level. Each discipline had an educator team member participating.

The EVIBEC learning outcomes framework

The EVIBEC Learning Outcomes Framework was produced following the iterative four step co-design approach and the target outcomes were achieved for each step (Fig. 1). The framework is the result of a collation of the work from educators, students, and participants from the INHED conference. Student and conference participant data were collected in the format of virtual sticky notes from their mural boards and summary documents created by each of the room facilitators. Due to time constraints and lack of student familiarity with learning outcome development, student data was not written in the form of learning outcomes. Instead, sticky notes consisted of short terms or single words, indicating the knowledge and skills students perceived essential for an effective evidence-based practitioner. For example, in step 2, short statements such as “know how to perform a search” or “use Boolean operators' ' or “which CASP tool?” were received.

Step 3 outcomes comprised a list of innovative non-traditional learning activities designed to meet the learning outcomes. These have been incorporated into the EVIBEC learning outcomes framework as separate Padlet pages [43]. Overall, students actively favoured contextual and situated learning activities as might be expected of adult learners [44,45,46]. Gamification of learning activities were proposed such as “Who`s who: what source are you” and “navigating virtual cases with graduated information release.” In the discussions students voiced their opinion that inclusion of a gaming component improves motivation. They also added that incorporating conceptual levels of difficulty or “cognitive scaffolds” into activities, similar to performance levels found in gaming, promote learning, and keep learners engaged. Students favoured peer-learning and interactive teaching strategies such as “final years presenting to 1st years” and “group case discussions using online forums”. They were also explicit about beginning to learn EBP early in their programmes with the incorporation of EBP throughout different modules and all stages. However, notably only one group identified teaching strategies that incorporated EBP in clinical placements: “be provided with a tricky clinical dilemma—what's the evidence to guide practice—can be in clinical or classroom setting.”

The final step occurred during the INHED conference. In this workshop participants were clinical HCP educators. They also suggested EBP learning activities, however these tended to focus on learning in clinical practice. They advocated for active learning strategies, problem-based learning, and work-based learning approaches. Suggested activities included: the practical use of audit tools; sharing examples; having a repository of “real” examples and having patients involved. They also agreed that the purposeful teaching of EBP should be positioned throughout healthcare curricula, within practice placements in addition to multiple academic modules. To aid this it was suggested that stronger links be made between academia and clinicians, with a two-way sharing of resources. The project team later reviewed suggested activities from workshops 3 and 4 and assigned them to the appropriate ‘A’ domains of the evolving learning outcomes framework.

Steps 1–4 resulted in the generation of the preliminary EVIBEC Learning Outcomes Framework. The framework can be found on https://www.ucd.ie/chas/research/evibec/. It is structured with the 5 As presented as columns from left to right across the screen and the colour coded cognitive levels of Bloom’s taxonomy as rows. Learning outcomes have been added to each column. Additional screens can be accessed from the main page, with a link at the top of each column providing access to a page for each ‘A’ and one at the bottom of each column providing a link to suggested activities for each A, derived from the workshop participants.

It must be stressed that this framework is evolving as EBP experts engage and contribute to it. The framework is hosted on Padlet (San Francisco, California, U.S.), a cloud-based collaborative platform designed to act like a virtual notice board, allowing real-time interaction and sharing of materials. This approach was selected as it allows what would have been a very large and unwieldy table to become easy to navigate and find the relevant sections. It also offers huge potential for open access sharing of materials and ideas, and it is hoped that it would form the basis to establish a community of practice approach to support validation and refinement of the framework. An iterative approach involving different stakeholders and experts from other disciplines and countries is essential and welcomed and will drive this refinement.

Discussion

Through a rigorous, systematic co-design process, involving an interdisciplinary group of HCP educators and relevant student groups, the EVIBEC framework is presented for the first time. This framework consists of a series of learning outcomes, aligned to EBP competencies, classified according to the 5 As of EBP and mapped to the cognitive levels of Bloom’s taxonomy. The framework is designed to support the integration of teaching and learning related to evidence-based practice throughout all entry level HCP programmes. There is clear value in designing an interdisciplinary syllabus for EBP. Firstly, it has core relevance across health professional programmes with very similar approaches relevant to each and hence significant potential gains from the sharing of resources [8, 19]. EBP competencies transect the healthcare disciplines regardless of the scope of practice and additionally feature heavily in the learning outcomes and pedagogical approaches within individual disciplines, with a common overall goal of effective EBP. Secondly, and of significant importance, as many professionals practise as a team, a trans-curricular approach will equip students with shared EBP language, knowledge, and skills. This will facilitate their future engagement in multi-disciplinary teams, debates, and practice development, all progressing an effective, constructive EBP agenda and future alignment of the professions.

The EVIBEC learning outcomes framework was developed with recognition of the requirement for progression of learning from early to late programme stages, a requirement articulated strongly by both students and educators. A programmatic approach with deliberate progression of learning using scaffolded delivery approaches should enhance student EBP skill development, allowing students to build on learning gained during previous and parallel modules [27]. The explicit detailing of the knowledge, skills and attitudes required for competency in each of the 5 As should enable HCP educators to recognise and appreciate critical elements of EBP and should in turn afford greater consistency and transparency in curricular content. This framework is one that will support both entry level HCP education and continuous professional development for HCPs in clinical practice, especially important where HCPs undertook their professional education at a time where less emphasis may have been placed on EBP. Now hosted on the freely accessible website this framework will support educators in adopting a more systematic and explicit approach to the implementation of EBP learning in HCP programmes. It is also designed as a mechanism to promote the sharing of resources and the development of a community of practice aligned to the common aim of enhanced interprofessional EBP teaching and learning.

The underpinning beliefs of the interdisciplinary project team led to the choice of a co-design approach to this framework. In agreement with Moriña [47] and O`Connor [48] the team were of the belief that students have a valuable contribution to make and represent a valuable source of knowledge and insight for the effective development of teaching and learning resources. Participatory co-design effectively fosters negotiation, participation, and inclusion of the voice of the students [49]. As end users of teaching pedagogies, it is recommended that students should be co-collaborators in the development and evaluation of teaching and learning approaches to ensure an effective user-centred design [50]. In this study students appreciated being invited to contribute to enhancing curriculum and felt valued and empowered by this process. As one student voiced: “workshops were very useful and the dialogue beneficial to give a voice to and feel heard.” The student perspective provided unique insights—students were comfortable outlining the knowledge and skills required for the early EBP steps of Ask, Acquire and Appraise. However, in line with findings from other studies [22, 51], they struggled to express the components of Apply and Assess. They also required facilitator prompting to consider the connection between shared decision making, patient preferences and EBP. This difficulty of expressing progression has been described as the novice-expert divide where experts in a domain have attained and organised a large amount of information and may not appreciate the step-by-step learning needs of novice or advanced beginner students [44]. Co-creating a scaffolded learning outcomes framework with students moving from lower-level learning outcomes before progression to higher level learning was very instructive for both parties. It facilitated the incorporation of important learning outcomes that may have otherwise been overlooked by the project team. It also provided clear insight to educators about areas students find more challenging and in turn how best they can scaffold the curriculum to meet the student needs.

An important element of this study`s co-design process concerns the general features of student suggested learning activities. The students emphasised their preference for a wide range of contextual activities and emphasised that these had to involve active contextual interactions as this met a broader range of student learning preferences. They also suggested activities that would trigger their curiosity and enhance their motivation to learn. This is in line with other studies showing that students value access to different kinds of resources to support, motivate and enhance their learning with an adult learner preference for 'real life' learning [45, 46]. The diversity of the student group and their disciplinary differences enriched these discussions exposing participants, both educator and student alike, to a rich variety of learning experiences and activities. As voiced by one student “the interdisciplinary approach allowed us to get out of our own profession’s ‘bubble’ and think in new /alternative ways". The awareness of ideas and approaches from different disciplines also widens the variety over a single disciplinary approach.

The voice of clinical educators is equally valuable as their experiential knowledge is important in helping academics acknowledge the “multiple realities and meanings” of EBP in action [29, 52]. HCP educators in workshop four provided important insight into how learning outcomes and suggested aligned learning activities might support the learning and application of EBP knowledge in different learning spaces. There was a stark contrast between the focus of HCPs on learning in the clinical context, while students focused on learning in the HEI. Lewin et al. [53] argue collaboration and co-design opens new ways to connect and combine different learning sites in higher education in which both universities and clinical institutions are equal partners in influencing learning and competence development in students. This certainly appeared evident in this study. Clinical educators welcomed being collaborators in the development of the EVIBEC framework, ensuring the inclusion of learning outcomes which cross clinical-HEI boundaries The setting up of mechanisms to facilitate sharing of online resources was seen as a pragmatic example of this partnership as they identified their unique ability to share the practical use of audit tools, to share a repository of “real examples” and databases of evaluations of patient and practice outcomes. They also suggested joint professional development sessions for academics and clinicians to facilitate theory practice transfer and appointments of adjunct professor positions. This type of collaboration fosters seamless learning processes thus enabling students to navigate between different spaces and different roles promoting and developing transferability between learning contexts in HCP education and decreasing the theory–practice gap [54, 55]. It is important that HEIs focus on EBP learning as part of clinical/professional work placement education and future developments and implementation should occur in close collaboration with students, the university, and the clinical setting thus maximising the overall integration of EBP education.

Limitations

The framework was developed within a single university in Ireland, a situation counterbalanced by external stakeholder feedback from the INHED conference and several validation exercises by way of conference presentations, an Irish National Forum seminar, and the seeking of expert opinion from EBP Ireland. Further validation and refinement will be sought from other experts and international colleagues. Participating students in this study were self-selected and might not necessarily represent the full diversity of the university’s student population. The university has a wide range of representative HCP educational programmes mapping to the One Health agenda, but some HCP disciplines were not represented (e.g., Dentistry, Pharmacy, Occupational therapy, etc.) which may limit the study’s generalisability. However, the aim of hosting the EVIBEC framework on the Padlet platform is that it will establish a community of practice and ongoing framework iterations. Further external validation is warranted, and the patient voice could also be sought to further progress this collaborative approach.

Conclusions: brief summary and potential implications

Combining student and clinical educator experiential knowledge with scientific knowledge gained from a literature search and the EVIBEC team’s expert EBP knowledge facilitated the development of a shared understanding of EBP and the development of this consensus framework. The freely accessible EVIBEC framework will support entry level HCP EBP education, offering the potential for interdisciplinary approach development and sharing of valuable resources. The overall goal is the support of evidence-based healthcare with a view to enhancing healthcare efficiency and effectiveness. The co-design process involving HCP educators and students proved highly successful in development of the EVIBEC learning outcome framework.

Availability of data and materials

The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71.

    Article  Google Scholar 

  2. Institute of Medicine. Leadership Commitments to Improve Value in Health Care : Finding Common Ground: Workshop Summary. Washington: D.C. National Academies Press; 2009.

    Google Scholar 

  3. Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLos One. 2014;9(1):e86706–e86706.

    Article  Google Scholar 

  4. Larsen CM, Terkelsen AS. Carlsen A-MF, Kristensen HK: Methods for teaching evidence-based practice: a scoping review. BMC Med Educ. 2019;19(1):259–259.

    Article  Google Scholar 

  5. World Health Organization. European strategic directions for strengthening nursing and midwifery towards Health 2020 goals. Copenhagen: WHO Regional Office for Europe; 2015.

  6. Lehane E, Leahy Warren P, O’Riordan C, Savage E, Drennan J, O ‘Tuathaigh C, O’Connor M, Corrigan M, Burke F, Hayes M, et al editors. Research on teaching of evidence based practice in Ireland - to health care professionals and healthcare students. report prepared for Department of Health, clinical effectiveness unit. Dublin: Department of Health; 2017.

  7. Lehane E, Leahy-Warren P, O’Riordan C, Savage E, Drennan J, O’Tuathaigh C, O’Connor M, Corrigan M, Burke F, Hayes M. Evidence-based practice education for healthcare professions: an expert view. BMJ Evid Based Med. 2019;24(3):103–8.

    Article  Google Scholar 

  8. Albarqouni L, Hoffmann T, Glasziou P. Evidence-based practice educational intervention studies: a systematic review of what is taught and how it is measured. BMC Med Educ. 2018;18(1):1–8.

    Article  Google Scholar 

  9. Skela-Savič B, Gotlib J, Panczyk M, Patelarou AE, Bole U, Ramos-Morcillo AJ, Finotto S, Mecugni D, Jarosova D, Patelarou E, et al. Teaching evidence-based practice (EBP) in nursing curricula in six European countries—A descriptive study. Nurse Educ Today. 2020;94:104561–104561.

    Article  Google Scholar 

  10. Lehane E, Agreli H, O’ Connor S, Hegarty J, Leahy-Warren P, Bennet D, Blake C, Burke F, Corrigan M, Drennan J, et al. Development of a competency framework for clinical effectiveness education. Report prepared for Department of Health, clinical effectiveness unit, national patient safety office, Ireland. Dublin: Department of Health; 2018.

    Google Scholar 

  11. Daly MM, DeAngelis TM. Teaching evidence-based practice across curricula-an overview of a professional development course for occupational therapy educators. Occup Ther Health Care. 2017;31(1):102–9.

    Article  Google Scholar 

  12. Janicke H, Johnson MA, Baillie S, Warman S, Stone D, Paparo S, Debnath NC. Creating the next generation of evidence-based veterinary practitioners and researchers: what are the options for globally diverse veterinary curricula? J Vet Med Educ. 2020;47(5):647–58.

    Article  Google Scholar 

  13. Stichler JF, Fields W, Kim SC, Brown CE. Faculty knowledge, attitudes, and perceived barriers to teaching evidence-based nursing. J Prof Nurs. 2011;27(2):92–100.

    Article  Google Scholar 

  14. Melnyk BM. The current research to evidence-based practice time gap is now 15 instead of 17 years: urgent action is needed. Worldviews Evid Based Nurs. 2021;18(6):318–9.

    Article  Google Scholar 

  15. Melnyk BM, Gallagher-Ford L, Zellefrow C, Tucker S, Thomas B, Sinnott LT, Tan A. The First U.S. Study on nurses’ evidence-based practice competencies indicates major deficits that threaten healthcare quality, safety, and patient outcomes: U.S. Study on nurses’ evidence-based practice competencies. Worldviews Evid Based Nurs. 2018;15(1):16–25.

    Article  Google Scholar 

  16. Saunders H, Vehviläinen-Julkunen K. The state of readiness for evidence-based practice among nurses: an integrative review. Int J Nurs Stud. 2016;56:126.

    Article  Google Scholar 

  17. Patelarou E, Vlasiadis K, Panczyk M, Dolezel J, Jarosova D, Gotlib J, Skela-Savič B, Mecugni D, Finotto S, Ruzafa-Martinez M, et al. Providing a teaching and learning open and innovative toolkit for evidence-based practice to nursing European curriculum (EBP e-Toolkit): Project rationale and design. Popul Med. 2020;2(November):1–7.

    Article  Google Scholar 

  18. Ruzafa-Martínez M. EBP e-toolkit project: Providing a teaching and learning open and innovative toolkit for evidence-based practice to nursing european curriculum. Cent Eur J Nurs Midwifery. 2019;10(2):1017–8.

    Article  Google Scholar 

  19. Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, Porzsolt F, Burls A, Osborne J. Second international conference of evidence-based health care T et al: Sicily statement on evidence-based practice. BMC Med Educ. 2005;5(1):1–1.

    Article  Google Scholar 

  20. Melnyk BM, Gallagher-Ford L, Long LE, Fineout-Overholt E. The establishment of evidence-based practice competencies for practicing registered nurses and advanced practice nurses in real-world clinical settings: proficiencies to improve healthcare quality, reliability, patient outcomes, and costs. Worldviews Evid Based Nurs. 2014;11(1):5–15.

    Article  Google Scholar 

  21. Hitch D, Nicola-Richmond K. Instructional practices for evidence-based practice with pre-registration allied health students: a review of recent research and developments. Adv Health Sci Educ. 2017;22:1031–45.

    Article  Google Scholar 

  22. Albarqouni L, Hoffmann T, Straus S, Olsen NR, Young T, Ilic D, Shaneyfelt T, Haynes RB, Guyatt G, Glasziou P. Core competencies in evidence-based practice for health professionals consensus statement based on a systematic review and delphi survey. JAMA Netw Open. 2018;1:e180281–e180281.

    Article  Google Scholar 

  23. The European higher education area. Joint Declaration of the European Ministers of Education [https://www.ehea.info/media.ehea.info/file/Ministerial_conferences/02/8/1999_Bologna_Declaration_English_553028.pdf]

  24. Pérez-Montoro M, Tammaro AM. Outcomes of the Bologna process in LIS higher education: comparing two programs in Europe. Int Inf Libr Rev. 2012;44(4):233–42.

    Article  Google Scholar 

  25. The European Parliament and The Council of The European Union. Directive 2005/36/EC of the European Parliament and of the Council of 7 September 2005 on the recognition of professional qualifications. In: Official Journal vol. L255. 2005. p. 22–142.

  26. Allan J. Learning outcomes in higher education. Stud High Educ. 1996;21(1):93–108.

    Article  Google Scholar 

  27. Biggs J. Enhancing teaching through constructive alignment. High Educ. 1996;32(3):347–64.

    Article  Google Scholar 

  28. Theobald KA, Coyer FM, Henderson AJ, Fox R, Thomson BF, McCarthy AL. Developing a postgraduate professional education framework for emergency nursing: a co-design approach. BMC Nurs. 2021;20(1):43.

    Article  Google Scholar 

  29. Hardie P, Murray A, Jarvis S, Redmond C. Experienced based co design: nursing preceptorship educational programme. Res Involv Engagem. 2022;8(1):1–53.

    Article  Google Scholar 

  30. Adams S. Using learning outcomes: A consideration of the nature, role, application and implications for European education of employing ‘learning outcomes’ at the local, national and international levels. In. Edinburgh: Heriot-Watt University; 2004.

    Google Scholar 

  31. Thistlethwaite JE, Forman D, Matthews LR, Rogers GD, Steketee C, Yassine T. Competencies and frameworks in interprofessional education: a comparative analysis. Acad Med. 2014;89(6):869–75.

    Article  Google Scholar 

  32. Slattery P, Saeri AK, Bragge P. Research co-design in health: a rapid overview of reviews. Health Res Pol Syst. 2020;18(1):17.

    Article  Google Scholar 

  33. Dederichs M, Nitsch FJ, Apolinário-Hagen J. Piloting an innovative concept of e–mental health and mhealth workshops with medical students using a participatory co-design approach and app prototyping: case study. JMIR Med Educ. 2022;8(1): e32017.

    Article  Google Scholar 

  34. O’Donnell D, Ní Shé É, McCarthy M, Thornton S, Doran T, Smith F, O’Brien B, Milton J, Savin B, Donnellan A, et al. Enabling public, patient and practitioner involvement in co-designing frailty pathways in the acute care setting. BMC Health Serv Res. 2019;19(1):797.

    Article  Google Scholar 

  35. Black A, Strain K, Wallsworth C, Charlton S-G, Chang W, McNamee K, Hamilton C. What constitutes meaningful engagement for patients and families as partners on research teams? J Health Serv Res Policy. 2018;23(3):158–67.

    Article  Google Scholar 

  36. Institute for Innovation and Improvement: Experience Based Design. Guide & Tools. Leeds: NHS; 2009.

    Google Scholar 

  37. Lehane E, Agreli H, O’Connor S, Hegarty J, Leahy Warren P, Bennett D, Blake C, Burke F, Corrigan M, Drennan J, et al. Building capacity getting evidence-based practice into healthcare professional curricula. BMJ Evid Based Med. 2020;26(5):246–246.

    Article  Google Scholar 

  38. Harden RM. What is a spiral curriculum? Med Teach. 1999;21(2):141–3.

    Article  Google Scholar 

  39. Kennedy D. Writing and using learning outcomes: a practical guide. Cork: University College Cork; 2006.

    Google Scholar 

  40. Kuluski K, Ho JW, Cadel L, Shearkhani S, Levy C, Marcinow M, Peckham A, Sandercock J, Willison DJ, Guilcher SJT. An alternate level of care plan: Co-designing components of an intervention with patients, caregivers and providers to address delayed hospital discharge challenges. Health Expect. 2020;23(5):1155–65.

    Article  Google Scholar 

  41. Boyko JA, Kothari A, Wathen CN. Moving knowledge about family violence into public health policy and practice: a mixed method study of a deliberative dialogue. Health Res Pol Syst. 2016;14(1):31–31.

    Article  Google Scholar 

  42. Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N. Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Soc Sci Med. 2012;75(11):1938–45.

    Article  Google Scholar 

  43. The Evidenced-Based Education Collaborative (EVIBEC) Learning Outcomes Framework [https://www.ucd.ie/chas/research/evibec/]

  44. Persky AM, Robinson JD. Moving from novice to expertise and its implications for instruction. Am J Pharm Educ. 2017;81(9):6065.

    Article  Google Scholar 

  45. Lall P, Rees R, Law GCY, Dunleavy G, Cotič Ž, Car J. Influences on the implementation of mobile learning for medical and nursing education: qualitative systematic review by the digital health education collaboration. J Med Internet Res. 2019;21(2): e12895.

    Article  Google Scholar 

  46. Koivisto J-M, Haavisto E, Niemi H, Katajisto J, Multisilta J. Elements Explaining Learning Clinical Reasoning Using Simulation Games. Int J Serious Games. 2016;3(4):29–43.

    Article  Google Scholar 

  47. Moriña A. Approaches to inclusive pedagogy: A systematic literature review. Pedagog. 2021;140(4):134–54.

    Article  Google Scholar 

  48. O’Connor S, Zhang M, Trout KK, Snibsoer AK. Co-production in nursing and midwifery education: a systematic review of the literature. Nurse Educ Today. 2021;102:104900.

    Article  Google Scholar 

  49. Carballo R, Cotán A, Spinola-Elias Y. An inclusive pedagogy in arts and humanities university classrooms: what faculty members do. Arts Humanit High Ed. 2021;20(1):21–41.

    Article  Google Scholar 

  50. Ösp Egilsdottir H, Heyn LG, Brembo EA, Byermoen KR, Moen A, Eide H. Configuration of mobile learning tools to support basic physical assessment in nursing education: longitudinal participatory design approach. JMIR Mhealth Uhealth. 2021;9(1):e22633–e22633.

    Article  Google Scholar 

  51. Hoffmann TC, Montori VM, Del Mar C. The connection between evidence-based medicine and shared decision making. JAMA. 2014;312(13):1295–6.

    Article  Google Scholar 

  52. Green T, Bonner A, Teleni L, Bradford N, Purtell L, Douglas C, Yates P, MacAndrew M, Dao HY, Chan RJ. Use and reporting of experience-based codesign studies in the healthcare setting: a systematic review. BMJ Qual Saf. 2020;29(1):64–76.

    Article  Google Scholar 

  53. Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright M, Bohren MA, Tunçalp Ö, Colvin CJ, Garside R, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implement Sci. 2018;13(suppl 1 2):1–10.

    Google Scholar 

  54. Verkuyl M, Djafarova N, Mastrilli P, Atack L. Virtual gaming simulation: evaluating players’ experiences. Clin Simul Nurs. 2022;63:16–22.

    Article  Google Scholar 

  55. Zieber M, Wojtowicz B. To dwell within: bridging the theory–practice gap. Nurs Philos. 2020;21(2):12296-n/a.

    Article  Google Scholar 

Download references

Funding

This research was funded by University College Dublin Learning Enhancement Project, jointly supported by the HEA and the National Forum for the Enhancement of Teaching and Learning, Ireland.

Author information

Authors and Affiliations

Authors

Contributions

All authors made substantial contributions to the conception and design of this work and contributed to the acquisition and interpretation of data. CR, RF, EON, DOD, DS, AD, SF and CC wrote the main manuscript text and SOR prepared Fig. 1 and Table 1. EON prepared the Padlet pages and the website. CR revised the manuscript and all authors read and approved the final manuscript.

Corresponding author

Correspondence to Deirdre O’Donnell.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for the project was obtained from University College Dublin, Human Research Ethics Committee (LS-E-20–166-Redmond). Informed consent was obtained from all subjects.

Consent for publication

Consent to publish data in an online, open-access journal was obtained from the study participants. All raw data has been de-identified.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Redmond, C., Farrell, R., Cunningham, C. et al. Development of the EVIBEC Learning Outcomes Framework to support the delivery of evidence-based practice curricula in health care professional programmes: a codesign approach. BMC Med Educ 24, 3 (2024). https://doi.org/10.1186/s12909-023-04972-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12909-023-04972-0

Keywords