Changes in physiotherapy students’ knowledge and perceptions of EBP from first year to graduation: a mixed methods study
BMC Medical Education volume 18, Article number: 109 (2018)
Dedicated Evidence-Based Practice (EBP) courses are often included in health professional education programs. It is important to understand the effectiveness of this training. This study investigated EBP outcomes in entry-level physiotherapy students from baseline to completion of all EBP training (graduation).
Mixed methods with an explanatory sequential design. Physiotherapy students completed two psychometrically–tested health professional EBP instruments at baseline and graduation. The Evidence-Based Practice Profile questionnaire collected self-reported data (Terminology, Confidence, Practice, Relevance, Sympathy), and the Knowledge of Research Evidence Competencies instrument collected objective data (Actual Knowledge). Focus groups with students were conducted at graduation to gain a deeper understanding of the factors impacting changes in students’ EBP knowledge, attitudes, behaviour and competency. Descriptive statistics, paired t-tests, 95% CI and effect sizes (ES) were used to examine changes in outcome scores from baseline to graduation. Transcribed focus group data were analysed following a qualitative descriptive approach with thematic analysis. A second stage of merged data analysis for mixed methods studies was undertaken using side-by-side comparisons to explore quantitatively assessed EBP measures with participants’ personal perceptions.
Data were analysed from 56 participants who completed both instruments at baseline and graduation, and from 21 focus group participants. Large ES were reported across most outcomes: Relevance (ES 2.29, p ≤ 0.001), Practice (1.8, p ≤ 0.001), Confidence (1.67, p ≤ 0.001), Terminology (3.13, p ≤ 0.001) and Actual Knowledge (4.3, p ≤ 0.001). A medium ES was found for Sympathy (0.49, p = 0.008). Qualitative and quantitative findings mostly aligned but for statistical terminology, participants’ self-reported understanding was disparate with focus group reported experiences. Qualitative findings highlighted the importance of providing relevant context and positive role models for students during EBP training.
Following EBP training across an entry-level physiotherapy program, there were qualitative and significant quantitative changes in participants’ knowledge and perceptions of EBP. The qualitative and quantitative findings were mainly well-aligned with the exception of the Terminology domain, where the qualitative findings did not support the strength of the effect reported quantitatively. The findings of this study have implications for the timing and content of EBP curricula in entry-level health professional programs.
Evidence based practice (EBP) where patient values, clinical findings and best research evidence are integrated in patient decision-making, is now widely accepted as an essential component both in clinical practice and education of entry-level health professionals [1, 2]. The principles of EBP dovetail well with the pedagogical principles applied by educational institutions in health professional education. These include: creating an academic culture that values and understands the relevance of EBP, providing sound training in client and inter-professional communication skills, profession-specific technical skills, and skills in the five steps of EBP (ask, access, appraise, apply, assess) . The challenge has been in designing, teaching and evaluating EBP courses to meet the expectations of accreditation bodies of health professional programs.
The effectiveness of EBP training has been investigated largely in the medical profession as shown in systematic reviews by Coomarasamy and Khan  and Ilic and Maloney . A further systematic review by Wong et al.  on changes in EBP outcomes after entry-level EBP training, encompassed all entry-level health professions, but found 22 of the 27 included studies were conducted in medicine. Most quantitative studies investigating the effectiveness of EBP training outside medicine have been pre-post in design with short term follow-up after one or two courses, and included students [6, 7] or clinicians . Very few studies have investigated EBP outcomes across a wider time frame in the non-medical health professions; the exceptions were McEvoy et al.  who investigated quantitative changes in EBP outcomes in physiotherapy students from graduation to one or 2 years in the workforce and Lewis et al.  who evaluated changes in entry-level health professional students’ EBP knowledge, attitudes and behaviours after two sequential EBP courses spanning up to 2 years.
A solely qualitative study on student perceptions of EBP was undertaken by Ilic and Forbes  but in recent years there has been an increase in the use of mixed methods studies, integrating both qualitative and quantitative findings. These have included studies in medical students , physiotherapy clinicians  and concurrent samples of first, second and third year physiotherapy students . However, no previous study has presented quantitative findings of the same students before and after completion of EBP courses undertaken over the duration of a health professional program i.e. matched data comparing first year to graduation, and integrated this with qualitative findings; this is the niche of the current study.
The aim of this study was to investigate and integrate qualitative perceptions, and quantitative changes in physiotherapy student EBP outcomes (self-reported knowledge, attitudes and behaviours, and actual knowledge), from ‘baseline’ (prior to any EBP training) to ‘graduation’ (after completion of all EBP training).
This mixed methods study used an explanatory sequential design where a quantitative sub-study was followed by a qualitative sub-study. Qualitative data were sought to gain a deeper and richer understanding of the factors impacting changes in physiotherapy student knowledge, attitudes, behaviour and competency with EBP. Quantitative and qualitative findings were then merged to satisfy the aims of the study . Ethical approval for the study was gained from the Human Research Ethics Committee of the University of South Australia (protocol numbers 0000021077 for the questionnaire data and 0000030567 for the focus group data). Data were collected in 2010 and 2013.
Participants were Bachelor of Physiotherapy (+/− Honours) entry–level students enrolled in 2010 and graduating in 2014 from the University of South Australia. In the standard Bachelor of Physiotherapy program all students completed three mandatory EBP courses in the 1st, 2nd and 4th years of the program. For students undertaking the Bachelor of Physiotherapy program with Honours, the final EBP course was replaced by a ‘Health Sciences Honours Thesis’ course. The EBP curriculum across the three courses is presented in Table 1. A single cohort was invited to participate; as the study investigated changes from baseline (where students had no known EBP training) to graduation (which was the completion of the EBP training), participants were included only if they provided matched data for the two test occasions and were entitled to graduate in 2014.
Data collection questionnaires
Data were collected using two questionnaires, both administered to participants at baseline and at graduation (two test occasions). The Evidence-Based Practice Profile (EBP2) questionnaire measured self-reported knowledge, attitudes and behaviours  and the Knowledge of Research Evidence Competencies (K-REC) instrument measured participants’ actual EBP knowledge . The EBP2 questionnaire includes 58 five-point Likert scale items that address five EBP domains [Relevance (14 items): the value, emphasis and importance placed on EBP; Terminology (17 items): understanding of common research terms; Confidence (11 items): perception of ability with EBP skills; Practice (9 items): the use of EBP in clinical situations; Sympathy (7 items): perception of the compatibility of EBP with professional work]. The K-REC instrument evaluates Actual Knowledge of EBP using nine items (short answers to clinical scenarios, multiple choice and true/false questions, maximum score of 12). Both instruments were developed specifically for the health professional disciplines, are relatively quick for participants to complete and have demonstrated psychometric properties [15, 16]. The details of both instruments, including the domains, items, and detailed psychometric properties are presented in Table 2.
All graduating students who had completed the final EBP questionnaires and were eligible for graduation were invited by email to participate in semi-structured focus groups. Students who responded were provided with additional information, and written informed consent was obtained. The focus groups were conducted on the university campus by an experienced facilitator who had not been involved in participants’ EBP teaching (JL). A semi-structured question guide used by the facilitator ensured all areas were covered in each group (Additional file 1) but did not limit the flow of relevant information that arose during group discussions. The research team had developed the question guide to cover areas relevant to the study aims including gaining the perspective of final year students on EBP training across and at completion of the physiotherapy program, and perceptions of future use of EBP in the first year in the workplace. Focus group discussions explored students’ perceptions of EBP training (across the three courses), including knowledge and skill development, value and relevance of EBP, EBP behaviours, current use of EBP and role models during the program. Sessions were digitally audio-recorded. An AUD25 honorarium was paid to each focus group participant.
Data management and analysis
De-identified data were entered into Predictive Analytic Software (PASW) Statistics 17.0 (Chicago, IL). Participants were only included if matched data were available for the two test occasions. For the EBP2 questionnaire, only domain data where participants had completed 100% of domain items, for that particular domain, on both the first and last occasions of testing were included for analysis. For the K-REC, participants were required to have completed at least 70% of items on both test occasions for inclusion in the analysis. A less stringent inclusion criteria was applied for the K-REC as participants with no knowledge of EBP at baseline were considered more likely to omit items. For the EBP2 questionnaire the Likert scores were treated as interval data; maximum domain scores varied (Relevance 70, Terminology 85, Confidence 55, Practice 45, Sympathy 35), due to the different number of items per domain. The K-REC instrument for Actual Knowledge was scored according to set scoring guidelines, with a maximum score of 12. Descriptive statistics were calculated for each of the five EBP2 domain scores, K-REC Actual Knowledge domain total score and demographic information for the test occasions. Paired t-tests, 95% CI and effect sizes were used to examine the changes in domain scores between the two test occasions. Alpha levels of less than 0.05 were considered statistically significant. Effect sizes (ES) were classified as small (≤ 0.20), medium (0.50) and large (0.80) .
Digitally audio recorded focus group sessions were transcribed verbatim. Transcribed data were de-identified and entered onto NVivo 10 software (QRS International Pty Ltd) for coding and data management. Data analysis followed a qualitative descriptive approach with thematic analysis as described by Stanley . Information relevant to the study aims was inductively coded and thematic development was conducted in stages. Each analytical stage involved independent consideration, discussions and consensus by two researchers (LB, JL), who were naïve to the quantitative results. Initially, codes were inductively derived from these data in an iterative process of attributing codes to small sections of meaning, moving back and forward across focus groups and constantly comparing data and codes. Codes were then grouped into logical and meaningful clusters in a hierarchical tree structure, forming descriptive themes and sub-themes. These themes were described, along with illustrative quotes from the focus groups.
A second stage of merged data analysis for mixed methods studies was undertaken using side-by-side comparisons as described by Creswell and Plano Clark . This explored the alignment of quantitatively assessed measures of participants’ EBP development with their personal perceptions. Using this technique, the tabulating of quantitative alongside qualitative results became the means for merging these data. It enabled the comparison and communication of congruent and divergent interpretations, and facilitated the drawing of conclusions.
There were 127 physiotherapy students enrolled in the first EBP course and 96 (77.2%) completed the initial questionnaire; there were 125 students enrolled in the third EBP or Honours course and 109 (87.2%) completed the final questionnaire. Overall, there were 56 physiotherapy students who provided matched data by completing both the initial and final questionnaires. Baseline characteristics for the 56 participants: gender 41F/15M, age [mean (SD) range in years] 19.1 (3.6), 17–44 years, English was first language for n = 45, not first language for n = 8 and not reported for n = 3. The baseline characteristics of the students who completed the final questionnaires but were not included in the matched dataset were: gender 40F/29M, age [mean (SD) range in years] 19.8 (4.5) 17–46 years. There were 21 participants involved across four focus groups conducted in November 2013 (n = 6, 5, 6 and 4 participants in the respective groups). Baseline characteristics for the 21 focus group participants: gender 16F/5M, age [mean (SD) range in years] 18.9 (1.9), 17–25 years, English was first language for n = 16, not first language for n = 5. No time limit was imposed but focus groups lasted an average of 46 min (range 43–51 min).
While there were 56 participants with matched data from baseline to ‘graduation’, there was not complete matched data from all 56 participants for each of the six domains. The number of matched sets of data for the individual domains were: Relevance, Confidence and Sympathy n = 55 each, Terminology n = 53, Practice n = 51. For the K-REC instrument (Actual Knowledge) 50 participants completed at least 70% of the items on both occasions. Changes in scores for each domain (mean, 95% CI, p-value and ES) from baseline to ‘graduation’ are presented in Table 3.
Maximum domain scores varied (Relevance 70, Terminology 85, Confidence 55, Practice 45, Sympathy 35, Actual Knowledge 12) due to the differences in the number of items in each domain. Figure 1 presents the domain scores as a percentage of the possible maximum (100%) score from baseline to ‘graduation’.
Four major descriptive themes with sub-themes were found that reflected the experiences and perceptions of students over the years of EBP education (see Table 4). These were: 1) a shift in thinking over time; 2) the need for relevance and context; 3) learning by doing; and 4) getting the timing right. Participant quotes that support these findings are presented in Additional file 2.
1. Shift in thinking over time
Participants described a substantial shift in their understanding and perceptions of EBP over the 4 years of their program. They were initially almost completely naïve about the relevance of EBP to physiotherapy. Participants described the first 2 years of EBP education as difficult, uninteresting and irrelevant. In the second half of their program, students became increasingly aware that EBP played a major role in many other courses within their program, and also that ‘…it is a big thing’ in their future physiotherapy profession. By the time of graduation, their perceptions of EBP had become very positive. “I think when it’s your third and fourth year you realise ‘wow this is really important and it’s interesting and it guides our practice and it’s really helpful.’”
At the commencement of their program, the role of research in clinical practice was not appreciated by students. Some students described their expectations of physiotherapy as a static practice, with established treatment practices and protocols. Research as a component of the program came as a shock to many students. “I thought it was something separate, so physios actually do the work … it’s not really aligned with research.”
Students reported a change in perception and confidence by graduation and that they possessed the knowledge and skills they would need in the workplace. “I’m pretty confident that I can now search for it well, that I can go through all the databases and know what I have to use for search terms and that I can do a fairly good appraisal, and that I can apply it to a patient.”
Participants varied in their understanding of some EBP terminology, particularly statistical terms used in manuscripts, however this did not appear to negatively affect their confidence. “…some of the statistical terminology that you look at and go… I don’t know what that means. But I do feel equipped to find that information myself …”.
“…sometimes reading articles, you know like the numbers in brackets and some parts that I still don’t quite get… but I think basically I’ve got the gist of it; I can do it.”
2. Need for relevance and context
In the first 2 years of the program participants wanted more context to understand the role and relevance of EBP, both within the entire program, and within the profession they were entering. “Like at the beginning it just feels like a big burden, what’s the point and why are we doing this?”
In first and second year, physiotherapy students studied EBP with students from other health professional programs. Although they appreciated meeting students from other disciplines, they were disappointed when general examples were provided in lectures and assignments which lacked physiotherapy context. Later in the program, participants grew to appreciate the importance of EBP for their work, but they continued to want greater clinical context and relevance to physiotherapy. Examples were given of a systematic review assignment undertaken during clinical placements. Students were critical of the assignment’s strong emphasis on procedural writing and the poor relationship to the needs of students’ clinical placement patients. “It was all about how you wrote it and what you included in your writing”. “It would’ve been a lot more helpful for me to be doing more … writing about my patients and researching things for them which I would do in the clinical world…”.
Early in the program, an inability to see the relevance of EBP to their future work impacted on students’ learning strategies. Students described rote-learning for first and second year examinations, with minimal carry over to long-term knowledge retention. Carry over learning into third year was described as a vague familiarity with some words and concepts. “We rote learnt it... then out of my head again, … I was just like I just want to get this so I get a good mark then leave.”
Participants expressed their learning preferences for EBP, and considered the on-line lectures delivered in first year to be less effective than face-to-face lectures and tutorials. “I remember everyone was complaining about it, because as first years we couldn’t do our own learning. We didn’t know how... I think that’s why when we got to second year, we actually didn’t know anything.”
The repetition of content between first year and second year EBP courses was regarded negatively by several participants. Conversely, some viewed repetition more positively and admitted it provided an opportunity to ‘catch up’ and consolidate their learning.
Later in the program, students worked in groups to conduct a systematic review. Many students found the group assignment disadvantageous, as tasks for the review were shared out amongst the group and some felt they missed out on practising important skills.“ I suppose I don’t feel as confident as probably other people do, because I did miss out on implementing the search strategy actually in the databases and learning properly how to do it.”
3. Learning by doing
The third year of EBP education was a turning point for participants.“EBP3 has been the best…We’ve learned the most and how to actually apply it.”
At this time students learned how to conduct database searches, and how to appraise and select the best evidence to answer their clinical questions. Students reported that these processes were easier to understand when a demonstration lecture was followed by a practical session.
Participants also associated the greater understanding and appreciation of EBP with the commencement of clinical placements in third year, where they could apply EBP principles practically to their own patients. “…once you start going on placement and your supervisors ask you ‘what kind of evidence is behind this treatment?’ you see the link.”
Some students distinguished between third and fourth year, highlighting their final year as the point where they commenced customising the research evidence specifically to their individual patients. “… this year [4th] it’s very specific to your patients and making sure you look at the patient perspective and say, this is the evidence, but is it relevant to them?”
Participants reported another effective way of ‘learning by doing’ was through conducting their own systematic reviews, either as group projects or for an individual patient’s needs. “It was helpful in the sense that I have a way better understanding of how to search and also how to critically appraise things. So when I get an article myself, I can look at it and say, ‘no, this is bad methodology’.”
Having role models and reinforcement helped students consolidate their understanding of EBP. Students recalled university staff referring to evidence in lectures and tutorials of non-EBP courses, which reinforced its importance. “I think they definitely push it, it is a big thing.”
Clinical placements provided important reinforcement of EBP principles, and students provided many examples of personally using their new EBP skills during placements. These placements also afforded some positive role models for the use of EBP in the workplace. “Last placement, my tutor often referred to “new evidence in this” to his patients …would explain it to them”. Students witnessed research evidence being circulated in clinical staff development sessions, and also evidence provided to patients by clinical tutors.
Although participants sometimes witnessed negative EBP role models during clinical placements, such as staff who continued to use disproven interventions, they tended to be disparaging of this non-EBP approach and saw it as outdated.
4. Getting the timing right
There were certain optimal times when participants wanted to receive particular EBP training or information. The current timing of the EBP education had not always met their needs. For example, many believed they would have benefitted from a larger view of the relevance and importance of EBP at the outset of their physiotherapy program. “I didn’t realise …I was just like, ‘EBP just get through it’ … so maybe if I knew it was going to be such a big part, maybe I would’ve taken it bit more seriously”.
Advanced evidence searching and management skills were needed early in the program. These had been taught in the final year, and were then reinforced by practical library tutorials and were seen as highly valuable by the students. Several students said that they would have appreciated learning these searching skills earlier in the program.
“Just one library session and it changed everything for us. It was like all those times I spent hours researching for an essay, still only got a credit because my resources weren’t good, and if they put that earlier on I think it would help....”
“…teaching how to use Endnote properly as well, ‘cause that was particularly good…”
The skills learned in the final year (i.e. refining database searching, critical appraisal) were considered to be very important for optimising clinical placement experiences. Students who attended clinical placements prior to learning these skills felt disadvantaged by this poor timing. Further to this, some participants felt that the perceptions of irrelevance experienced during their first 2 years of EBP education would have been lessened if it had followed a clinical placement experience “…would definitely appreciate it more in second year after a placement.”
Merged data results
The findings from this analytical work are summarised in Table 5. The side-by-side comparison table compared the quantitative questionnaire data with focus group data on eight major findings. These were drawn from the key findings of the quantitative and qualitative research, thus summarising evidence from each sub-study.
The quantitative and qualitative findings were congruent regarding participants’ perceptions of improvements in several domains of EBP learning over the 4 year program for several EBP domains (Relevance, Confidence, Practice, Sympathy).
The quantitative and qualitative components of this research provided different perspectives on only one aspect of EBP learning. While the questionnaires demonstrated a large increase in participants’ understanding of research terminology (ES 3.13), during the focus groups some participants declared poor understanding of statistical terminology.
There was a significant increase in Actual EBP knowledge measured by the K-REC questionnaire over the program (p = 0.001; ES 4.3). Data on Actual Knowledge were not collected qualitatively, however the increase is reflected in qualitative findings of perceived increased understanding, appreciation and confidence in EBP.
The focus groups were able to uncover some unique information that was unavailable in the quantitative questionnaires. Participants declared strongly that their EBP learning experience was dependent on the how they perceived relevance and context of the topic. Early in their program, participants could not see how the EBP information could be relevant to their future physiotherapy practice, and found the courses arduous. An important positive change in the perception and understanding of EBP followed the commencement of clinical placements in third year. Valuable information for educators was provided on getting the timing right i.e. the importance of developing EBP skills and knowledge at the optimal time in their 4 year program.
This study found that there were significant changes with mostly large effect sizes for the quantitatively assessed domains of Relevance, Terminology, Practice, Confidence, Sympathy and Actual Knowledge in physiotherapy students from first year to graduation. These findings mostly coincided with the qualitative findings but were disparate for ‘statistical terminology’ where participant perceptions of large changes in their self-reported understanding of terms was not matched by focus group-reported experiences. Additional themes that emerged in the qualitative data were the need for relevance and context, and the importance of getting the timing right for EBP training in entry-level health professional education.
A mixed methods pre-post study was undertaken over a period of 7 months by Bozzolan et al.  in three concurrent courses in three different groups of 1st (n = 26), 2nd (n = 28), and 3rd (n = 19) year physiotherapy students. Scores for EBP knowledge, attitudes, skills, practice and competency were substantially different before the first course and after the final EBP course. This supports findings in the current study where there were significant changes in matched student data after three courses. However, interestingly in Bozzolan et al.  the scores changed significantly within the 1st and 2nd year students but not within the 3rd year students. As reported in the current study, the qualitative analysis in Bozzolan et al.  revealed that students valued EBP but recognised the barriers and observed variations in the support and role-modelling provided by clinicians. Further support for the current study findings were found by McEvoy et al. ; significant differences in self-reported EBP domains (Relevance, Terminology, Confidence, Sympathy and Practice) were reported in a cross-sectional study of 914 health professional students across five disciplines, based on exposure to EBP training (p < 0.003) and stage of training (year level of professional program) (p < 0.001).
More recently, Ilic and co-authors have conducted a series of quantitative and qualitative studies researching medical students’ perceptions of the importance of EBP  and approaches to teaching beyond a didactic presentation style [11, 20]. The qualitative findings suggested that medical students recognise the importance of Evidence-Based Medicine principles and application in their training and future clinical practice .
As indicated in the curriculum (Table 1) the current study used a multifaceted approach with a combination of face-to-face lectures, workshops, practical training, hand-on experiences, on-line library skills training progressing to more sophisticated searching, and individual to more independent group work. The assessments were similarly scaffolded from written quizzes and exams to tutorial presentations toward undertaking a systematic review as a group under the guidance of a facilitator. An overview of systematic reviews into teaching of EBP supports undergraduate training strategies that focus on multifaceted, clinically integrated approaches .
While the formal EBP curriculum delivered over the three EBP course in the current study, were multifaceted, the included topics may be considered as ‘classic’ EBP within the taxonomy developed by Shaughnessy et al. . Further levels of training involving ‘information mastery’ where EBP principles are incorporated into patient care with reflection on clinical outcomes and integration with patient values, will be developed with further clinical experience. However, in the qualitative component of the current study participants alluded to experiencing some initial progression toward these later stages presented in the taxonomy, in final year clinical placements, which is promising.
The lack of congruency between data gained through questionnaires and focus groups about statistical terminology knowledge may initially appear surprising; that students acknowledged, or felt ‘safe’ to reveal their inadequacies with statistical terminology in the focus groups amongst peers, adds validity to this method of data collection for EBP knowledge, attitudes and behaviours. However, it was not surprising that the lack of knowledge seemed to be more about the application of the terminology in reading and interpreting the results sections of research studies, which was the element teased out in the focus groups. Interestingly, while aware of a lack of skill in more immediate application, some students commented on their ability to know where to look for the information.
Students need context for EBP and this may not be apparent in the early years of the program where many courses focus on the foundational sciences (anatomy, physiology, biomechanics). With progression to courses involving clinical application, the appreciation of, and commitment to EBP grows. All possible opportunities, particularly early in the program to provide realistic examples of EBP in practice within the profession, and in practical activities that support the theory of EBP, are perceived as important to students. Associated with this is determining the optimal timing of formal EBP training in three to 4 year physiotherapy (and other) programs, to achieve the most favourable outcomes for EBP domains. While students value early exposure to EBP, they are unsure about whether this should commence in the format of formal EBP courses, as early as the first year of University study. The students indicated that they struggled to understand the relevance and context for EBP in the early years of training. It is possible that 1st and 2nd year students find it difficult to comprehend how EBP principles might be integrated into practice, when they have had limited exposure to clinical placements, and an evidence-based approach to clinical decision making in the ‘real world’.
Despite this difficulty with EBP training in the early years, it is clear that reinforcement of EBP skills at many levels is needed for students. For example, repetition of database searching with a demonstration of the skills, followed by practical application, are perceived to enhance abilities. Role-modelling by lecturers and clinical educators, who introduce the language and examples of EBP is observed by students, as is the lack of this in other teachers. This modelling may occur in clinical courses, which students perceive to provide more meaningful experiences for application and assessment of EBP skills to complement formal EBP courses. Questions about the value of stand-alone versus integrated EBP courses and/or whether clinical courses are places for reinforcement of EBP skills, are ongoing. As EBP educators we need to explicitly identify and recognise the informal EBP training that occurs in other courses. It may be, that with maturity in our understanding of EBP and the importance of EBP as an underpinning concept, there is a shift toward greater immersion of EBP in the teaching of all courses.
The study is not without limitations; the findings are generalizable only to physiotherapy students in an Australian University and the lack of a control group may have contributed to maturation bias. However, the strength of the current study was in being the first to use matched data to explore changes in students’ knowledge, attitudes and behaviours following a series of EBP courses across an entire entry-level program, using both matched quantitative, and qualitative data, rather than a focus on comparing singular approaches. The combined mixed methods approach of the current study has allowed the disentanglement of elements of self–reported EBP domains of Terminology, Relevance, Sympathy, Confidence and Actual Knowledge, which may contribute to changes in practice.
Completion of EBP courses over a 4 year entry-level physiotherapy program resulted in significant changes, with large effect sizes (1.7–4.3) for five of six domains relating to EBP: Relevance, Confidence, Terminology, Actual Knowledge and Practice. For the domain of Sympathy, the ES was medium (0.5). The qualitative and quantitative findings were mainly well-aligned with the exception of the Terminology domain, where the qualitative findings did not support the strength of the effect reported quantitatively. The findings of this study have implications for the timing and content of EBP curricula in entry-level health professional programs.
- EBP2 :
Evidence-Based Practice Profile
Knowledge of Research Evidence Competencies
Predictive Analytic Software
Dawes M, Summerskill W, Glasziou P, Cartabellotta A, Martin J, Hopayian K, Porzsolt F, Burls A, Osborne J. Sicily statement on evidence-based practice. BMC Med Educ. 2005;5:1–7.
Sackett DL, Rosenberg WMC, Muir Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. Br Med J. 1996;312:71–2.
Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching of evidence based medicine changes anything? A systematic review. Br Med J. 2004;329:117–1021.
Ilic D, Maloney S. Methods of teaching medical trainees evidence-based medicine: a systematic review. Med Educ. 2014;48:124–35.
Wong CS, McEvoy MP, Wiles L, Lewis LK. Magnitude of change in outcomes following entry-level evidence-based practice training: a systematic review. Int J Med Educ. 2013;4:107–14.
Bennett S, Hoffmann T, Arkins M. A multi-professional evidence-based practice course improved allied health students' confidence and knowledge. J Eval Clin Pract. 2011;17:635–9.
Long K, McEvoy M, Lewis L, Wiles L, Williams M, Olds T. Entry-level evidence-based practice (EBP) training in physiotherapy students – does it change knowledge, attitudes and behaviours? A longitudinal study. IJAHSP. 2011;9:1–11.
McEvoy MP, Williams MT, Olds TS, Lewis LK, Petkov J. Evidence-based practice profiles of physiotherapists transitioning into the workforce: a study of two cohorts. BMC Med Educ. 2011;11:100.
Lewis LK, Wong SC, Wiles L, McEvoy MP. Diminishing effect sizes with repeated exposure to evidence-based practice training in entry-level health professional students: a longitudinal study. Physiother Can. 2015;68:73–80.
Ilic D, Forbes K. Undergraduate medical student perceptions and use of Evidence Based Medicine: A qualitative study. BMC Med Educ. 2010;10:58.
Ilic D, Hart W, Fiddes P, Misso M, Villanueva E. Adopting a blended learning approach to teaching evidence based medicine: a mixed methods study. BMC Med Educ. 2013;13:169.
Tilson J, Mickan S, Sum J, Zibell M, Dylla J, Howard R. Promoting physical therapists’ use of research evidence to inform clinical practice: part 2-a mixed methods evaluation of the PEAK program. BMC Med Educ. 2014;14:125.
Bozzolan M, Simoni G, Balboni M, Fiorini F, Bombardi S, Bertin N, Da Roit M. Undergraduate physiotherapy students’ competencies, attitudes and perceptions after integrated educational pathways in evidence-based practice: a mixed methods study. Physiother Theory Pract. 2014;30:557–71.
Creswell J, Plano Clark V. Designing and conducting mixed methods research. 2nd ed. California: Sage Publications; 2011.
McEvoy MP, Williams MT, Olds TS. Development and psychometric testing of a trans-professional evidence-based practice profile questionnaire. Med Teach. 2010;32:e373–80.
Lewis LK, Williams MT, Olds TS. Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals. BMC Med Educ. 2011;11:77.
Cohen J. Statistical power analysis for the Behavioural sciences. 2nd ed. Hillsdale: Lawrence Erlbaum Associates; 1988.
Stanley M. Qualitative descriptive - a very good place to start. In: Nayar S, Stanley M, editors. Qualitative research methodologies for occupational sciences and therapy. New York: Routledge; 2015. p. 21–36.
McEvoy MP, Williams MT, Olds TS. Evidence based practice profiles: differences among allied health professions. BMC Med Educ. 2010;10:69.
Ilic D, Nordin RB, Glasziou P, Tilson J, Villaneuva E. A randomized controlled trial of a blended learning education intervention for teaching evidence-based medicine. BMC Med Educ. 2015;15:39.
Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS One. 2014;9:e86706.
Shaughnessy AF, Torro JR, Frame KA, Bakshi M. Evidence-based medicine teaching requirements in the USA: taxonomy and themes. JEBM. 2016;9:53–8.
Upton D, Upton P. Development of an evidence-based practice questionnaire for nurses. J Adv Nurs. 2006;53(4):454–8.
The authors would like to acknowledge and thank statistician Alvin Atlas (International Centre of Allied Health Evidence, iCAHE, University of South Australia) for his assistance with data management and statistical analysis and Leanne Bennett for her assistance with quantitative analysis for this study.
This research was supported by a University of South Australia Learning and Teaching Grant. The funding body did not participate in the design of the study, data collection, analysis, interpretation of data or in writing the manuscript.
Availability of data and materials
The datasets analyzed during this study are available from the corresponding author on reasonable request.
MM is a lecturer and researcher in musculoskeletal physiotherapy and evidence-based practice, with an emerging interest in mixed methods research. Her PhD in 2011 investigated differences in quantitative EBP knowledge attitudes and behaviours across health professional students, year level and exposure to EBP training and longitudinal changes.
LL is a Senior Lecturer in the physiotherapy discipline at Flinders University. Lucy teaches topics relating to research design and methods to undergraduate and postgraduate students, and is interested in student outcomes following Evidence-based practice training, and instruments to measure these outcomes.
JL is a mixed methods researcher and physiotherapist with particular interests in evidence-informed clinical practice and stroke rehabilitation. Julie’s PhD in 2012 investigated the factors that influence the quality of care patients receive following stroke. Her work is supported by a NHMRC Research Fellowship (APP1052524).
Ethics approval and consent to participate
Ethical approval for the study was gained from the Human Research Ethics Committee of the University of South Australia (protocol numbers 0000021077 for the questionnaire data and 0000030567 for the focus group data). In accordance with ethical requirements, participants were provided with full information about the research in a participant information sheet and consented to participate if they then chose to complete the questionnaires. Participants in focus groups provided written informed consent to participate. All participants were informed of their right to decline to participate, and their right to withdraw from the study at any time.
Consent for publication
Group data and no individual person’s quantitative data are reported in the manuscript. Focus group participants provided written informed consent for publication; non-identifiable participant comments from the focus groups are reported in the manuscript.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Semi-structured question guide for focus groups. Objectives of focus group questions and guide to focus group questions (DOCX 13 kb)
Qualitative participant data supporting the themes. Examples of supporting quotations for the themes and sub-themes of the qualitative data (DOCX 21 kb)
About this article
Cite this article
McEvoy, M.P., Lewis, L.K. & Luker, J. Changes in physiotherapy students’ knowledge and perceptions of EBP from first year to graduation: a mixed methods study. BMC Med Educ 18, 109 (2018). https://doi.org/10.1186/s12909-018-1212-4