Evidence‐searching capability among health care professionals: a comparative study

Abstract

Background

Evidence-based practice is among core competencies of health care professionals (HCPs). However, the levels of evidence-searching capability may differ among various disciplines of HCPs as they receive different education and trainings for various durations in medical schools and teaching hospitals.

Methods

This study aimed to compare the evidence-searching capability among different disciplines of HCPs and identify which aspects need to be reinforced. From a teaching hospital, we recruited 80 HCPs of various disciplines and compared their evidence-searching capability by using a validated scale. To examine if sex and education levels affect evidence-searching capability, we performed a multiple linear regression analysis with collinearity diagnostics.

Results

Physicians and pharmacists performed significantly better than other disciplines in the seven formative assessment items and the summative item (all P < 0.05). No collinearity was detected between discipline and age nor level of education. Except for the 2nd formative assessment item (correlation coefficient 0.24 ± 0.12, P = 0.04), participant’s levels of education did not affect evidence-searching capability. Age was associated with lower evidence-searching capability in five formative and the summative assessment items.

Conclusions

We found a better evidence-searching capability among physicians and pharmacists than other HCPs who may require more training on evidence-searching skills. Also, evidence-searching skills training should be provided to HCPs irrespective of age and education levels.

Background

Over the past few decades, evidence-based medicine (EBM) has become an important issue in clinical practice, medical education, and clinical research. Also, evidence-based practice has been listed by the US Institute of Medicine as one of five core competencies of health care professionals (HCPs). EBM aims to apply the best evidence in resolving clinical uncertainty and efficiently provide effective health care [1]. The practice of EBM is composed of five steps (i.e. ‘5A’): (1) asking a focused question; (2) acquiring relevant evidence; (3) appraisal of the evidence obtained; (4) applying the evidence to clinical care; and (5) auditing of the evidence-based performance [2]. However, in the real world, HCPs are usually too busy to have adequate time to complete the whole 5A process [3]. Only EBM researchers have enough ability and time in practicing 5A and producing the best evidence for clinical application. Therefore, the goal of EBM education is no longer expecting all HCPs to have the ability to go through the 5A process, but enabling them to acquire the best evidence efficiently [4, 5].

There are various disciplines of HCPs including physicians, registered nurses, pharmacists, allied health professionals such as respiratory therapists and dietitians. To maintain the normal operation of health care industry, administrative staff, clinical teachers, and research staff are also needed. They receive different education and trainings for various durations in medical schools and teaching hospitals. Also, the college entry requirements vary among different disciplines of HCPs, with physicians usually the strictest. Therefore, the levels of evidence-searching capability may differ among various disciplines of HCPs. The objective of this study was to compare the evidence-searching capability among different disciplines of HCPs and to identify which aspects need to be reinforced in EBM education.

Methods

This study was a sub-analysis of a previous study in which a scale for measuring evidence-searching capability has been developed and validated [4]. From a teaching hospital, we recruited a total of 80 HCPs. We used a validated scale to measure the evidence-searching capability of these participants as shown in Table 1 [4]. This scale is composed of 15 items (numbered from F1 to F15) that assess different aspects of evidence-searching skills (formative assessment) and 1 summative rating item (numbered as S1) that assess the overall ability in evidence-searching (summative assessment). The first four formative assessment items (F1 to F4) examine the ability to build up the PICO structure (P standing for patient or population, I for intervention, C for comparison, and O for outcome) [6] of a clinical question and to propose associated search terms. The 5th to 15th formative assessment items (F5 to F15) test the ability in devising searching strategy and searching skills [4].

Table 1 Scale for measuring evidence-searching capability.

Statistical analysis

We compared the evidence-searching skills between different disciplines of HCPs and analyzed the 15 items and 1 summative rating item, respectively, by using the Kruskal-Wallis test with post-hoc Dunn’s test. To examine if age group and level of education affect evidence-searching capability, we performed a multiple linear regression analysis with collinearity diagnostics. Collinearity refers to near perfect linear combinations of two variables and multicollinearity involves more than two variables, leading to unstable regression estimates with high standard errors. We used variance inflation factor (VIF) to evaluate multicollinearity, with a VIF of > 10 indicating multicollinearity [7]. We considered HCPs aged > 35 years as having completed trainings and thus separated the participants’ age into two categories: junior HCPs (aged ≤ 35 years) and senior HCPs (aged > 35 years). The levels of education were separated into two levels: undergraduate degree (diploma or bachelor) and graduate degree (master or doctorate). A P value of < 0.05 was considered significant. The Stata 13.1 for Windows (StataCorp LP, College Station, US) was used for statistical analyses.

Results

The data on the evidence-searching capability of the 80 participants, including 23 men (28.8 %) and 57 women (71.3 %), were collected and analyzed. The disciplines and education levels of participants are presented in Table 2. Data on 2 administrative staff, 5 research assistants, and 2 clinical teachers were pooled to form an ‘Others’ group because the sample size was small (n = 9).

Table 2 Disciplines and education levels of participants

The data on the 15 formative assessment items and 1 summative assessment item of participants are shown in Table 3. Various disciplines of HCPs significantly differed in the seven formative assessment items (F1 to F6 and F8) and the summative item (S1) (all P < 0.05). By using the post-hoc Dunn’s test, physicians and pharmacists did not differ in these item (all P > 0.05). However, both physicians and pharmacists differed from registered nurses, allied health professionals, and others in most of these items (P < 0.05, data not shown).

Table 3 Evidence-searching capability of various health care professionals

As shown in Table 4, the multiple linear regression analysis with collinearity diagnostics detected no collinearity among the age group, level of education, and discipline of participants (all VIF < 10). Except for the 2nd formative assessment item (correlation coefficient 0.24 ± 0.12, P = 0.04), participants’ levels of education did not significantly affect evidence-searching capability. Senior age (> 35 years) was negatively associated with evidence-searching capability in five formative assessment items (F1, F2, F7, F8, and F10) and the summative assessment (S1) item.

Table 4 Multiple linear regression analysis with collinearity diagnostics. For each coefficient, corresponding standard error was reported

Discussion

This study found a higher level of evidence-searching capability of physicians and pharmacists than other disciplines of HCPs, especially in the building up of search terms, prioritized use of secondary databases, use of both Medical Subject Headings (MeSH) term and free text in searching, and devising search strategy by using appropriate Boolean operators. Also, physicians and pharmacists had a better overall ability in evidence-searching than other HCPs. Therefore, more EBM trainings including evidence-searching capability in the aforementioned skills should be provided for HCPs, especially those disciplines other than physicians and pharmacists.

The better evidence-searching capability found in physicians and pharmacists may be related to the different levels of contact with latest evidence. When compared to other disciplines of HCPs, physicians and pharmacists are in greater demand for up-to-date evidence on clinical trials as newly developed drugs are springing up over the past several decades. Therefore, physicians and pharmacists may have more experiences in searching databases than other HCPs.

Compared to HCPs of younger age, senior HCPs should have possessed better evidence-searching capability because of having a better understanding of the context of the clinical PICO question. By contrast, one previous study suggested younger age as a facilitating factor for evidence-based practice [8]. Also, our study illustrated that senior HCPs had a slightly lower ability in certain items of evidence-searching capability. Similarly, HCPs with graduate degree should have received more training in graduate school and thus a better ability and more experiences in searching databases than those with undergraduate degree. One Taiwanese study has found better performance in EBM skills including literature search in HCPs with a master degree than those with an undergraduate degree [9]. Another Australian study found more frequent access to electronic evidence resources among HCPs who had received postgraduate education than those who had not [10]. Nevertheless, our study found only in the second formative assessment item (‘Propose the search terms and synonyms for I (Intervention)’) did HCPs with a graduate degree have a slight better performance than those without a graduate degree. Our findings suggest that evidence-searching skills training should be provided to HCPs irrespective of age and education levels.

There are several limitations in this study. First, this pilot study was a sub-analysis and the participants were recruited from a single teaching hospital. However, the HCPs received education and training from various medical schools and teaching hospitals. Therefore, our findings may represent a miniature of EBM education. Also, one previous nationwide survey of 605 physicians and 551 registered nurses in 61 hospitals across Taiwan found that physicians possessed better self-rated evidence-acquiring skills than registered nurses [11]. Second, the sample size of the pharmacist group was only 4 in the present study and might have led to underpowering (type II error). Nevertheless, we detected a higher evidence-searching capability in pharmacists than other HCPs. Third, the setting of this study was limited to one country. The findings that certain evidence-searching skills need to be reinforced in EBM curriculum may not be extrapolated to other countries; yet there has been a trend advocating training on evidence-acquiring ability at the point of care [5, 12].

Conclusions

Our study suggested a higher level of evidence-searching capability among physicians and pharmacists than other HCPs. HCPs other than the two disciplines may require more training on evidence-searching skills, especially in the building up of search terms and strategy as well as prioritized use of appropriate secondary databases. Also, our study illustrated that senior age and graduate education did not increase evidence-searching capability. Therefore evidence-searching skills training should be provided to HCPs irrespective of age and education levels.