Skip to main content

Table 4 Summary of the psychometric properties of the K-REC*

From: Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals

Test property Measure used Acceptable results K-REC performance
Test-retest reliability
(comparison of scores on an initial test to scores by the same participant on a retest)
Inter-rater reliability
(the degree of agreement between two independent raters)
Cohen's Kappa, ICC and percentage agreement At least a moderate level of agreement (0.50) between testing occasions (test-retest) or raters (inter-rater) Test-retest reliability
Each of the items and total scores achieved at least a moderate level of agreement (range: 0.62 to perfect agreement)
Inter-rater reliability
Each of the items and total scores achieved a very good level of agreement between raters (range: 0.83 to perfect agreement)
Content validity (instrument covers entire topic of interest) Expert opinion Test covers all of the main aspects of EBP Content and revisions based on experts' suggestions
Item difficulty (relative difficulty of each item) The % of candidates who answer achieve a passing score A wide range of difficulties allows a test to be used with both 'exposed' and 'not exposed' groups Ranged from moderate (84% question 2) to difficult (15% question 8b)
Construct validity (evidence that the instrument measures the construct that it intends to) Discriminative validity
Mean scores of 'exposed' and 'not-exposed' compared by t test
Significant difference, higher 'exposed' student scores 'Not exposed' mean (human movement) was 4.2 and 'exposed' mean (physiotherapy) was 8.4 (p < 0.0001)
  % passing for 'exposed' and 'not-exposed' groups compared by the z-test for comparing proportions Higher % of 'exposed' students passing For 9 out of 10 instrument items a higher proportion of 'exposed' (physiotherapy students) than 'not exposed' (human movement students) passed. There was a significant difference between groups for 7 of the 10 instrument items.
  1. EBP Evidence based practice
  2. * Responsive validity established in a follow on study [25], with the K-REC demonstrating the ability to detect impact of EBP training in a cohort of 77 physiotherapy students (p < 0.001, effect size 1.13).