Table 4 Summary of the psychometric properties of the K-REC*
Test property | Measure used | Acceptable results | K-REC performance |
|---|---|---|---|
Test-retest reliability (comparison of scores on an initial test to scores by the same participant on a retest) Inter-rater reliability (the degree of agreement between two independent raters) | Cohen's Kappa, ICC and percentage agreement | At least a moderate level of agreement (0.50) between testing occasions (test-retest) or raters (inter-rater) | Test-retest reliability Each of the items and total scores achieved at least a moderate level of agreement (range: 0.62 to perfect agreement) Inter-rater reliability Each of the items and total scores achieved a very good level of agreement between raters (range: 0.83 to perfect agreement) |
Content validity (instrument covers entire topic of interest) | Expert opinion | Test covers all of the main aspects of EBP | Content and revisions based on experts' suggestions |
Item difficulty (relative difficulty of each item) | The % of candidates who answer achieve a passing score | A wide range of difficulties allows a test to be used with both 'exposed' and 'not exposed' groups | Ranged from moderate (84% question 2) to difficult (15% question 8b) |
Construct validity (evidence that the instrument measures the construct that it intends to) | Discriminative validity Mean scores of 'exposed' and 'not-exposed' compared by t test | Significant difference, higher 'exposed' student scores | 'Not exposed' mean (human movement) was 4.2 and 'exposed' mean (physiotherapy) was 8.4 (p < 0.0001) |
| Â | % passing for 'exposed' and 'not-exposed' groups compared by the z-test for comparing proportions | Higher % of 'exposed' students passing | For 9 out of 10 instrument items a higher proportion of 'exposed' (physiotherapy students) than 'not exposed' (human movement students) passed. There was a significant difference between groups for 7 of the 10 instrument items. |