|
Content validity
|
Expert opinion
|
Test covers steps 1–4 of EBM
|
Acceptable
|
|
Item difficulty
|
Percentage of candidates who correctly answered the question
|
Wide range of results allows implementation across a wide range of participants including novice to expert
|
Ranged from 36% to 84%
|
|
Internal consistency
|
Cronbach’s alpha
|
Cronbach’s alpha 0.6-0.7 is considered acceptable, 0.70-0.90 good and >0.90 excellent
|
Cronbach’s α = 0.69
|
|
Internal reliability
|
Item-total correlation (ITC)
|
≥0.15 is considered acceptable
|
Ranged from 0.14 to 0.20 all items apart from three (0.03, 0.04 & 0.06)
|
|
Item discrimination index
|
Item discrimination index (ranges from −1.0 to 1.0)
|
All items should be positively indexed, ≥ 0.20 is considered acceptable
|
Ranged from 0.37 to 0.84
|
|
Construct validity
|
Mean scores three participant cohorts (EBM-novices, EBM-intermediate and EBM-advanced) compared by ANOVA
|
Significant differences in mean scores with EBM-advanced > EBM-intermediate > EBM-novice
|
On a 15-point test, mean scores were 8.6 EBM-novice; 9.5 EBM-intermediate; and 10.4 EBM-advanced (p < 0.0001)
|
