Table 1 Summary of the properties of the ACE tool

Content validity

Expert opinion

Test covers steps 1–4 of EBM

Acceptable

Item difficulty

Percentage of candidates who correctly answered the question

Wide range of results allows implementation across a wide range of participants including novice to expert

Ranged from 36% to 84%

Internal consistency

Cronbach’s alpha

Cronbach’s alpha 0.6-0.7 is considered acceptable, 0.70-0.90 good and >0.90 excellent

Cronbach’s α = 0.69

Internal reliability

Item-total correlation (ITC)

≥0.15 is considered acceptable

Ranged from 0.14 to 0.20 all items apart from three (0.03, 0.04 & 0.06)

Item discrimination index

Item discrimination index (ranges from −1.0 to 1.0)

All items should be positively indexed, ≥ 0.20 is considered acceptable

Ranged from 0.37 to 0.84

Construct validity

Mean scores three participant cohorts (EBM-novices, EBM-intermediate and EBM-advanced) compared by ANOVA

Significant differences in mean scores with EBM-advanced > EBM-intermediate > EBM-novice

On a 15-point test, mean scores were 8.6 EBM-novice; 9.5 EBM-intermediate; and 10.4 EBM-advanced (p < 0.0001)