Table 4 Psychometric properties by individual test item
Reliability | Pass Rate | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
Item | Inter-rater (n = 108) | Intra-rater 1 (n = 22) | Intra-rater 2 (n = 22) | IDI (n = 108) | ITC (n = 108) | All (n = 108) | Novice (n = 31) | Trained (n = 50) | Expert (n = 27) | p-value† |
1 | 0.92 | 0.97 | 0.95 | 0.39 | 0.44 | 75% | 68% | 74% | 85% | 0.30 |
2 | 0.90 | 0.83 | 0.96 | 0.46 | 0.40 | 71% | 68% | 74% | 70% | 0.83 |
3 | 0.89 | 0.95 | 0.89 | 0.64 | 0.65 | 56% | 32% | 60% | 78% | < 0.01 |
4 | 0.83 | 0.87 | 0.81 | 0.68 | 0.64 | 80% | 52% | 92% | 89% | < 0.01 |
5 | 0.73 | 0.96 | 0.81 | 0.61 | 0.52 | 56% | 39% | 56% | 74% | 0.03 |
6 | 0.69 | 0.77 | 0.62 | 0.46 | 0.52 | 76% | 61% | 72% | 100% | < 0.01 |
7 | 0.78 | 0.91 | 0.72 | 0.54 | 0.66 | 24% | 3% | 18% | 59% | < 0.01 |
8 | 0.47 | 0.62 | 0.95 | 0.61 | 0.46 | 36% | 42% | 78% | 81% | < 0.01 |
9 | 0.41 | 0.76 | 0.78 | 0.25 | 0.20 | 65% | 58% | 66% | 70% | 0.60 |
10 | 0.99 | 0.99 | 1.0 | 0.50 | 0.57 | 17% | 3% | 2% | 59% | < 0.01 |
11 | 0.96 | 0.97 | 0.96 | 0.29 | 0.62 | 10% | 0% | 4% | 33% | < 0.01 |
12 | 0.61 | 0.91 | 0.67 | 0.39 | 0.41 | 20% | 10% | 12% | 48% | < 0.01 |
13 | 0.91 | 1.0 | 1.0 | 0.54 | 0.48 | 25% | 19% | 14% | 52% | < 0.01 |
14 | 0.94 | 1.0 | 1.0 | 0.54 | 0.44 | 38% | 16% | 32% | 74% | < 0.01 |