Training and evaluating simulation debriefers in low-resource settings: lessons learned from Bihar, India

Table 2 Interrater reliability of CAPE and DASH variables in Bihar, India, 2015–2017 (N = 73 simulation debrief videos)

Indicator	N	Reliability	Level
CAPE Tool
Communication
Instructor questions	73	0.52 (0.24–0.69) ^§	Fair
Instructor statements	73	0.14 (−0.36–0.46) ^§	Poor
Trainee responses	73	0.96 (0.93–0.97) ^§	Excellent
Instructor questions to instructor statements ratio	73	0.19 (−0.29–0.49) ^#	Poor
Trainee responses to instructor questions + statements ratio	73	0.77 (0.63–0.85) ^§	Excellent
Objectives
Behavioral objectives mentioned	73	0.67 (0.48–0.79) ^§	Good
Cognitive objectives mentioned	73	0.34 (0.04–0.59) ^§	Poor
Technical objectives mentioned	73	0.35 (0.07–0.64) ^#	Fair
Structure
Length of debrief video	71 ^α	1.0 (0.99–1.0) §	Excellent
All three debrief structural phases present	73	0.72 (0.56–0.25) ^‡	Good
Length of description phase	73	0.78 (0.65–0.86) ^#	Excellent
Length of analysis phase	73	0.92 (0.87–0.95) ^§	Excellent
Length of application phase	73	0.67 (0.48–0.79) ^#	Good
Video Use
Incorporation of simulation video	73	1.0 (1.0–1.0) ^‡	Excellent
Length of tape segments played	73	0.98 (0.97–0.99) ^§	Excellent
Number of times tape paused during	73	0.99 (0.98–0.99) ^#	Excellent
DASH Tool
Elements
1: Maintain engaged learning environment	71 ^α	0.26 (−0.18–0.54) ^§	Poor
2: Organize the debrief	73	0.59 (0.34–0.74) ^§	Fair
3: Facilitate discussion	72 ^α	0.64 (0.43–0.77) ^§	Good
4: Identify growth opportunities	72 ^α	0.38(0.02–0.61) ^§	Poor
5: Create success plan	72 ^α	0.22 (−0.25–0.51) ^§	Poor
Mean DASH score	68 ^α	0.37 (0.02–0.61) ^§	Poor