Skip to main content

Table 5 Inter-rater agreement statistics. ICC and Kendall’s W

From: Collecting evidence of validity for an assessment tool for Norwegian medical students’ non-technical skills (NorMS-NTS): usability and reliability when used by novice raters

 

All raters

Rater 2 and 3

Score

ICC(3,1)

Kendall’s W

ICC(3,1)

Kendall’s W

Communication

0.49

0.69

0.37

0.71

Team communication

0.43

0.63

0.48

0.77

Establish mutual communication

0.55

0.75

0.45

0.80

Patient communication

0.54

0.68

0.45

0.74

Situational awareness

0.50

0.69

0.43

0.73

Situational assessment

0.27

0.51

0.07

0.56

Understanding of team members’ roles

0.39

0.63

0.13

0.58

Attentiveness

0.44

0.68

0.37

0.76

Teamwork

0.40

0.62

0.20

0.63

Professional modesty

0.25

0.51

0.02

0.55

Flexibility

0.41

0.67

0.40

0.76

Efficient use of team members

0.40

0.62

0.25

0.64

Decision making

0.44

0.68

0.49

0.79

Uncertainty management

0.36

0.57

0.46

0.75

Decision analysis

0.43

0.61

0.58

0.81

Leadership

0.49

0.72

0.48

0.82

Prioritization

0.33

0.56

0.37

0.71

Overall Global Score

0.53

0.73

0.55

0.80

Sum of communication elements

0.58

0.76

0.51

0.81

Sum of situational awareness elements

0.41

0.67

0.21

0.67

Sum of teamwork elements

0.42

0.68

0.28

0.71

Sum of decision-making elements

0.46

0.66

0.55

0.82

Sum of all elements

0.50

0.72

0.45

0.82

Sum of categories

0.52

0.73

0.45

0.80