Collecting evidence of validity for an assessment tool for Norwegian medical students’ non-technical skills (NorMS-NTS): usability and reliability when used by novice raters

Table 6 Consistency in scoring by Spearman correlation coefficient for category vs. elements or global score vs. categories

	Spearman correlation coefficient for category vs. elements or global score vs. categories				Cronbach’s alpha (standardized variables)
Score	Rater 1	Rater 2	Rater 3	Raters pooled	Rater 1	Rater 2	Rater 3	Raters pooled
Communication	-	-	-	-	0.94	0.74	0.90	0.92
Team communication	0.88	0.85	0.89	0.91	0.96	0.75	0.92	0.93
Establish mutual communication	0.89	0.68	0.85	0.86	0.96	0.84	0.92	0.94
Patient communication	0.90	0.43	0.77	0.82	0.95	0.87	0.96	0.95
Situational awareness	-	-	-	-	0.95	0.52	0.98	0.94
Situational assessment	0.96	0.47	0.91	0.88	0.95	0.84	0.98	0.96
Understanding of team members’ roles	0.88	0.82	0.91	0.87	0.97	0.66	0.98	0.96
Attentiveness	0.93	0.58	0.96	0.89	0.96	0.74	0.97	0.95
Teamwork	-	-	-	-	0.92	0.77	0.97	0.94
Professional modesty	0.85	0.71	0.97	0.88	0.94	0.80	0.98	0.95
Flexibility	0.92	0.52	0.97	0.88	0.93	0.88	0.98	0.95
Efficient use of team members	0.87	0.73	0.95	0.89	0.96	0.79	0.99	0.96
Decision making	-	-	-	-	0.94	0.88	0.97	0.95
Uncertainty management	0.88	0.85	0.89	0.87	0.95	0.88	0.97	0.95
Decision analysis	0.88	0.76	0.82	0.80	0.96	0.91	0.98	0.96
Leadership	0.92	0.57	0.92	0.86	0.95	0.94	0.97	0.96
Prioritization	0.87	0.90	0.95	0.90	0.95	0.89	0.97	0.95
Overall global score	-	-	-	-	0.94	0.82	0.98	0.95
Communication	0.88	0.82	0.90	0.81	0.96	0.84	0.98	0.95
Situational awareness	0.93	0.74	0.89	0.77	0.95	0.85	0.98	0.95
Teamwork	0.86	0.55	0.94	0.79	0.97	0.90	0.98	0.95
Decision making	0.94	0.69	0.93	0.86	0.95	0.86	0.98	0.95

ISSN: 1472-6920