Table 2 Studies that develop or test scoring systems for peer evaluation of invasive medical procedure performance
From: Peer evaluation and feedback for invasive medical procedures: a systematic review
Scoring System (Procedure) | Source | Agreement testing | Number of Evaluators | Number of Clinicians Evaluated | Total Cases Reviewed |
|---|---|---|---|---|---|
CARS (Coronary Angiography) | Rader 2014 | ICC = 0.42 and 0.49 | 2 | 10 | 40 |
CIQ (Colonoscopy) | Keswani 2020 | Spearman r = 0.92 | 9 | 17 | 102 |
Lee 2011 | None | 4 | 11 | 110 | |
Rex 2000 | None | 4 | 2 | 20 | |
CSPAT (Cold-snare Polypectomy) | Patel 2019 | Kappa = 0.58 (overall) | 13 | Not specified | 55 |
DOPS (Colonoscopy) | Barton 2012 | G-theory analysis: G = 0.81 | 28 | 147 | 162 |
DOPyS (Polypectomy) | Duloy 2018 | Weighted Cohen's k = 0.80 | 2 | 13 | 130 |
Duloy 2019 | Weighted Cohen's k = 0.69 | 2 | 11 | 220 | |
Gupta 2011 | G-theory analysis | 7 | 23 | 60 | |
Gupta 2012 | Kappa = 0.32 (trained assessors), Kappa = 0.07 (untrained assessors) | 4 | 8 | 32 | |
EBUSAT (EBUS-TBNA) | Konge 2015 | Cronbach's alpha = 0.95 | 3 | 18 | 58 |
EVAT (Endoscopic Submucosal Dissection) | Takao 2020 | ICC = 0.87 | 3 | 8 | 17 |
GAGES (Colonoscopy, Upper Endoscopy) | Vassiliou 2010 | ICC 0.97 for GAGES-C and 0.96 for GAGES-UE | Not specified | 139 | 139 |
GiECAT (Colonoscopy) | Scaffidi 2018 | ICC = 0.91 (inter-rater), 0.85 (live vs. video) | 7 | 40 | 40 |
|  | Walsh 2015 | ICC = 0.85 | Not specified | 61 | 116 |
Unnamed (Percutaneous Coronary Intervention) | Blows 2012 | None | 3 | 10 | 326 |
Unnamed (Colonoscopy) | Boyle 2012 | None | Not specified | 8 | 24 |
Unnamed (Bronchoscopy) | Konge 2012 (1) | Cronbach's alpha = 0.86 | 2 | 19 | 57 |
Unnamed (FNA guided by EUS) | Konge 2012 (2) | Cronbach's alpha = 0.93 | 3 | 6 | 30 |
Unnamed (Colonoscopy and flexible sigmoidoscopy) | Sarker 2008 | Cronbach’s alpha = 0.80 and 0.83 | Not specified | 9 | 77 |
Unnamed (Colonoscopy) | Shah 2002 | Kappa = 0.63 (overall) | 3 | 18 | 22 |
Unnamed (Flexible Sigmoidoscopy) | Thomas-Gibson 2006 | ICC = 0.89 | Not specified | Not specified | 43 |