Skip to main content

Table 5 Fleiss Kappa coefficient for each station item (inter-rater reliability)

From: Validation of an instrument for the evaluation of exchange transfusion (INEXTUS) via an OSCE

Station Fleiss’ kappa 95% CI p value
item
Station 1. Clinical history and risk approach of the patient with neonatal jaundice
 Identifies background of pregnancy and childbirth 0.75 (0.59, 0.91)  < 0.001
 Reviews prenatal exams 0.67 (0.50, 0.83)  < 0.001
 Identifies history and risk factors for severe neonatal jaundicea - - -
Station 2. Comprehensive evaluation of the patient with jaundice
 Evaluates skin and mucosal colouration describing percentage of extent of involvementa - - -
 Evaluates signs and symptoms of acute hyperbilirubinemic encephalopathy 0.61 (0.45, 0.78)  < 0.001
 Evaluates signs and symptoms compatible with haemolysis 0.84 (0.68, 1.00)  < 0.001
Station 3: Definition of clinical behaviour
 Determines whether to use diagnostic testing and requests the appropriate ones 0.75 (0.59, 0.92)  < 0.001
 Performs comprehensive interpretation of the data and classifies the patient 0.60 (0.44, 0.76)  < 0.001
 Indicates that exchange transfusion is required according to the use of the nomograma - - -
 Explains the behaviour to the parents and resolves their concerns 0.78 (0.62, 0.95)  < 0.001
 Signed informed consent 0.69 (0.52, 0.85)  < 0.001
Station 4. Preparation of the equipment for the procedure
 The student verifies that they have the appropriate personnel to perform the procedure 1.00 (0.83, 1.1)  < 0.001
 Properly select the cathetera - - -
 Requests, prepares and verifies equipment for catheter passage 1.00 (0.83, 1.1)  < 0.001
 Requests, prepares and checks equipment for exchange transfusion - - -
 Requests the blood from the laboratory and verifies it when it is delivered 0.88 (0.79, 1.04)  < 0.001
 Determines the type of fluid to be used and the volume of replacementa - - -
Station 5. Performing exchange transfusion
 Verifies patient condition and monitors vital signsa - - -
 Performs the passage of the umbilical catheter properlya - - -
 Performs sequential and orderly exchange transfusiona - - -
 Indicates post-exchange transfusion monitoring to detect the appearance of early complicationsa 1.00 (0.83, 1.1)  < 0.001
 Requests diagnostic testing at the end of the procedure and 6 h after ita 0.70 (0.53, 0.86)  < 0.001
Station 6. Information to parents
 Reports the outcomes of the procedure to parentsa 0.75 (0.59, 0.91)  < 0.001
  1. Interpretation of the kappa values: < 0.2 poor, 0.21—0.40 weak, 0.41—0.60 moderate, 0.61—0.80 good, 0.81—1.00 very good
  2. aThe Fleiss Kappa coefficient could not be determined in these items since all the ratings were equal, either because it was not performed or all were performed adequately