Skip to main content

Table 2 Computer-based calculation of the scores in the six categories

From: Automatic analysis of summary statements in virtual patients - a pilot study evaluating a machine learning approach

Category Method Score formula
Use of semantic qualifiers (SQ) Identification of semantic qualifiers in the statements based on the list provided by Connell et al. [11] and application of rules to compare results, occurrences and the semantic context with the NLP tree. <  2 SQ: Score = 0
> = 2 and < =4 SQ: Score = 1
>  4 SQ: Score = 2
Appropriate narrowing of differential diagnosis Identification of findings, differential diagnoses, and anatomical terms based on an adapted MeSH thesaurus and comparison of the result with analysis of the expert statement and VP metadata. (found terms of expert - terms of learner matching with expert -) / found terms of expert:
>  0.75: Score = 0
<= 0.75 and > = 0.25: Score = 1
<  0.25: Score = 2
Transformation of information Identification of transformed terms and non-transformed terms based on a list of SI units and the MeSH thesaurus and comparison with transformed terms by expert and overall length of the statement. (transformed terms - non-transformed terms /2)/ (transformed terms of expert + text length factor)
<  0.16: Score = 0
> = 0.16 and < = 0.7: Score = 1
>  0.7: Score = 2
Factual accuracy Identification of contradicting use of SQ in the learner and expert statement contradicting information found: score = 0, else score = 1.
Patient name used Identification of a person token in the NLP tree person identified: score = 1, else score = 0.
Global rating Sum of the five categories Sum <=2: Score = 0
Sum > 2 and < =5: Score = 1
Sum > 5: Score = 2