Category | Method | Score formula |
---|---|---|
Use of semantic qualifiers (SQ) | Identification of semantic qualifiers in the statements based on the list provided by Connell et al. [11] and application of rules to compare results, occurrences and the semantic context with the NLP tree. | < 2 SQ: Score = 0 > = 2 and < =4 SQ: Score = 1 > 4 SQ: Score = 2 |
Appropriate narrowing of differential diagnosis | Identification of findings, differential diagnoses, and anatomical terms based on an adapted MeSH thesaurus and comparison of the result with analysis of the expert statement and VP metadata. | (found terms of expert - terms of learner matching with expert -) / found terms of expert: > 0.75: Score = 0 <= 0.75 and > = 0.25: Score = 1 < 0.25: Score = 2 |
Transformation of information | Identification of transformed terms and non-transformed terms based on a list of SI units and the MeSH thesaurus and comparison with transformed terms by expert and overall length of the statement. | (transformed terms - non-transformed terms /2)/ (transformed terms of expert + text length factor) < 0.16: Score = 0 > = 0.16 and < = 0.7: Score = 1 > 0.7: Score = 2 |
Factual accuracy | Identification of contradicting use of SQ in the learner and expert statement | contradicting information found: score = 0, else score = 1. |
Patient name used | Identification of a person token in the NLP tree | person identified: score = 1, else score = 0. |
Global rating | Sum of the five categories | Sum <=2: Score = 0 Sum > 2 and < =5: Score = 1 Sum > 5: Score = 2 |