Skip to main content

Table 1 Sources of validity evidence (adapted from Messick 1995, Andreatta and Gruppen 2009 and Cook and Beckman 2006)

From: Validity of a new assessment rubric for a short-answer test of clinical reasoning

Source of validity evidence Description of validity evidence Method of generating validity evidence for the CHAT
1.Instrument content Extent to which instrument content is relevant to the construct of interest Developing a test blueprint for clinical reasoning in OMPT
2.Response process Extent to which the cognitive and physical processes required by the instrument can represent the construct of interest Examining feasibility and acceptability of scoring procedure
3.Instrument’s internal structure Extent to which the transformation of assessment results into a score reflects the underlying construct Calculating internal consistency
Establishing inter-rater reliability
Conducting factor analysis
4.Relationships between assessment scores and other variables Extent to which assessment results relate with other variables that possess a predicted association with the construct of interest Examining correlation with other clinical reasoning measures
5.Consequences of decisions made based on assessment results Evidence pertaining to intended and unintended consequences of interpreting and using assessment results Establishing and examining method of determining pass/fail on case history examination