Development and validation of the critical thinking disposition inventory for Chinese medical college students (CTDI-M)

Background This study aimed to develop and conduct psychometric testing of the Critical Thinking Disposition Inventory to measure the critical thinking disposition of Chinese medical college students. Methods The study was conducted in two stages: (a) item generation, reliability analysis and exploratory factor analysis (EFA) and (b) confirmatory factor analysis (CFA) and testing of psychometric properties (Cronbach’ s alpha, test-retest reliability and convergent validity). The subjects included 1035 Chinese medical college students. The test-retest reliability of the instrument was determined at a two-week interval (n = 61). A general linear regression model was developed to examine the predictive effects of gender, age and major on CT disposition. The data were analysed with SPSS 22.0 and Amos 21.0 during item development and the reliability and validity analyses. Vista was utilized for parallel analysis during the principal axis analysis. Results Eighteen final items were sorted into 3 factors, which were identified as “Open-mindedness”, “Systematicity/Analyticity” and “Truth-seeking”, with cumulative variance of 41.37, 46.00 and 49.59%, respectively. The Cronbach’s alpha was 0.924, and the factors’ alphas ranged from 0.824 to 0.862. The correlational analysis indicated significant correlations between the subscales of the CTDI-CM and the total scores of the CTDI-CV, indicating modest evidence for the convergent validity of the CTDI-CM. Gender, age and education significantly predicted the CT disposition of Chinese medical students. Open-mindedness and Systematicity/Analyticity were higher for medical students than for nursing students. Conclusions This study presents a reliable and valid instrument for clinical thinking disposition. Future studies should explore other predictive factors of CT dispositions (e.g., cognitive/motivational) and criterion validity.


Background
Critical thinking is increasingly needed to produce adaptive and flexible learners in the information age (Dwyer, Hogan, & Stewart, 2014). The importance of being 'critical' for medical students and practitioners has also been increasingly emphasized [1]. The Delphi Report presented in 1990 by experts from the US and Canada defined critical thinking (CT) as the ability to apply cognitive skills (interpretation, analysis, inference, evaluation, explanation, and self-regulation) and the disposition towards CT (being open-minded or intellectually honest) [2,3]. Critical thinking has frequently been referred to as individuals' cognitive ability to think and make correct decisions independently and to utilize rational/logical thought [4,5].
Increasing attention has been paid to the individual differences in critical thinking disposition, which is defined as the tendency or attitude to utilize a particular skill voluntarily and the willingness to make the effort to apply it [6], or, simply put, the attitude towards critical thinking. Dispositions towards critical thinking are vital to critical-thinking performance [7] and professional clinical judgement [8]. Practically, both disposition and ability are necessary for critical thinking [9]. The assessment of CT dispositions may help to identify targets to promote critical thinking through training programmes in both professional and educational contexts.
The most widely used measurement tool in China for this purpose is the translated version of the California Critical Thinking Disposition Inventory (CCTDI) [10,11]. The CCTDI is designed for the general adult working population at all levels and for students in grades 10 and above. The CCTDI includes seven subscales: "Inquisitiveness", "Systematicity", "Analyticity", "Truth-seeking", "Open-mindedness", "Self-confidence" and "Maturity" [12]. Yeh et al. translated the CCTDI into Chinese and administered it to a sample of nursing undergraduate students in Taiwan. Compared to the English CCTDI (alpha = 0.79), the overall alpha (0.71) of the Chinese CCTDI was inferior, and the internal consistencies (Cronbach's α) of three subscales were inadequate (Open-mindedness = 0.34, Analyticity = 0.40, and systematicity = 0.47) [11]. In addition, the content validity of these three subscales was moderate (CVI = 0.50 to 0.67) compared to the English CCTDI subscales (CVI = 0.82 to 1). Peng et al. developed a conceptually equivalent version of the CCTDI, the CTDI-CV (Critical Thinking Disposition Inventory-Chinese Version), which showed more satisfactory subscale alphas ranging from 0.54 to 0.77 and an overall alpha of 0.90. The CVIs for the "Open-mindedness", "Analyticity" and "Systematicity" subscales were improved to 0.90-1. However, the Cronbach's α (Chinese CCDTI = 0.46, CTDI-CV = 0.57) and the CVI (Chinese CCDTI = 0.70, CTDI-CV = 0.60) of the "Maturity" subscale were lower for both the Chinese CCTDI and the CTDI-CV than for the English CCTDI (alpha = .64, CVI = .90) [10]. In another translated Chinese version of the CCTDI for university students, the Cronbach's α of the "Open-mindedness" (0.39), "Systematicity" (0.43) and "Maturity" (0.45) subscales were also not satisfactory [13]. The conceptualizations and measurement of CT dispositions in the Chinese-speaking population merit further exploration.
An important factor that may explain the diversity of the psychometric characteristics of versions of the CCTDI is cultural sensitivity. According to a literature review, Asian students tend to show less critical thinking dispositions compared with students from non-Asian countries [14][15][16].
(1). Analyticity and Systematicity are the cognitive components of CT dispositions that are culturally sensitive. "Analyticity" means the use of evidence and the anticipation of possible consequences to resolve problems. "Systematicity" means being organized, focused and diligent in resolving problems. In comparison with American university students, the percentage of students with lower than moderate levels of "Analyticity" and "Systematicity" is greater in the Chinese population [13]. Specifically, for Chinese medical undergraduates, the average score for "Systematicity" was at the lowest level of all subscales of CT dispositions [17]. The Systematicity of Chinese nursing students was at a moderate level [10]. A previous study suggested that Western cultures tend to be analytic, whereas traditional Chinese societies tend to be holistic and synthetic, which is manifested in language [18], thinking models of medicine [19] and preferences for dialectical proverbs and dialectical resolution of social contradictions [20]. Since the cognitive components of CT dispositions are crucial for effective critical thinking, cultural differences in thinking patterns need to be considered in the context of Chinese culture. (2). "Inquisitiveness", "Truth-seeking" and "Openmindedness" are the motivation components of CT dispositions. CT ability is presumed to be significantly related to learning motivation [21]. First, Inquisitiveness refers to the inclination to be curious and eager to learn knowledge that may not be of immediate use. This may encourage learners to engage in deep and creative reasoning [22]. Second, the average score for Truth-seeking was at the lowest level of all subscales of CT dispositions among Chinese medical undergraduates [17]. The Truth-seeking of Chinese nursing students was at a moderate level [10]. Asian university students tended to learn for pragmatic purposes compared to American university students, who tend to possess an attitude that values truth [23]. While Open-mindedness and Truth-seeking have been deemed important in good critical thinkers, only Truth-seeking significantly predicts Chinese students' critical thinking performance, and their responses are more concerned with seeking solutions from authorities or preconceptions rather than seeking independent evidence or reasoning [24]. Third, in comparison with American university students, the proportion of students displaying a lower than moderate level of Openmindedness is greater in the Chinese population [13]. Open-minded people in Asian culture may be more inclined to accept contradictory propositions and avoid social conflicts. These cultural diversities may explain the low internal consistency of "Openmindedness" for the CCTDI in Chinese nursing students [11]. Therefore, the motivation components of CT dispositions should focus on these culturally sensitive traits and examine how these traits may influence medical performance.
(3). "Self-confidence" and "Maturity" are the personality components of CT dispositions. The Self-confidence subscale measures individuals' confidence in their thinking and reasoning processes. Emotionally taxing situations, threats to self-identity (e.g., gender prejudice) or inappropriate priorities of values challenge the self-confidence of critical thinkers and impair their self-reflection ability [3]. Thus, Self-confidence is particularly important to CT dispositions. The Maturity scale assesses the disposition towards judicious decision-making and thus requires self-reflection, which develops gradually from adolescence to adulthood. Chinese students who exhibit a lower than moderate level of Maturity constitute a greater proportion compared with American university students [13]. More research is needed to develop specialized instruments for the Chinese context.
Empirically, critical thinking is valued both for nursing [25] and clinical expertise [4]. Critical thinking can improve diagnostic skills and reduce errors in management [4]. Critical thinking constitutes not only logical thinking ability but also problem-solving ability, which is content dependent [26]. For instance, CT enhances the capacity to transfer knowledge and skills obtained from the classroom to the clinical context [27]. As a result, CT skills have been emphasized by the Global Minimum Essential Requirements (GMER) as one of the seven student competence domains [28]. The competences contained in the GMER define the learning outcomes of medical graduates that are required for medical practice [29]. These domains have been assessed with an objective structured clinical examination (OSCE) that defined competence in critical thinking and research as the ability to generate and test hypotheses with scientific methods [30]. Although critical thinking competencies are generic abilities, CT behaviours may be more effectively learned or taught in specific discipline settings [31]. Therefore, knowledge about the individual differences of CT dispositions specific to the medical discipline could facilitate the teaching of critical thinking. In contrast to the two Chinese versions of the CCTDI that were directed at nursing students only, the current study also included medical students during the development of the CT disposition assessment tool.
An instrument for measuring critical thinking dispositions developed independently for Chinese medical students aims to (1) increase the content validity of specific factors of CT disposition (i.e., Open-mindedness, Analyticity, Systematicity and Maturity) and (2) identify those traits with greater cultural differences and evaluate the criterion validity of their measurement.  Only comparisons between the CTDI-CM and the CTDI-CV were reported due to the lack of a scoring system for the CCTDI. The current study was approved by the ethics committee of the university. The data collection in each phase was conducted after the enrolment of each student (from April to June) and before the school year began (Fig. 1). This was intended to alleviate the learning effects of college course studies and academic training on CT disposition. Verbal informed consent was obtained from each participant prior to the surveys. All of the participants' information was confidential, and the participants could withdraw from the study at any time on a voluntary basis. The sample size had sufficient power to detect significant differences revealed by the power analysis.

Statistics
Statistical analysis was conducted using the Statistical Package for the Social Sciences (SPSS) 22.0 for Windows (SPSS, Chicago, IL, USA).

Development of CTDI-CM and item analysis
The remaining 18 items (Table 1) survived the two criteria of item analysis. (1) Item discriminability. A critical ratio (CR) index was used to quantify the difference of each item for the 27% highest-scoring group versus the 27% lowest-scoring group. Two independent-sample t tests with a CR value exceeding 3 indicated good item discriminability [32]. (2) Item homogeneity test. Item-total statistics were computed to examine the content homogeneity between items, with a value over 0.3 indicating that the item correlated well with the total scale [33].

Reliability analysis of the CTDI-CM
(1) For internal consistency, Cronbach's alpha coefficient was calculated for the scores of each subscale and the total scale based on the assumption that the CTDI-CM measures the single trait of CT disposition. The questionnaire had substantial internal consistency according to the following rules of thumb: > 0.9 as excellent, > 0.8 as good, > 0.7 as acceptable, > 0.6 as questionable, > 0.5 as poor and < 0.5 as unacceptable [34]. (2) Item-to-total correlations were examined with the recommended value of up to 0.3 [35]. (3) Test-retest reliability was analysed by the intra-class correlation coefficients (ICCs) with a two-week interval, which was long enough to avoid confounding effects of practice and allow for a natural change in the construct. ICC provides an estimate of the reproducibility (stability) of the assessments according to the following criteria: > 0.6 indicates good reliability and > 0.74 indicates excellent reliability [36].

Validity analysis of the CTDI-CM
Exploratory factor analysis (EFA) Principal axis factor analysis was performed to identify the latent variables of CT dispositions. The Kaiser-Meyer-Olkin (KMO) measure and Bartlett's Test of Sphericity were examined for the adequacy of the factor analysis. Factors were extracted based on Kaiser's (1960) eigenvalues-greater-than-1 rule and the scree test after a direct oblimin rotation solution. The validation procedures were implemented with SPSS 22.0. To replicate these results, we also conducted parallel analysis, which is recommended as a statistically based and more validated procedure for determining the number of components [37]. The parallel analysis was conducted with ViSta (visual statistics system) (https://www.uv.es/ visualstats/Book/DownloadBook.html) [38]. Parallel analysis using Monte Carlo simulations of permutations of 1000 randomly generated datasets was conducted. The eigenvalue of the raw data exceeding the eigenvalue of the 95th percentile (and mean) of random data could be extracted as a factor [37]. To replicate the results, we also performed regression analyses for convergent validity with the CTDI-CM subscales as the predictors and the CTDI-CV total score as the outcome variable, with a positive beta coefficient of 0.40-0.59 as recommended [40].

GLM analysis of CT dispositions of Chinese medical college students
The general linear regression model (model 1) was utilized to estimate the significant predictive factor of CT dispositions, with gender (male = 1; female = 2), age and education (undergraduate student = 1; graduate student = 2) as independent variables and the subscale scores of the CTDI-CM (Open-mindedness, Systematicity/Analyticity, Truth-seeking) as dependent variables. The information on major for graduate students was omitted during the data collection. Thus, we only report the comparison between the two majors (medical vs. nursing) in undergraduate students.

Item and reliability analysis
We obtained an 18-item questionnaire on Chinese medical college students' critical thinking dispositions. Item analyses showed that the items of the CTDI-M were recognized by the respondents as relevant to CT disposition. Additionally, these items could discriminate those with higher versus lower levels of CT disposition (Fig. 2) (Ps < 0.003, Bonferroni corrected). The item-total analysis confirmed the homogeneity of content among items ( Table 2). The results also showed that the questionnaire had excellent internal consistency (0.776~0.965) and two-week test-retest reliability of the CDTI-CM (0.808~0.965) ( Table 3).

Validity analysis
The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy was 0.946, which was greater than 0.5 (and close to one), indicating satisfactory factor analysis. Bartlett's Test of Sphericity (χ 2 = 3410.42, P < 0.001) (less than 0.05) suggested the appropriateness of the factor analysis model. The principal axis factor analysis identified a three-factor model, with the eigenvalues of 3 factors exceeding one ( Table 4). The scree test confirmed the three-factor model of CT dispositions (Fig. 3). The first factor (Open-mindedness) explained 41.37% of the accumulated variance with seven items (with an eigenvalue of 7.95), which indicated an open attitude and willingness to listen to and consider other people's ideas and suggestions before arriving at conclusions. The second factor (Systematicity/Analyticity) explained 46.00% of the accumulated variance with six items (with an eigenvalue of 1.32), which indicated the traits of being painstaking and careful and demonstrating effective decision-making and problem-solving. The third factor (Truth-seeking) explained 49.59% of the accumulated variance with five items (with an eigenvalue of 1.12), which indicated the state of active curiosity, active engagement in thinking, and avoiding the negative effects of mental state. To replicate these results, three factors were extracted through parallel analysis after comparison of the actual data eigenvalues with the eigenvalues The convergent validity was examined by correlating the subscales of the CTDI-CM with the CTDI-CV. Open-mindedness on the CTDI-CM was positively correlated with Maturity and Self-confidence on the CTDI-CV (r = 0.170, P < 0.005, n = 282). The Systematicity/Analyticity of the CTDI-CM was positively correlated with self-confidence (r = 0.215, P < 0.001, n = 282) on the CTDI-CV and negatively correlated with Inquisitiveness on the CTDI-CV (r = − 0.219, P < 0.001, n = 284). Truth-seeking on the CTDI-CM was positively correlated with self-confidence on the CTDI-CV (r = 0.200, P < 0.005, n = 280) and negatively correlated with Inquisitiveness (r = − 0.318, P < 0.001) and Analyticity (r = − 0.129, P < 0.05, n = 179) on the CTDI-CV. However, there was no significant correlation between the total scores of the CTDI-CV and the CTDI-CM (r = 0.028, P = 0.643) ( Table 5).

Discussion
This study provided a preliminary instrument to measure the critical thinking dispositions of Chinese medical college students and presented its psychometric properties. The critical thinking dispositions of Chinese medical college students include Open-mindedness, Systematicity/Analyticity and Truth-seeking. Open-mindedness targets the openness to different viewpoints and possibilities before making a decision. Systematicity/Analyticity involves values such as fairness and truth and the skills to strive for sound and unbiased judgements. Truth-seeking refers to the enthusiasm for true knowledge and active engagement in thinking. The present study revealed that the Chinese version of the CDTI-CM for medical college students showed acceptable psychometric properties.

Comparison among different versions of inventories for critical thinking dispositions
Since non-cognitive factors may have a great impact on participants' critical thinking, such as culture and motivation [41], we compared the psychometric properties of the CTDI-CM with the Chinese CCTDI (California Critical Thinking Dispositions Inventory) [11] and the CTDI-CV (Critical Thinking Disposition Inventory--Chinese Version) [10], which were developed to be  conceptually or semantically equivalent to the original CCTDI, respectively. While the Chinese CCTDI verified construct validity for the Truth-seeking, Open-mindedness, Systematicity and Maturity subscales [11], the content validity (alpha = 0.34) of Open-mindedness for the Chinese CCTDI was less satisfactory than the current study (alpha = 0.86). Additionally, the criterion validity analysis revealed that Open-mindedness on the CTDI-CV was irrelevant to the three factors of the CTDI-CM. Thus, the content validity of the CTDI-CM is different from that of the CTDI-CV. Open-mindedness (CTDI-CM) means being open to divergent views and prudent in decision making and, most importantly, not being submissive to authority. By contrast, Open-mindedness (CTDI-CV) addresses the tolerance of divergent world views/cultures and readiness to monitor one's own cognitive bias. The attitude towards authorities may differentiate the content of the two measurements. Indirect evidence showed that Chinese undergraduate nursing students are not as open-minded as their American student counterparts [42] and have ambivalent attitudes towards this disposition [43]. Chinese students, as obedient learners, may be more submissive to their teachers and dependent on rules, which may hinder their willingness to be open-minded [44]. Consequently, the inclination towards open-mindedness may lead to more solid decision-related reasoning and prevent nurses from making medical errors when they implement doctors' clinical decisions. Therefore, the content of the Open-mindedness factor of the CTDI-CM may be more suitable to detect the potential inclination of these individuals. Furthermore, the results provided only modest support for the convergent validity of the CTDI-CM with the CTDI-CV. (1) Open-mindedness on the CTDI-CM is significantly correlated with Maturity on the CTDI-CV. A plausible explanation is that open-mindedness may depend on self-reflection, which shows different developmental trajectories between young adults of different cultures. However, the developmental characteristics of Open-mindedness merit further exploration. (2) Systematicity/Analyticity and Truth-seeking on the CTDI-CM were positively related to self-confidence on the CTDI-CV and negatively related to Inquisitiveness on the CTDI-CV. The latter result may reflect the different emphases of Inquisitiveness (CTDI-CV) and Truth-seeking (CTDI-CM). Inquisitiveness represents eagerness to explore the unknown and interest in mechanisms behind phenomena, while Truth-seeking and Systematicity/Analyticity measure cognitive operations following informal and formal logical rules. Therefore, the motivation aspect of CT dispositions was less emphasized in the CTDI-CM than in the CTDI-CV, which explains the inverse relationship between the two groups of factors. Chinese philosophy and Confucius' teaching emphasize thinking as reflection in the context of relationships and identification with the interests of the whole [44], which may help to explain the negative relationship between   Inquisitiveness (CTDI-CV) and Systematicity/Analyticity (CTDI-CM).
Compared with a previous study [11], the test-retest analysis of the CTDI for medical students yielded more stable results across two assessment occasions (2 weeks apart) in the current study. All correlations were statistically significant, ranging from 0.808 to 0.965 with an overall correlation of 0.881. In addition, the results supported the internal consistency reliability of the Chinese version of the CTDI for medical students, which performed better than the two Chinese versions of the CCTDI [10,11]. These results confirm the current inventory as a reliable instrument for measuring the critical thinking disposition.

The predictors of CT dispositions
In health care, medicine and nursing may both require high critical thinking dispositions, which can lead to increased quality of care and better treatment outcomes. Our results suggest that medical students performed better on Open-mindedness and Systematicity/Analyticity than nursing students, which contrasts with previous findings. A survey conducted with the Chinese version of the CTDI (CTDI-CV) indicated that general performance for critical thinking ability in medicine and nursing was positive (overall score > 280) [10]. Dispositional differences using the CCTDI among several majors [(practice disciplines, i.e., nursing, education, business) and nonpractice disciplines (i.e., English, history, psychology)] were found in a previous study, with nursing students achieving among the highest scores [45]. Another study found that the average scores for the CTDI-CV and Analyticity in nursing were higher than those of medical students [46]. Due to the imbalance of the sample size of nursing versus medical students, further studies are needed to explore the differences in CT dispositions among majors.

Limitations
There are limitations of the current study that await further exploration. First, direct comparison of three Chinese versions of the critical thinking disposition inventory should be considered. Furthermore, due to the limitations of time and resources, we did not obtain external measures of the constructs, such as academic or professional performance, which should be addressed in the future to improve the criterion validity. Second, like most studies on critical thinking disposition, the current study was descriptive without analysing the causes of different critical thinking dispositions across cultures, such as teaching and learning strategies. Further studies may utilize active learning approaches such as problem-based learning (PBL) [47,48] and intervention programmes that target motivation components, such as self-awareness and mindfulness [9], to confirm the effectiveness and validity of the CDTI-CM. Third, we mainly focused on demographic variables that may affect the critical thinking disposition. More studies are necessary to broaden the understanding of other relevant factors (e.g., motivational and cognitive variables) of the CT disposition. Finally, the self-reported disposition may be subject to demand characteristics and social desirability, which are common to CT disposition scales. Future studies require the development of more reliable tests, such as behavioural and cognitive tasks (e.g., a cognitive reflection test, which can measure Analyticity), and comparison of these different measures of critical thinking.

Conclusions
The current study developed a questionnaire (CTDI-CM) to evaluate the critical thinking dispositions of Chinese medical students. The CTDI-CM includes three factors, Open-mindedness, Systematicity/Analyticity and Truth-seeking, which measure the motivation and cognitive Note: N = 305. * P < 0.05, ** P < 0.001 components of the CT dispositions of medical college students. The CTDI-CM was confirmed to be a reliable and valid CT measurement. Age, gender and major were significant predictors of the CT dispositions of Chinese medical college students.