Establishment and validation of a nomogram for dropout intention in Chinese early year medical undergraduates

Abstract

Background

The attrition rate of Chinese medical students is high. This study utilizes a nomogram technique to develop a predictive model for dropout intention among Chinese medical undergraduates based on 19 individual and work-related characteristics.

Method

A repeated cross-sectional study was conducted, enrolling 3536 medical undergraduates in T1 (August 2020-April 2021) and 969 participants in T2 (October 2022) through snowball sampling. Demographics (age, sex, study phase, income, relationship status, history of mental illness) and mental health factors (including depression, anxiety, stress, burnout, alcohol use disorder, sleepiness, quality of life, fatigue, history of suicidal attempts (SA), and somatic symptoms), as well as work-related variables (career choice regret and reasons, workplace violence experience, and overall satisfaction with the Chinese healthcare environment), were gathered via questionnaires. Data from T1 was split into a training cohort and an internal validation cohort, while T2 data served as an external validation cohort. The nomogram’s performance was evaluated for discrimination, calibration, clinical applicability, and generalization using receiver operating characteristic curves (ROC), area under the curve (AUC), calibration curves, and decision curve analysis (DCA).

Result

From 19 individual and work-related factors, five were identified as significant predictors for the construction of the nomogram: history of SA, career choice regret, experience of workplace violence, depressive symptoms, and burnout. The AUC values for the training, internal validation, and external validation cohorts were 0.762, 0.761, and 0.817, respectively. The nomogram demonstrated reliable prediction and discrimination, with adequate calibration and generalization across both the training and validation cohorts.

Conclusion

This nomogram exhibits reasonable accuracy in foreseeing dropout intentions among Chinese medical undergraduates. It could guide colleges, hospitals, and policymakers in pinpointing students at risk, thus informing targeted interventions. Addressing underlying factors such as depressive symptoms, burnout, career choice regret, and workplace violence may help reduce the attrition of medical undergraduates.

Trial registration

This is an observational study. There is no Clinical Trial Number associated with this manuscript.

Introduction

The global shortage of healthcare workers is a pervasive problem [1, 2], and the attrition of medical students has undeniably exacerbated this crisis. A meta-analysis has shown an average 9.1% attrition rate among global medical students [3], with a more recent study revealing even higher rates in low- and middle-income countries—up to 16% in Pakistan and 20.8% in Nigeria [4]. In China, a recent in 2019 recorded a 13.04% dropout rate for Chinese medical students [5].

Dropout from high school has been conceptualized as a gradual process that starts with a perception of unsuitability and thoughts of dropping out, culminating in the final decision to actually dropout [6]. An empirical longitudinal study suggests that dropout intentions and actual dropouts share similar risk and protective factors [7]. This study also identifies a positive association between dropout intentions and actual dropouts [7]. Taken together, these findings suggest that dropout intention could be early stage of the dropout process, and identifying and targeting dropout intention might contribute to the prevention of actual dropout. In China, the prevalence rate of dropout intention among medical students is high [5, 8,9,10]. For example, Cai et al. reported that nearly three-fifths of Chinese medical undergraduates expressed strong turnover intentions [10], a rate substantially higher than those in other countries such as U.S., Czech Republic, and Thailand (roughly 20%) [11,12,13]. These alarming statistics highlight an urgent need to explore potential risk factors and identify students at risk of dropping out.

Although mental health issues such as burnout, depression, stress, and substance use have been recognized as significant predictors for dropout intention in medical students worldwide [12,13,14,15], their interrelation has been comparatively overlooked in the Chinese context. Previous studies on Chinese medical students have primarily concentrated on work-related factors like violence, working hours, and prestige tied to medical careers [9, 10]. A more comprehensive approach, integrating both individual and work-related elements, is essential to fully comprehend the potential risk factors contributing to attrition.

A nomogram is a graphical representation of a predictive function stemming from a logistic regression model, facilitating the quantification of individual probabilities for adverse outcomes by synthesizing various variables [16]. Unlike conventional statistical techniques, a nomogram offers an accessible, visually intuitive method to evaluate risk, rendering it valuable to educators and policymakers. Recent studies have increasingly utilized nomograms to ascertain risks of adverse events in students, such as suicidality, anxiety, and sleep disturbances [17,18,19]. However, no known prior study has established a nomogram for medical students’ dropout intention.

Therefore, this study aims to amalgamate an extensive range of individual and work-related factors to construct a nomogram for dropout intention among Chinese medical undergraduates. The application of this model holds potential for the early identification of students at risk, thereby contributing to a broader understanding and prevention of attrition within medical education.

Method

Study procedure and participants

This study utilized a repeated web-based cross-sectional design, conducted in two phases. The initial survey (T1) was carried out from August 2020 to April 2021, followed by a second survey (T2) in October 2022. A detailed description of the study procedure and measurements in T1 can be found in previous publication [8].

Briefly, medical undergraduates from various Chinese colleges were recruited using snowball sampling. The online questionnaire, hosted on WJX (a popular questionnaire website), was disseminated through WeChat, a prominent Chinese social media platform. Participants were encouraged to share the survey link with their peers. Eligibility criteria included being above 18 years of age and enrolled as a medical undergraduate in a Chinese college.

Participation was anonymous and voluntary, and students had the right to withdraw at any stage. Completion of all questions was required to submit the questionnaire due to the settings of WJX. Informed consent was obtained from all participants, who subsequently received a summary report of their mental health status. Ethical approval for the study was granted by the ethics committee of the Second Xiangya Hospital of Central South University (JY20200326).

Measurements

The online questionnaire consisted of three sections. All questionnaires were administrated in Chinese. Section 1 comprised self-designed questionnaires to collect basic demographic information, such as monthly income, age, sex, grade, relationship status (single/partnered), and history of psychiatric illness. Based on their grades, students were classified into preclinical (grades 1 to 3) and clinical students (grades 4 and 5). A trap question, “What is the Chinese national day?” was included to filter out invalid responses. Participants who answered this question incorrectly were excluded from the final analysis.

Section 2 encompassed a series of well-established questionnaires to comprehensively measure the mental health of medical undergraduates. The Chinese version of the following questionnaires were administrated, including the nine-item Patient Health Questionnaire (PHQ-9) for depressive symptoms, the seven-item Generalized Anxiety Disorder Scale (GAD-7) for anxiety symptoms, the 20-item Learning Burnout Scale (LBS) for learning burnout, the Epworth Sleepiness Scale (ESS) for sleepiness, the 10-item Perceived Stress Scale (PSS) for high perceived stress, the 15-item Patient Health Questionnaire (PHQ-15) for somatic symptoms, and the Alcohol Use Disorders Identification Test Concise (AUDIT-C) for alcohol abuse/dependence. All of these scales demonstrated excellent reliability and validity and were widely used in the Chinese context [20,21,22,23,24,25,26]. The cutoff points and Cronbach’s alpha for each questionnaire in the two surveys are presented in Table 1.

Table 1 Measurement tools

In addition to these mental distress measurements, a standardized linear analog was adopted to assess the quality of life (QOL) and fatigue. Participants were asked to rate their quality of life and fatigue within the past week on a standardized linear analog scale, with scores ranging from 0 (representing the poorest possible QOL/fatigue) to 10 (indicating the highest possible QOL/fatigue) [27]. Following our previous study [8], we operationalized low QOL and high fatigue as scores equal to or exceeding a threshold of 0.5 standard deviations below the mean score.

We also included a single question to capture lifetime suicidal attempts (SA). In T1, a yes or no question was used (“Have you ever attempted to kill yourself during your lifetime?). Those who affirmatively responded to this question were considered as having SA. This single item has been widely used in epidemiological surveys [28]. In T2, we used the first item of the Suicidal Behaviors Questionnaire-Revised. Participants who chose “I have attempted to kill myself but did not want to die” and “I have attempted to kill myself, and really hoped to die” were identified as having SA.

In Section 3, we evaluated dropout intention and several work-related variables, including career choice regret, experience of workplace violence, and overall satisfaction with the Chinese medical environment. Following a previous study [8], participants were first asked if they had contemplated leaving medical school or shifting to a non-medical major within the last year. A positive response classified them as having dropout intention. The seriousness of this intention was further assessed using a five-point Likert scale, ranging from “not serious” to “extremely serious.”

We assessed career choice regret via two items. All medical students were questioned if they have regretted medicine learning within the last year. We further questioned them about the reason for their career choice regret using a multiple-choice question. The options in T1 included “low income,” “poor doctor-patient relationships,” “long study period,” “overwork,” “intense competition,” “experience of violence,” “high expectations of patients,” and “other causes.” [8]. In T2, we additionally added several options including “high research pressure”, “high graduation pressure”, “too many shifts during holidays”, “low respect from the society”, “high pressure from standardized training”, and “few career development opportunities” based on the comments received in T1.

The workplace violence was measured with a single question: “Have you experienced workplace violence inflicted by patients or their family members/visitors?” The options included “No experience,” “Experiencing verbal violence,” “Experiencing physical violence,” and “Experiencing both verbal and physical violence.” We binarized this variable into “with violence” and “without violence” for the subsequent analysis.

We used a 5-point Likert scale to evaluate medical students’ overall satisfaction with the Chinese healthcare environment. The scores ranged from 1 (very dissatisfied) to 5 (very satisfied). We categorized those who reported dissatisfaction or very dissatisfaction with the Chinese medical environment into the non-satisfaction group.

Statistical analysis

All statistical analyses were performed using the R software (version 4.2.2), along with MSTATA software (www.mstata.com). The data from phase T1 was randomly partitioned into training and internal validation cohorts at a ratio of 7:3, with the data from phase T2 serving as an external validation cohort. The training cohort was deployed to identify potential predictors and construct the model, while the internal and external validation cohorts were utilized to evaluate its performance. Continuous data were presented as median and quartiles and categorized variables were displayed as frequency and percentage. Chi-square tests or rank-sum tests were employed to determine the factors associated with dropout intention. Tests were two-tailed, with p below 0.05 suggesting statistical significance.

In the training cohort, the least absolute shrinkage and selection operator (LASSO) regression analysis was used for multivariate analysis to screen the independent risk factors. Factors displaying statistically significant association with dropout intention in the univariate analysis were included in the LASSO regression model. The most regularized and parsimonious model, with a tenfold cross-validated error within one standard error of the minimum, was selected. The selected factors in LASSO regression were then included in the multivariate logistic regression model using the enter method to build the model and nomogram.

We tested the performance of the nomogram in the training and validation cohort. We performed the receiver operating characteristic (ROC) curve and calculated the area under the ROC curve (AUC) values. An AUC value greater than 0.7 indicated reasonable discrimination ability [29]. We evaluate the model’s calibration with a calibration curve plot created using a 1000-bootstrap resampling procedure. Additionally, we conducted a decision curve analysis (DCA) to determine the net benefit threshold of prediction. Additionally, we performed separate ROC and AUC analyses for preclinical and clinical students to assess the model’s performance.

Result

Sample characteristics

In the first survey (T1), a cohort of 3,648 undergraduates was recruited. Of these, 112 responses were deemed invalid due to failure in answering the trap question correctly, yielding a valid sample size of 3,536 participants (effective rate:96.9%). In the second survey (T2), 1,002 undergraduates were recruited. Here, 33 responses were invalidated, culminating in a final sample size of 969 participants (effective rate: 96.7%). The demographic information and mental health status of the participants are summarized in Table 2.

Table 2 Sample characteristics

Of the 4505 medical undergraduates, approximately 37% (n = 1671) reported intentions to drop out. The incidence of various mental distress was as follows: depressive symptoms were identified in 18% of participants, anxiety symptoms in 9.1%, somatic symptoms in 28%, alcohol use disorder in 8.5%, sleepiness in 40%, high perceived stress in 28%, high fatigue in 25%, low QOL in 21%, and burnout in 37%. Additionally, a small proportion reported a history of SA (9.8%) and previous diagnosis of mental disorder (3.2%). About one-tenth of the participants reported victimization of workplace violence and expressed dissatisfaction with the Chinese healthcare environment.

Career choice regret was expressed by 37% of the participants. The most frequently reported reasons for this regret during T1 included “overwork” (n = 881), “long study period” (n = 731), “intense competition” (n = 705), “poor doctor-patient relationship” (n = 694), and “high risk of violence victimization” (n = 694). In the subsequent T2 survey, the prevalent complaints shifted to “intense competition” (n = 253), “overwork” (n = 238), “long study period” (n = 203), “low income” (n = 178), “high graduation pressure” (n = 163), and “poor doctor-patient relationship” (n = 161).

Variable selection and prediction model establishment

Table 3 compared the demographic information, mental health, and work-related variables between participants with and without dropout intention in the training cohort. Dropout groups were more likely to be clinical undergraduates and female. They reported much higher risk of mental distress, including depressive symptoms, anxiety symptoms, somatic symptoms, high perceived stress, burnout, sleepiness, high fatigue, and low QOL. They were also more likely to report history of lifetime SA and mental illness. However, no significant difference was found in alcohol use disorder between the two groups. In addition, participants with dropout intention were more likely to be victims of workplace violence, report career choice regret, and were more dissatisfied with the Chinese medical environment.

Table 3 Factors associated with dropout intention in the training cohort

The original model included fifteen candidate predictors: history of SA, career choice regret, experience of workplace violence, dissatisfaction with the healthcare environment, history of mental illness, depressive symptoms, anxiety symptoms, sleepiness, high stress, high fatigue, low QOL, burnout, study phase, sex, and somatic symptoms. Utilizing the LASSO regression analysis, this was reduced to seven potential predictors in the training cohort (Fig. 1). The final multivariate logistic regression model for dropout intention encompassed these seven predictors: history of SA, career choice regret, violence experience, depressive symptoms, high perceived stress, low QOL, and learning burnout.

Fig. 1

Results of Lasso regression. a Lasso Regression Cross-Validation Plot; b Lasso Regression Coefficient Path Plot

Five predictors were independently associated with dropout intention in the final model: history of SA, career choice regret, workplace violence experience, depressive symptoms, and learning burnout (Table 4). This final logistic model was synthesized into a user-friendly nomogram, illustrated in Fig. 2. The total risk score represents a continuum, with higher scores indicating a greater likelihood of dropout intention. As an illustrative example, a medical undergraduate with no history of suicidal attempts or depressive symptoms, but who experienced workplace violence, learning burnout, and reported career choice regret, would obtain a score of approximately 153 (0 + 0 + 25 + 28 + 100). This would correspond to an estimated probability of dropout intention of roughly 75%.

Table 4 Logistic regression model for dropout intention

Fig. 2

Nomogram prediction model for dropout intention

Model performance evaluation

The AUC values of the nomogram model were 0.762 in the training cohort, 0.761 in the internal test cohort, and 0.817 in the external test cohort, which suggested reasonable accuracy of our model (Fig. 3). The calibration and DCA curves of the nomogram were presented in Figs. 4 and 5. The calibration plots demonstrate a good correlation between the observed and predicted dropout intention. The results showed that the original nomogram was still valid for use in the validation sets, and the calibration curve of this model was relatively close to the ideal curve, which indicates that the predicted results were consistent with the actual findings. The DCA curves suggested satisfactory clinical usefulness of our model to screen for students with dropout intention. The subgroup analysis revealed that the model exhibited nice accuracy among both preclinical and clinical students. For preclinical students, the AUC values were 0.763 in the training cohort, 0.758 in the internal cohort, and 0.831 in the external cohort. Among clinical students, the AUC values were 0.760 in the training cohort, 0.793 in the internal cohort, and 0.799 in the external cohort.

Fig. 3

ROC curves of the nomogram prediction model

Fig. 4

Calibration curve of the nomogram prediction mode. a Calibration curve of the nomogram prediction mode in the training cohort; b Calibration curve of the nomogram prediction mode in the internal validation cohort; c Calibration curve of the nomogram prediction mode in the external validation cohort

Fig. 5

Decision curve analysis of the nomogram. a Decision curve analysis of the nomogram in the training cohort; b Decision curve analysis of the nomogram in the internal validation cohort; c Decision curve analysis of the nomogram in the external validation cohort

Discussion

In this study, we have constructed the first nomogram to predict dropout intention among medical undergraduates. Out of 19 individual and work-related factors, five were selected as significant predictors: history of SA, career choice regret, experience of workplace violence, depressive symptoms, and burnout. The resulting nomogram demonstrated both satisfactory accuracy and calibration, contributing to the timely identification of medical undergraduates at high risk for dropout.

We found that 37% of the medical undergraduates reported dropout intention within the last year. The results were consistent with a recent study conducted by Yu et al., which also found that one-third of the medical undergraduates hoped to attrition from medicine [9]. The prevalence of dropout intention was lower than our previous findings in Chinese medical postgraduates (58%) using the same questionnaire [8]. Possible explanations could include differences in workload, expectations, mentoring support, and professional development opportunities between undergraduate and postgraduate levels.

Consistent with prior studies [14, 15, 30], our results demonstrated substantial association between dropout intention and mental health problems. Notably, burnout and depressive symptoms emerged as the independent predicting factors for dropout intention. This finding was in line with much of the previous literature on the positive association between burnout, depressive symptoms, and attrition of healthcare workers and students [13, 14, 31]. Burnout and depressive symptoms were highly prevalent and distressful, affecting around two-fifth of the global medical students [30]. Notably, our study demonstrated a relatively stable prevalence of burnout and depression in both study period (2020 and 2022). This finding could possibly reflect a persistent mental health challenge in Chinese medical students rather than a short-term response to the pandemic. Hence, the results warrant a more profound investigation into the causes that contribute to the enduring nature of these mental problems.

Furthermore, our study identified a history of suicidal attempts as another independent factor associated with dropout intention. Despite both suicidality and dropout being viewed as severe negative outcomes for medical students, scant research has explored their connection. This gap emphasizes the need for further study and increased attention to the mental health of medical students who express dropout intention.

Our results also highlight a substantial positive association between career choice regret and dropout intention, consistent with prior findings [8, 32]. Both surveys in our study identified “intense competition,” “overwork,” and “long study period” as the most frequent reasons for this regret. These findings resonate with recent reports linking prolonged training and heavy workload with decreased willingness to pursue medical careers after graduation [9]. Another study also showed that 92% of the medical undergraduates complained that Chinese doctors suffered from tremendous workload [10]. Interestingly, previous studies in Chinese medical postgraduates and physicians produced consistent results [8, 33,34,35,36], which also demonstrated that low income and heavy workload greatly contributed to turnover intention. The consistency in the results, across different stages of medical training and career, underscores the pervasive nature of these challenges. There is a critical need for systemic interventions by policymakers, educational institutions, and healthcare organizations, such as curricular reforms, workload regulations, and support measures to enhance well-being and job satisfaction.

Another salient factor contributing to dropout intention is the experience of workplace violence. Our findings corroborate previous research that frequent violence against healthcare workers significantly impacts medical students’ career choices and work intentions [37, 38]. Notably, one-tenth of the medical undergraduates experienced workplace violence from parents or their families or visitors. Although existing studies have documented the high risk of violence victimization among Chinese healthcare workers [39,40,41], there is a surprising lack of attention to this issue among medical undergraduates. To our knowledge, only one study have investigated the incidence of workplace violence experience among 157 Chinese medical students, which found 30.6% of them experienced violence [42]. The lack of existing literature called for further studies to explore the specific nature, underlying causes, and the extent of workplace violence experienced by Chinese medical undergraduates. Hospitals should be aware of the risk of violence experienced by medical undergraduates and provide timely interventions, such as improving security measures and providing counseling and support for those affected by workplace violence.

Our model, which included work-related variables, demonstrated nice accuracy in predicting dropout intentions among preclinical students. Over the past decade, Chinese medical education has introduced “early contact with clinics” into its curriculum, offering preclinical students early clinical exposure [43]. Consequently, preclinical students might experience notable work-related issues: in our study, 10.2% faced workplace violence, 35.2% regretted their career choice, and 9.3% were dissatisfied with the Chinese healthcare environment. These negative experiences were linked to higher dropout intentions. Our findings indicated that adverse work-related experiences might lead to attrition at the very early stages of medical education. Therefore, more attention should be paid to these factors to improve retention rates and support the well-being of preclinical students.

Our study possesses several notable strengths. First, to our knowledge, our study is the first to comprehensively evaluate the association between dropout intention and a wide array of mental health problems as well as work-related factors. Integrating both individual and work-related factors, our study offers a more thorough understanding of the risk factors for attrition among Chinese medical undergraduates. Second, our research pioneers the use of a nomogram to forecast dropout intentions among medical undergraduates. The nomogram’s primary advantage is its intuitiveness and user-friendliness, facilitating the integration of multiple predictive factors for tailored risk assessments. Validated in an independent cohort, the nomogram has proven both accurate and effective, thus offering a valuable tool for identifying students at risk of attrition. Third, the repeated cross-sectional study design confirmed a consistent high incidence of dropout intention and mental distress among medical undergraduates. A strong positive association between dropout intention and mental distress, particularly burnout and depressive symptoms was identified. This finding highlights the need for long-term monitoring and intervention for mental health problems in this population. Our findings could inform colleges, hospitals, and policymakers, enabling timely policy adjustments to reduce attrition rates among medical students. Specifically, interventions addressing heavy workload, extended training periods, low income, and violence against healthcare workers and students may be beneficial.

Several limitations should be recognized. First, the cross-sectional design constrains our ability to establish a causal relationship. Second, our sample was unbalanced, predominantly consisting of preclinical students at the early stage of medical training. This imbalance likely resulted from convenience sampling, as clinical students were busier with graduation and clinical work. The higher proportion of early-year students may have led to an underestimation of dropout intention in our sample. In contrast, our recent study of medical postgraduates found a much higher dropout intention (about 58%), indicating the potential influence of training stage on dropout intention of medical students [8]. While our models showed similar accuracy in identifying dropout intention in both preclinical and clinical students, studies with larger and more-balanced samples are needed to further assess prevalence and associated factors for dropout intention and to validate our prediction models. A third limitation pertains to the data collection for the initial survey, which occurred in the year following the pandemic. During this period, students experienced college lockdowns and a shift to online learning, potentially impacting dropout intentions. However, the validation of our model in a subsequent independent survey conducted two years later indicates a reasonable generalizability of our findings. Finally, while previous studies have established the substantial association between dropout intention and actual dropout [6, 7], assessing the incidence and risk factors for actual dropout might provide a better insight into the attrition of medical students.

Conclusion

In conclusion, our study developed a predictive nomogram for dropout intention among medical undergraduates, integrating five key factors: history of suicidal attempts, depressive symptoms, burnout, career choice regret, and workplace violence experience. This nomogram has demonstrated strong internal and external validation, performing well in terms of both discrimination and calibration. It could help identify medical undergraduates more likely to dropout, and provides a foundation for targeted interventions to support their retention.