Establishment and validation a relapse prediction model for bipolar disorder

1 Introduction

Bipolar Disorder (BD) is a mental disorder characterized by recurrent manic and depressive symptoms (1). BD is often associated with drug abuse, anxiety, suicide, and other diseases. The proportion of comorbidity is high, seriously impacting patient health and social stability (2, 3). Even if patients with BD, especially hospitalized patients, receive standardized treatment, the disease is characterized by recurrent episodes of mania, depression, or mixed states (4),which places enormous pressure on families and healthcare providers. The treatment effect in patients with BD is often difficult to maintain; therefore, the main clinical feature is symptom recurrence (5). Understanding the long-term response of patients to treatment and the predictive factors for disease recurrence helps clinicians and patients monitor treatment effectiveness and reduces direct and indirect treatment costs. Studies have reported that the annual recurrence rate of patients with BD varies from 39% to 52% during different treatment and maintenance plans (6). Therefore, the prevention and treatment of symptom recurrence are important for disease treatment.

Although the relapse rate in patients with BD is high, there is still a lack of effective clinical prediction models to predict the risk of relapse. However, developing nomogram can effectively help solve this problem; as it integrates several important factors and predicts whether the condition in these patients will relapse after active standardized treatment (7). Therefore, we conducted a retrospective study on patients with BD who received standardized treatment, collected information such as demographic characteristics and general clinical characteristics of patients, constructed and verified a risk prediction model of relapse within 1 year in patients with BD, and predicted the relapse risk of individual patients, to provide a certain reference value for clinical treatment response and prognosis judgment.

2 Materials and methods2.1 Patients

In this study, 303 patients with bipolar disorder were retrospectively collected from March 2021 to May 2022 at Anhui Mental Health Centre. At the same time, we also collected 81 patients who met the enrolment criteria at the Fourth People’s Hospital of Wuhu City. Patients were followed up for 1 year after reaching the recovery standard after treatment.

2.1.1 Inclusion criteria

(1) BD compliance with the International Classification of Diseases (ICD-10); (2) Unilateral seizures; (3) Age: 14-65 years; (4) A Hamilton Depression Scale (HAMD) (8)score or the Bech Rafaelsen Mania Rating Scale (BRMS) score on the Mania Rating Scale of >75% after treatment with administration of mood stabilizers and antipsychotic drugs; (5) A follow-up period of 1 year; (6) Presence of at least one informant who understands the patient’s situation;

2.1.2 Exclusion criteria

(1) Patients with organic brain lesions or a history of major brain trauma; (2) Neurological disorders; (3) Concomitant conditions, such as schizophrenia and vascular dementia; (4) Illiteracy (inability to complete the questionnaire); (5) Visual, auditory, reading, and writing impairments; (6) Missing visit.

BD relapse was defined as: (1) Patient visit or hospitalization because of mood disorders; (2) Patients met the ICD-10 criteria for manic, hypomanic, or depressive episodes. Patients were divided into relapse and non-relapse groups based on whether they relapsed within 1 year of effective treatment.

The study was approved by the ethical committees of the Anhui Mental Health Centre (approval number: HFSY-IRB-YJ-LWTG-ZXQ( (2023-061-01)) and the Wuhu Fourth People’s Hospital (approval number:[2021]-KY-17).

2.2 Predictive variables and follow-up outcomes

This study collected patient-related information from patient’s medical records and questionnaires. Demographic characteristics and clinical variables including age, sex, duration of illness, and medications taken by the patients were assessed. The Pittsburgh Sleep Quality Index (PSQI) (9) and Social Disability Screening Schedule (SDSS) (10) scales were used to evaluate patients’ social function and sleep at discharge. In addition, we also collected multidimensional characteristics related to patients’ general demographic characteristics, past medical history, and treatment.

Meanwhile, we checked patients’ medical records or telephone follow-ups to investigate the recurrence of BD. We defined a positive outcome based on the recurrence of symptoms within 1 year of discharge for inpatients with BD and a negative outcome based on the lack of symptom recurrence.

2.3 Statistical method

In this study, we analyzed the data using SPSS 26.0 and R 4.3.1. The Kolmogorov-Smirnov test was used to determine the normal distribution of continuous variables, and the double sample t-test was used for normal distribution data, shown in the form of mean ± standard deviation. Non-parametric data were compared between groups using the Mann-Whitney U test and are presented as quartiles (P25–P75). For categorical variables, the chi-square test or Fisher’s exact test was employed to compare groups, and the results were expressed as frequency (%). This study used univariate regression analysis to analyze the predictive variables. To eliminate the impact of multicollinearity on the model, we perform covariance screening when including variables in the model. Variables with a Variance Inflation Factor (VIF) greater than 5 are addressed accordingly. The binary logistic regression included variables with p <0.05, and the regression prediction model was established and transformed into a nomogram model. A calibration curve, receiver operating characteristic (ROC) curve, decision curve analysis (DCA) curve and C-index were used to evaluate and verify the nomogram model. This study defined a significant difference as a two-tailed p < 0.05.

3 Results3.1 Clinical characteristics

Table 1 demonstrates the comparison of the relevant feature information between the training set and the external validation set. The training set included a total of 303 patients, of which 139 were female (45.9%) and 164 were male (54.1%). A total of 96 (31.7%) patients experienced recurrence and 207 (68.3%) did not. A total of 44 (54.3%) females and 37 (45.7%) males were included in the external validation center. A total of 38 (46.9%) patients in the external validation set experienced recurrence and 43 (53.1%) did not.

Table 1. Demographics and clinical characteristics of study in the training and validation set.

Table 2 demonstrates the comparison between the baseline characteristics of the relapse and non-relapse groups of patients with BD in the training set. In the relapse group, the duration of disease was significantly higher than in the non-relapse group (10.313 ± 8.68 vs. 7.784 ± 6.994, p=0.007). Age at onset was significantly lower in the relapse group than in the non-relapse group (22.19 ± 7.32 vs 25.55 ± 9.556, p=0.002). The number of previous episodes was also significantly higher in the relapse group than in the non-relapse group (4.77 ± 3.922 vs. 2.92 ± 2.209, p=0.000). The number of years of education was also significantly higher in the non-relapse group than in the relapse group (11.81 ± 3.319 vs 10.89 ± 3.115, p=0.022). In the comparison of SDSS scores, it was significantly higher in the relapse group than in the non-relapse group (4.33 ± 2.405 VS 2.24 ± 1.582, p=0.000). In the comparison of PSQI scores, the relapse group was significantly higher than the non-relapse group (9.45 ± 3.808 VS 5.24 ± 2.543, p=0.000). In the comparison of the number of recurrence visits, the non-recurrence group was significantly higher than the recurrence group (7.03 ± 4.463 VS 3.81 ± 3.203, p=0.000). In the relapse group, the percentage of patients with suicidal behavior was significantly higher than in the non-relapse group (24.0% vs 10.1%, p=0.001). In addition, the proportion of ECT was significantly higher in the non-relapse group than in the relapse group (35.7% vs 24.0%, p=0.041).

Table 2. Comparison of demographic and clinical characteristics of the relapse and no-Relapse groups of BD patients in the training set.

3.2 Results of univariate regression analysis

The results of univariate regression analysis showed that disease duration (odds ratio [OR]: 1.042, p = 0.009), age at onset (OR: 0.953, p = 0.003), number of previous episodes (OR: 1.233, p = 0.03), years of education(OR: 0.915, p = 0.023), SDSS score (OR: 1.75, p = 0.000), PSQI score(OR: 1.499, p = 0.000), Number of visits(OR: 0.768, p = 0.000), working status(OR: 0.367, p = 0.000), housing conditions(OR: 2.321, p = 0.049), mood stabilizer(p = 0.000), suicidal behaviors(OR: 2.791, p = 0.002) and ECT(OR: 0.566, p = 0.042) were the risk factors for relapse in patients with BD, with statistically significant differences (all p < 0.05) (Table 3).

Table 3. Univariable regression analysis of the various factors and relapse outcome.

3.3 Multivariate regression analysis results

Multivariate regression analyses showed that the number of previous episodes (OR: 1.38, p = 0.000), Number of visits (OR: 0.806, p = 0.000), SDSS score (OR: 1.303, p = 0.029), PSQI scores (OR: 1.476, p = 0.000), suicidal behaviors (OR: 5.54, p = 0.003) and ECT (OR: 0.382, p =0.036) were statistically significant independent risk factors for relapse within 1 year in patients with BD (all p < 0.05) (Table 4).

Table 4. Multivariate regression analysis of the various factors and relapse outcome.

3.4 Development of the nomogram model

Based on the logistic regression analyses described above, we developed a nomogram model to predict the risk of recurrence within 1 year in patients with BD (Figure 1). Each clinical factor corresponds to a specific score, and a straight- line point axis was drawn to calculate the total score, which corresponds to a high risk of BD relapse and vice versa.

Figure 1. Nomogram for predicting the disease relapse of BD patients. For MECT therapy, 0=no, 1=yes. For attempted suicide, 0=no, 1=yes. SDSS, social disability screening schedule; PSQI, pittsburgh sleep quality index; MECT, modified electroconvulsive therapy. *, **, ***: indicates statistical significance in multivariate regression analysis.

3.5 Validation of the nomogram model

The nomogram developed in this study have high predictive performance. Calibration curves were plotted from the predicted values of the nomogram model to the actual values. The results in Figures 2, 3 show that the agreement between the predicted and actual values of the nomogram model is high for both the training set and the validation set. In addition, Figure 4 shows that the clinical prediction curves (DCA) have high clinical applicability on both the training set and the validation set. As shown in Figure 5, the area under the ROC curve suggests that the nomogram has high classification and prediction ability in the training set (AUC: 0.924, 95% CI: 0.888 - 0.951). Figure 6 suggests that this nomogram model also has good generalization ability on the external test set (AUC:0.741, 95% CI:0.632 - 0.832).

Figure 2. The calibration curves for the nomogram in the training set.

Figure 3. The calibration curves for the nomogram in the validation set.

Figure 4. Decision curves for the clinical nomogram in the training and validation set.

Figure 5. Receiver operating characteristic curves to evaluate the discriminating capability of the nomogram in the training set.

Figure 6. Receiver operating characteristic curves to evaluate the discriminating capability of the nomogram in the validation set.

4 Discussion

The high prevalence and relapse rate of BD place great pressure on the rehabilitation and treatment of patients (11). Therefore, establishing a risk prediction model for short-term recurrence is significance for early intervention and treatment guidance in patients with BD. In this study, 384 patients with BD were retrospectively included to analyze the risk factors that may affect short-term recurrence. Based on binary logistic regression analysis, this study found that the number of previous episodes, SDSS score, PSQI score, number of visits, suicidal behaviors and ECT were independent risk factors for short-term relapse in patients with this disease. Nomograms created by combining the above risk factors have a high predictive efficacy. Meanwhile, the performance of the external validation set demonstrated that the nomogram has a good panchromatic ability. The results showed that the model had a high predictive accuracy and clinical application value.

Recurrent relapses of symptoms in patients with BD can further deteriorate the condition and lead to poor prognosis (12), highlighting the importance of identifying factors associated with an increased risk of relapse in patients. Patients with BD have a large heterogeneity in the disease course and symptoms, and it is crucial to develop and validate relapse prediction models and guide individualized treatment. The results of multivariate logistic regression analysis showed that t the number of previous episodes, SDSS score, PSQI score, number of visits, suicidal behaviors and ECT were independent risk factors for relapse within 1 year.

Previous studies have shown that the number of previous episodes of a patient’s symptoms is an important predictor of relapse, which is consistent with our findings (13). On the one hand, prolonged depressive or manic episodes may disrupt the balance of neurotransmitters in the brain, such as dopamine, norepinephrine, and serotonin, making patients more susceptible to recurrent mood swings (14, 15). On the other hand, frequent symptomatic episodes can lead to cognitive decline, particularly impairments in memory, attention, and executive functioning, and may also make it more difficult for patients to effectively recognize and manage early signs of symptoms, thereby increasing the risk of relapse (16). Moreover, the results of this study indicate that the higher the number of previous episodes, the higher the risk of recurrence, and the higher the number of episodes, the more severe and complex the patient’s condition is, which in turn causes the patient’s disease to prolong, thus increasing the patient’s risk of recurrence (17). Our study also found that the number of outpatient visits was a protective factor against relapse in patients with BD. We found that the higher the number of outpatient follow-up visits, the lower the risk of relapse. Patients seek timely medical help based on dynamic changes in clinical symptoms; in contrast, doctors provide standardized medical management, suggesting the importance of patient compliance and standardized outpatient management. In addition, this study found that the higher the PSQI score, the higher the risk of recurrence; the worse the sleep quality, the higher the risk of BD recurrence. Some studies have found that sleep disorders may be precursors of BD (18). Some studies have confirmed that sleep disorders can affect circadian rhythms, change brain electrical activity during sleep, and impair patients’ emotional processing ability (19). Therefore, early maintenance of a good sleep state is of great value for the prognosis of patients with BD. Similarly, as SDSS scores increase, the risk of relapse increases in patients with BD. The SDSS scale is an instrument that responds to social and occupational functioning, and as the SDSS score increases, the poorer the social functioning of the respondent. Previous studies have shown that social dysfunction significantly increases the risk of relapse in patients with bipolar disorder (20).

The present study also found that the presence of suicidal behavior in patients significantly increased the risk of recurrence. It is well known that suicidal behavior usually responds to a more severe stage of illness in people with bipolar disorder, which significantly increases the risk of fluctuations and relapses (21). Previous studies have shown that suicidal behavior is closely associated with damage to the frontal lobe, which plays a critical role in emotion regulation, decision-making, and impulse inhibition (22). A reduction in frontal lobe function in patients exhibiting suicidal behavior may impair their ability to regulate negative emotions and control impulsive actions, thereby heightening the risk of relapse (23). This study also found that patients treated with ECT significantly reduced the risk of relapse in patients with bipolar disorder. ECT can rapidly relieve severe depressive and manic symptoms in patients with bipolar disorder, thereby stabilizing mood and reducing the likelihood of relapse (24). ECT is equally effective in patients with refractory BD, further stabilizing symptoms and thereby significantly reducing the risk of relapse (25).

However, even if patients with BD have the same or similar risk factors, the predicted risk of symptom recurrence is different. The combined prediction of multiple risk factors can improve predictive performance. Nomograms can combine multiple risk factors for joint prediction and have been widely used in statistical analyses of various neurological disorders (26). However, nomograms are rarely used to assess the risk of BD recurrence. In this study, we recruited 384 patients with BD and constructed a prediction model for symptom recurrence within 1 year. According to the multivariate logistic regression analysis results, a line chart scoring system was constructed based on the previous episodes, SDSS score, PSQI score and suicidal behaviors, the number of visits and ECT. The score system has a high predictive efficiency, with a AUC of 0.924 (95% CI: 0.888 - 0.951). Moreover, in order to test the generalisation ability of this nomogram model in this study, the prediction performance of the external test set was kept high (AUC:0.741, 95% CI:0.632 - 0.832). In addition, the calibration curve suggested a high degree of consistency between the predicted and actual values. The model aligns with actual clinical scenarios; in clinical settings, doctors can use column-line graphs to visualize the risk of relapse in patients with BD. In addition, combining the individual specificity of patients for early and comprehensive interventions has greater clinical value for the prognosis of patients with BD.

This study had some limitations. Firstly, this was a retrospective study. In the future, we will conduct a prospective cohort study to improve the reliability and persuasiveness of the study. Secondly, we established a prediction model based on demographic and clinical characteristics, and the input characteristics were relatively simple. In the future, we will further develop multigroup, multi-model, and multidimensional models to improve the model’s accuracy and clinical practice value. Finally, we did not use more cutting-edge AI techniques applied in this study, such as machine learning, deep learning, etc. In the future, we will actively explore the performance of high-performance AI techniques applied in the occurrence and development of BD. Finally, the relatively small sample size may limit the performance and generalizability of the model. In the future, we plan to conduct a multicenter study to expand the sample size and further validate our findings.

5 Conclusion

This study found that the number of previous episodes, SDSS score, PSQI score and suicidal behaviors were independent risk factors for relapse of BD, while the number of visits and ECT were protective factor. Based on these factors, we developed and validated a nomogram for predicting relapse in patients with BD; that has clinical reference values.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material. Further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by the Anhui Mental Health Centre (approval number: HFSY-IRB-YJ-LWTG-ZXQ([2023-061-01]) and the Wuhu Fourth People’s Hospital (approval number:[2021]-KY-17). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Author contributions

XZ: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Software, Supervision, Validation, Writing – original draft, Writing – review & editing. MW: Data curation, Validation, Writing – review & editing. DW: Data curation, Validation, Writing – review & editing. LW: Data curation, Validation, Writing – review & editing. WX: Data curation, Funding acquisition, Investigation, Methodology, Supervision, Validation, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by National Clinical Key Specialty Construction Project of China.

Acknowledgments

The authors would like to thank all the individuals who offered help and advice on this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Findling RL, Stepanova E, Youngstrom EA, Young AS. Progress in diagnosis and treatment of bipolar disorder among children and adolescents: An international perspective. Evidence-Based Ment Health. (2018) 21:177–81. doi: 10.1136/eb-2018-102912

PubMed Abstract | Crossref Full Text | Google Scholar

2. Fenn HH, Bauer MS, Altshuler L, Evans DR, Williford WO, Kilbourne AM, et al. Medical comorbidity and health-related quality of life in bipolar disorder across the adult age span. J Affect Disord. (2005) 86:47–60. doi: 10.1016/j.jad.2004.12.006

PubMed Abstract | Crossref Full Text | Google Scholar

3. Gamage N, Senanayake S, Kumbukage M, Mendis J, Jayasekara A. The prevalence of anxiety and its association with the quality of life and illness severity among bipolar affective disorder patients in a developing country. Asian J Psychiatry. (2020) 52:102044. doi: 10.1016/j.ajp.2020.102044

PubMed Abstract | Crossref Full Text | Google Scholar

4. Ortiz A, Hintze A, Burnett R, Gonzalez-Torres C, Unger S, Yang D, et al. Identifying patient-specific behaviors to understand illness trajectories and predict relapses in bipolar disorder using passive sensing and deep anomaly detection: protocol for a contactless cohort study. BMC Psychiatry. (2022) 22:1–8. doi: 10.1186/s12888-022-03923-1

PubMed Abstract | Crossref Full Text | Google Scholar

5. Birmaher B, Merranko JA, Gill MK, Hafeman D, Keller M. Predicting personalized risk of mood recurrences inYouths and young adults with bipolar spectrumDisorder. J Am Acad Child Adolesc Psychiatry. (2020) 59:1156–64. doi: 10.1016/j.jaac.2019.12.005

PubMed Abstract | Crossref Full Text | Google Scholar

6. Tohen M, Zarate CA Jr., Hennen J, Khalsa H-MK, Strakowski SM, Gebre-Medhin P, et al. The McLean-Harvard first-episode mania study: prediction of recovery and first recurrence. Am J Psychiatry. (2003) 160:2099–107. doi: 10.1176/appi.ajp.160.12.2099