Prediction model for suicide based on back propagation neural network and multilayer perceptron

Introduction

While world suicide rates are slightly on the rise (WHO, 2014), China is experiencing a tremendous decline in its suicide rate in recent decades (Jiang et al., 2018). Although the Chinese suicide rate has decreased recently, it is still a leading cause of death (Zhang et al., 2011a; Liu et al., 2017) and serious threat to health in China. The average suicide rate in China was 6.75/100,000 from 2012 to 2015 years (Liu et al., 2017). This represents a decrease of about 65% (with national rate of 7.21/100,000) in China (Jiang et al., 2018), although this varies by gender, age, and location (Chen et al., 2019).

To date, there have been several studies examining risk factors for suicide in China. Phillips et al. (2002) researched the risk factors for suicide in China through a national case-control psychological autopsy (PA) study. Sun et al. (2011) report an annual peak in suicide in the spring and early summer. Scholars have explored sociological reasons for the trend of declining suicide rate in China and found that the declining of psychological strain was the main reasons (Zhang et al., 2011a). Further, risk factors, coping skills, and mental disorders among rural youth suicide in China have been examined in recent years (Zhang et al., 2011b; Li and Zhang, 2012). Subjects at risk of suicidal behavior usually search the information of self-harm and suicidal behaviors on internet, particularly for the young adolescents. The relationship between suicide-rates and suicide-related search volumes by internet still have been reported (Solano et al., 2016).

Much of the previous literature has focused on risk factors and established prediction models using traditional statistical methods. However, because the suicide population is unique, the distribution of the variables is likely to be skewed and the relationships are likely to non-linear (Lyu and Zhang, 2019). But most previous literature has examined prediction models through logistic regression (Lyu and Zhang, 2019), which is not suitable for the data of special populations. However, the neural network model is more robust to identify complex non-linear relationships between variables.

Artificial Neural Network (ANN) has been applied in medical research fields (Jiang and Jianfeng, 2011; Marques et al., 2020; Chiang et al., 2021; Hooper et al., 2021; Yang and Yu, 2021). There has been little literature applying the neural network model to establish the predicted model to discriminate patient or not (Modai et al., 2002). As the amount of data continues to grow at an almost incomprehensible rate, machine learning algorithms is better suitable to the present-day era of big data and data science. There are many famous algorithms such as Neural Network, Logistic Regression, SVM, Multilayer Perceptron, Naive Bayes, K-Means, Random Forest, etc. Back propagation (BP) neural network and multilayer perceptron are two classic artificial intelligence (AI) machine learning methods. They have been widely used in engineering and scientific research fields. With the development of AI and computer technology, there is a few of research have tried to apply the machine learning method to suicide and suicide attempt (Mezuk et al., 2018; Ryu et al., 2019; Chen et al., 2020; Gradus et al., 2020; Boudreaux et al., 2021). But there are few studies establishing the prediction model for suicide using the ANN. Furthermore, in much of the existing research, the predictor variables are difficult to measure or are not common in the general population. So it is important to develop a brief, easily administered, and widely applicable model for predicting suicide risk using AI methods basing on BP neural network and multilayer perceptron.

The current study aimed to establish the prediction model of suicide through logistic regression, Back Propagation Neural Network (BPNN) and multilayer perceptron methods, respectively, in order to verify the application of the neural network model and compare the accuracy of the discriminant effects. Because ANN can handle the complex fuzzy mapping relations and identify complex non-linear relationships between variables by simulating the human brain intelligently, so the research hypothesis was that the neural network model would be more accurate than the traditional method.

Materials and methods Data collection

Data came from a large case-control designed survey carried out in 16 counties (Xian) from three provinces (Shandong, Liaoning and Hunan provinces) in China. Shandong, Liaoning and Hunan province is locate in north, northeast, and south in China, respectively. And the countries (Xian) were chosen by the random sample method from each province. In China, all death incidents are strictly required to be reported to the civil affairs agency in order to obtain death certificates. Village doctors were recruited to collect the death certificate and report the death to the Centre for Disease Control (CDC). The suicide cases were traced by the death reporting system of CDC. In order to control the selective bias, strict inclusion and exclusion criteria were established. After suicide cases were identified, cases were confirmed and verified by the experts in order to exclude cases of deliberate self-injury, accidental injury, mistaken ingestion of drugs or pesticides (Lyu et al., 2018). After the suicide cases were confirmed, a living community control, matched for gender, community and age (± 2 years), was randomly selected for each suicide case.

Data for each suicide case and control were collected by the PA method. In order to decrease biased responses, two informants for each case and control were enlisted. The first informant was always the parent, spouse or another family member, and the second informant was always a close friend, co-worker or neighbor. The informants were interviewed by the professionally trained interviewer. After interviews, the responses from the two informants were combined using the principles of information integration (Fang, 2009). A detailed description of the data collection can be reviewed in a prior publication (Lyu et al., 2016).

Measurements

The questionnaires were self-designed to collect data, which main included demographic variables, the predictor variables or influencing factors and the widely used psychological scales. The information of the questionnaires is as followings.

Demographic variables such as age, gender, residence, highest degree earned, education year, marital status, whether or not lived alone were measured. The survey items, response options and recoding methods are shown in Table 1.

TABLE 1

Table 1. The demographic variables of suicide case and control groups.

The possible predictors of suicide such as number of family members, spousal relation, relation with parents, status in family, family financial status, superstition, health condition, severe chronic disease, mental illness, family mental disorder history, family suicide history, and pesticide stored at home, as well as whether aspirations were reached or not were assessed through the self-designed questionnaire. The options and records of the items are shown in Table 2.

TABLE 2

Table 2. Predictors of suicide in case and control groups.

In order to investigate the mental health of the subjects, several psychological scales were used. These included the Beck Hopelessness Scale (BHS), the Landerman Social Support Scale (1989), the Dickman Impulsivity Scale (DII), and the State-Trait Anxiety Inventory (STAI). Previous research has indicated that these scales have good reliability and validity in China (Kong et al., 2007; Zhang Y. et al., 2012).

Data analysis methods

The X¯⁢±⁢S⁢D was used to describe the quantitative data and n (%) was used to describe the qualitative data. Further, t-tests, chi-square tests and their adjusted tests methods were used to compare the differences between the two groups. Logistic regression was applied to preliminarily screen the factors. BP neural network and Multilayer Perceptron were used to examine the suicide prediction model. Significance levels were set to P≤0.05. The SPSS 21.0 and Matlab R2010b softwares were applied to analyze and establish the model.

Ethical approval

This study was approved by the ethics committee of Shandong University and written informed consent was obtained from all participants in the study.

Results Demographic description of the sample

The average age of the control group was 25.69 ± 6.16 and the age of case group was 26.84 ± 6.37. Results indicate that there is significant difference in age between two groups (P = 0.010). In the control group, there were 202 (48.6%) males and 214 (51.4%) females. In the case group, there were 214 (54.6%) males and 178 (45.4%) females. There was no significant difference by gender between two groups (P = 0.086).

Results indicate that there was no significant difference in urban vs. rural residence (P = 0.124), but there were significant differences on highest degree earned (P < 0.001), years of education (P < 0.001), marital status (P = 0.020) and whether or not lived alone (P = 0.005). The suicide cases were more likely to be older, to have lower levels of education, to be single/never married and to live alone. The statistical description for demographic variables and the comparative results of two groups are shown in Table 1.

Preliminary screening of predictors of suicide

The prevalence of the possible predictors of suicide for each group is found in Table 2. Comparisons indicated that all items are significantly different between groups. Specifically, the suicide cases are likely to have fewer family members (P = 0.003), poor relations with both spouse and parents, lower status in their family, lower family financial status, more superstition, poorer health condition, and higher rates of severe chronic disease, mental illness, family mental disorder history, family history of suicide, pesticide storage at home and unreached aspirations (All P < 0.001).

Table 3 describes the measures of mental health for the groups. Results demonstrate that the suicide case group has lower BHS despair scale scores, indicating higher levels of desperation (P < 0.001). Further, the suicide cases report lower social support, higher impulsivity and higher anxiety than the control group (P < 0.001).

TABLE 3

Table 3. Measures of mental health.

Further screening of predictive factors

In sum, 22 of 24 variables were significantly different by group. In order to create model that is precise and applicable for the general population, several variables were excluded from further analysis. Specifically, highest degree obtained and years of education are highly correlated and lead to collinearity issues. Thus, years of education was excluded. Further, although the spousal relation and relation with parents were both significant different in two groups, spousal relations variable does not apply to everyone. Therefore, a spousal relation was excluded as well. Finally, because the goal is to create a summary and concise model that can be quickly and easily deployed in prevention, however the measures of mental disorder needs numerous and tedious questionnaire, so the mental disorder were also excluded. With these deletions, the final model comprises 16 variables. Although it is possible that excluding any variables may reduce the predictive validity of the model, it is expected that the 16-variable model should have sufficient validity and will have the advantage of being easily implemented by those screening for suicide risk.

The logistic regression prediction model of suicide

Likelihood Ratio (LR) stepwise Multivariable logistic regression was applied to further screen the predictors of suicide. In the multivariable logistic regression, suicide group or control group (1 = Suicide, 0 = Control) was the dependent variable and the 16 variables were the independent variables. Table 4 shows that there are 10 variables (Age, Highest diploma, Marital status, Relation with parents, Family financial status, Superstition, Severe chronic disease, Mental illness, Family suicide history, Pesticide stored at home) in the optimal multivariable logistic regression, and Nagelkerke R2 is 0.537. This yields a percentage correct for the control group of 85.3%, percentage correct for the case group of 76.3%, and overall subjects is 80.9%.

TABLE 4

Table 4. The logistic regression prediction model of suicide.

Multivariate logistic regression indicates that a diagnosis of mental illness (odd ratio OR = 28.377), family suicide history (OR = 5.912), superstition (OR = 3.264), poor relation with parents (OR = 2.757), and severe chronic disease (OR = 1.947) are the top five risk factor of suicide in this Chinese sample.

Back propagation neural network prediction model for suicide

Artificial Neural Network, as an AI technology, imitates the structure and function of biological human brain to establish mathematical models. ANN is an adaptive system (Ma, 2010; Liu and Guo, 2014) consisting of a mass of neuron nodes. ANN has been shown to be an effectively applied in associative memory, non-linear mapping, classified recognition and optimization design (Ma, 2010). BPNN is a multi-layer back-propagation network model trained according to the error reverse propagation algorithm (Liu and Guo, 2014). The current study selected the typical three-layer BP neural network with one input layer, one hidden layer, and one output layer. The 10 variables screened by multiple logistic regression analysis were set as the input variables, and suicide or not (1 = Suicide, 0 = Control) was set as the output variable. The range of neurons number in the hidden layer were determined by H=M+N+α (Wang and Wang, 1999). (α is the constant from 1 to 10, N is No. of input neurons, M is the No. of output neurons). In this study the range of the hidden layer is 4 ∼13. Hidden layer was set to hyperbolic tangent function and output layer was set to logarithmic sigmoid function.

Training of back propagation neural network

The Graphical User Interfaces (GUI) of the Matlab R2010b software was used to complete the neural network model. The “Pattern Recognition Tool” module was used to process the classification pattern recognition interface. Data file of X.mat and Y.mat were imported as the input and output neurons. The Matlab R2010b software randomly divided the database into groups for difference testing purposes: 70% as the training sample, 15% as the verification sample and 15% as the test sample. Next, the neural network models were established through the data simulation process repeatedly by Matlab R2010b software under different number of hidden layers. Model evaluation indices, such as sensitivity (Se), specificity (Sp), total coincidence rate (π) and number of iterations are shown in Table 5. It indicates that when there are 10 hidden layers the model evaluation indices are optimized. The structure of BP neural network model, parameter settings and operation results parameters are showed in Figure 1.

TABLE 5

Table 5. Predicted evaluation indices of Back Propagation Neural Network (BPNN) with different hidden layers neurons.

FIGURE 1

Figure 1. Structure and the parameter settings of the back propagation (BP) neural network.

Verification and evaluation of back propagation neural network

The confusion matrixes output by Matlab R2010b are shown in Figure 2. The coincidence rates of each sample are 83.9, 81.8, and 79.3%. The total coincidence rate for all samples is 82.9%. The area under receiver operating characteristic (ROC) curve of each sample is about 82%, which is shown in Figure 3. The confusion matrixes and ROC curve indicate that the discrimination efficiency of the prediction model is superior to the logistic regression model.

FIGURE 2

Figure 2. Confusion matrixes of BP neural network model.

FIGURE 3

Figure 3. The receiver operating characteristic (ROC) curve of Back Propagation Neural Network (BPNN) for each sample.

Multilayer perceptron prediction model for suicide

Multilayer Perceptron is another common neural network method, which is widely applied in many fields. The 10 variables identified through multiple logistic regression analysis were set as the input variables and suicide or not was set as the output variable. The SPSS 21.0 software was applied to establish the multilayer perceptron prediction model. In this study the 3-layer (with 1 hidden layer) multilayer perceptron was chosen to establish the prediction model. The database was divided into 3 parts, 70% for the training sample, 15% for the test sample and 15% for the holdout sample. Different architectures and different type activation functions of the hidden layer and the output layer were set to simulate the database repeatedly. The different multilayer perceptron models and the evaluation indices (such as percent correct, area under the ROC curve) are displayed in Table 6. The result indicates that when the activation function of the hidden layer is a hyperbolic tangent, that of the output layer is identity and the number of units in hidden layer is 9 (Model 2), the multilayer perceptron model had the highest AUC. The ROC of Model 2 is displayed in Figure 4. Table 6 and Figure 4 both indicate that the AUC of the optimal multilayer perceptron prediction model is above 90%, which indicates that the discrimination efficiency of the multilayer perceptron model was superior to BPNN model and the logistic regression model. The multilayer perceptron neural network model of suicide has good discrimination efficiency. The importance chart of independent variables is displayed in Figure 5.

TABLE 6

Table 6. The multilayer perceptron models of suicide and the evaluation indices.

FIGURE 4

Figure 4. The receiver operating characteristic (ROC) curve of multilayer perceptron prediction model.

FIGURE 5

Figure 5. Importance chart of independent variables of multilayer perceptron.

Discussion

The multivariate logistic regression predictor model screened 10 main predictive factors for suicide and overall percentage correct was 80.9%. The total coincidence rate was 82.9% and ROC curve was about 82.0% in BPNN prediction model and the AUC of the optimal multilayer perceptron prediction model was above 90%. The current study verifies the hypnosis that the neural network prediction models have higher accuracy rates than traditional statistical methods. It is advisable to apply traditional statistical methods combined with neural network method.

Although there is considerable previous literature on suicide behavior (Zhang X. et al., 2012; Zhang Y. et al., 2012; Lyu and Zhang, 2014; Oexle and Rusch, 2018; Shah et al., 2018; Tong et al., 2018), the aim of most previous studies was to describe and identify risk factors for suicide. However, much of the prior work on prediction of risk has involved traditional linear statistical models. The current study aimed to establish the different prediction model of suicide and verify the application of the neural network model and compare the accuracy of the discriminant effects.

There are 10 variables identified through multivariate logistic regression, which are the main predictive factors for suicide. The odd ratio (OR) values indicate that mental illness (OR = 28.377), family suicide history (OR = 5.912), superstition (OR = 3.264), poor relation with parents (OR = 2.757), and severe chronic disease (OR = 1.947) are the most important risk factors of suicide in China. The results of current study are in accordance with previous research (Rajalin et al., 2012; Zhang X. et al., 2012; Zhang and Lv, 2013).

Similarly, this study yielded results consistent with prior work in mental health concerns. Greater desperation (Pompili et al., 2011; David Klonsky et al., 2012), lower social support (Rothon and Stansfeld, 2012; Saha et al., 2012; Panayiotou and Karekla, 2013), higher impulsivity (An et al., 2010; Gvion and Apter, 2011; Doihara et al., 2012) and higher anxiety were all significant risk factors for suicide (P < 0.001). However, because these variables are not quickly and easily measured they were excluded in the model establishment.

Comparing the three prediction models, the percentage correctly identified of control and case groups is 85.3 and 76.3%, respectively, and the overall percentage correct is 80.9% in the logistic regression model. For the BPNN prediction model, the coincidence rates of each sample were 83.9, 81.8, and 79.3%. The total coincidence rate for all samples was 82.9% and the area under ROC curve was about 82.0%. The coincidence rate is indeed not too high, the reason may be the research data come from the social survey and there are some information biases different to be controlled. Therefore, the conclusion can be drawn that the forecast effect of the BPNN exceeds the logistic regression model. The AUC of the optimal multilayer perceptron prediction model is above 90%, higher than even that of the BPNN model (82.0%). Thus, discrimination efficiency of the multilayer perceptron model was superior to BPNN model and the multilayer perceptron model basing on the neural network is the best prediction model.

The current study verifies that the neural network prediction models have higher accuracy rates than traditional statistical methods. The reason may be the advantages of the ANN. ANN can imitate the mechanisms of cerebral neurons to identify the complex, non-linear relationship of variables and not require specific distribution of the data. The multilayer perceptron model basing on the neural network appears to have clinical and preventive value to distinguish high-risk populations for suicide, and also has important theoretical value for developing auxiliary clinical diagnostic system for psychiatrists. The neural network prediction model based on BPNN and multilayer perceptron can be output and saved, and the saved model can be integrated with network technology to develop website diagnostic systems to identify suicide risk.

Although the neural network method is suitable for identifying potential suicide cases and has greater accuracy than logistic regression method, the neural network method still has strengths and weaknesses. The logistic regression gives the regression coefficients explicitly and is convenient for us to explain. Although the neural network model is not as helpful to identify predictor variables in the preliminary stages of research, it is superior in discriminant efficiency and data simulation. However, the neural network method still has some weaknesses. First, although the structure, weight coefficient and threshold value of neural network model are given, the parameters may change in different data simulation process, making it difficult to make professional interpretations. Second, there are many implicit relationships among variables, so the fixed relationship among them could not be easily displayed. It may be advisable to apply traditional statistical methods combined with neural network models.

Conclusion

In order to establish brief, accurate, and widely applicable prediction model of suicide, the current study used logistic regression, BPNN and multilayer perceptron methods to establish and compare the accuracy of three prediction models.

The overall percentage correct of samples is 80.9% in logistic regression model. The total coincidence rate for all samples was 82.9% and the area under ROC curve was about 82.0% in the BPNN prediction model. The AUC of the optimal multilayer perceptron prediction model is above 90% in multilayer perceptron model. Although the classification effect is not high excellently, the current study verified the hypothesis that the neural network prediction models have greater accuracy than traditional statistical methods. The discrimination efficiency of the multilayer perceptron model was superior to BPNN model and it is the best prediction model of suicide.

Although the neural network method is suitable to identify suicide cases and is more accurate than the logistic regression method, the neural network method still has strengths and weaknesses. It may be advisable to apply both traditional statistical methods and neural network models.

The neural network prediction model of suicide based on multilayer perceptron and BPNN has clinical and preventive significance to distinguish high-risk populations, and it has important theoretical significance for developing an AI auxiliary clinical diagnostic system for psychiatrists.

Limitations

Firstly, only the general macroscopical variables were considered, the microcosmic and fussy variables were excluded in this study. Secondly, the processes of data simulation were manually operated by software, and the times and the amount of simulation processes were limited. Further study should try to operate the simulation processes by computer automatically. Thirdly, data collected from different sample may lead to the bias of results. In the end, the effective variables for establishing successful classification still need to be filtrated by the two methods in the future. The more excellent, robust and successful machine learning algorithms should be investigated in the next research.

Data availability statement

The original data is available upon reasonable request, further inquiries can be directed to the corresponding author.

Ethics statement

This study was reviewed and approved by the Ethics Committee of Shandong University and written informed consent to participate was obtained from all participants in the study.

Author contributions

JL contributed to the data analysis, conception, and manuscript writing. HS revised the manuscript and funded the publishing charge. JZ contributed to the data collection and conception. JN polished the manuscript and English language. All authors listed have made a substantial, direct, and intellectual contribution to the work, and approved it for publication.

Funding

This research was supported by (1) the United States National Institute of Mental Health (NIMH): R01 MH068560, (2) Natural Science Foundation of Shandong Province: ZR2021MH408. (3) Priority Research and Development Program (Soft Science Project) of Shandong Province: 2021RKY05098 and (4) Shandong Provincial Youth Innovation Team Development Plan of Colleges and Universities: No. 2019-6-156, Lu-Jiao.

Acknowledgments

We thank all collaborators for their unique contribution to the study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

An, J., Phillips, M. R., and Conner, K. R. (2010). Validity of proxy-based reports of impulsivity and aggression in chinese research on suicidal behavior. Crisis 31, 137–142. doi: 10.1027/0227-5910/a000019

PubMed Abstract | CrossRef Full Text | Google Scholar

Boudreaux, E. D., Rundensteiner, E., Liu, F., Wang, B., Larkin, C., Agu, E., et al. (2021). Applying machine learning approaches to suicide prediction using healthcare data: overview and future directions. Front. Psychiatry 12:707916. doi: 10.3389/fpsyt.2021.707916

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, Q., Zhang-James, Y., Barnett, E. J., Lichtenstein, P., Jokinen, J., D’Onofrio, B. M., et al. (2020). Predicting suicide attempt or suicide death following a visit to psychiatric specialty care: a machine learning study using Swedish national registry data. PLoS Med. 17:e1003416. doi: 10.1371/journal.pmed.1003416

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, X., Sun, Y., Li, Z., Yu, B., Gao, G., and Wang, P. (2019). Historical trends in suicide risk for the residents of mainland China: APC modeling of the archived national suicide mortality rates during 1987–2012. Soc. Psychiatry Psychiatr. Epidemiol. 54, 99–110. doi: 10.1007/s00127-018-1593-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Chiang, C. H., Weng, C. L., and Chiu, H.-W. (2021). Automatic classification of medical image modality and anatomical location using convolutional neural network. PLoS One 16:e0253205. doi: 10.1371/journal.pone.0253205

PubMed Abstract | CrossRef Full Text | Google Scholar

David Klonsky, E., Kotov, R., Bakst, S., Rabinowitz, J., and Bromet, E. J. (2012). Hopelessness as a predictor of attempted suicide among first admission patients with psychosis: a 10-year cohort study. Sdicide Life Threat. Behav. 42, 1–10. doi: 10.1111/j.1943-278X.2011.00066.x