記住我
ORIGINAL ARTICLE
Year : 2021 | Volume
: 10
| Issue : 4 | Page : 106-111
Mortality prediction of mitral valve replacement surgery by machine learning
Marziyeh HosseiniNezhad1, Mostafa Langarizadeh1, Saeid Hosseini2
1 Department of Health Information Management, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran
2 Heart Valve Disease Research Center, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences, Tehran, Iran
Correspondence Address:
Dr. Mostafa Langarizadeh
Faculty of Health Management and Information Sciences Vali-e- Asr Av., Rashid Yasami St., Tehran
Iran
Source of Support: None, Conflict of Interest: None
DOI: 10.4103/rcm.rcm_50_21
Background: Mitral valve replacement procedure has increased in the Iran over the last years. For optimization of the results, as the other procedure, it needs statistical evaluation of the results, and then a system for the prediction of outcome. Hence, in this study, we generate a machine learning (ML)-based model to predict in-hospital mortality after isolated mitral valve replacement (IMVR). Materials and Methods: The patients who underwent IMVR from February 2005 to August 2016 were identified in a single tertiary heart hospital. Data were retrospectively gathered including baseline characteristics, echocardiographic and surgical features, and patient's outcome. Prediction models for in-hospital mortality were obtained using five supervised ML classifiers including: logistic regression (LR), linear discriminant analysis (LDA), support-vector machine (SVM), K-nearest neighbors (KNN), and multilayer perceptron (MLP). Results: A total of 1200 IMVRs were analyzed in our study. The study population was randomly divided into a training set (n = 840) and a testing set (n = 360). The overall in-hospital mortality was 4.2%. LR model had the best discrimination for 22 variables in predicting mortality after IMVR, with area under the receiver-operating curve (AUC), specificity, and sensitivity of 0.68, 0.73, and 0.58, respectively. A LDA model had an (AUC) of 0.73, compared to 0.56 for SVM, 0.51 for KNN, and 0.5 for MLP. Conclusions: We developed a robust ML-derived model to predict in-hospital mortality in patients undergoing IMVR. This model is promising for decision-making and deserves further clinical validation.
Keywords: Cardiac surgery risk stratification, machine learning, mitral valve replacement
Cardiovascular diseases (CVDs) are the leading cause of global death and a major contributor to disability.[1] In 2016, CVDs such as heart attacks and strokes are caused 31% of all world mortality. Low- and middle-income countries (LMICs) suffer a growing burden of death from CVD; of the 17.9 million global deaths resulting from CVD in 2016, over three-quarters happened in LMICs.[2] Nearly 280,000 prostheses are implanted over the world annually.[3] Preoperative assessment of a patient's surgical risk is an important component in cardiac surgery. Risk stratification can help patients and their families with insight into the existent risk of complications and mortality and guide the selection of cases for surgery versus alternative, nonsurgical therapies. It can also estimate the need for hospital care resources in cardiac surgery.[4]
During the last decades, several risk stratifications models have been developed to support clinical decision-making for cardiac surgery operations such as the European System for Cardiac Operative Risk Evaluation (EuroSCORE)[5] and Society of Thoracic Surgeons score.[6] However, some of these risk scores, such as the EuroSCORE, have shown major restrictions as they tend to overestimate the actual risk. This can potentially calculate inappropriate risk-averse practice that denies surgery to patients who would benefit from surgery, falsely reassuring conclusions about surgeon and center performance, patients and their doctors not being fully informed during the process of shared decision-making.[7] However, these risk scores are validated worldwide, but the data are not homogeneous and epidemiological characteristics of each population require local validation of these risk scores.[8] Most risk scoring systems have been developed by using a biostatistical technique based on a generalized linear model with assumptions of linear relationships.[4] In fact, all models in the current use are based on logistic regression (LR), which relies on the modeler input to manually specify interactions, such as complex interactions. Do not considering those relationships during the development of the risk scores may result in model misspecification. Because of this limitation in this context, machine learning (ML) techniques automatically learn the relationships from the data and do not require input from the modeler to specify interactions.[7] Herein, we tried to generate a ML-based model to predict in-hospital mortality after isolated mitral valve replacement (IMVR).
Materials and MethodsData
Data were obtained from all patients who underwent IMVR from February 2005 to August 2016 at a single academic institution (Rajaie Cardiovascular Medical and Research Center). Variables that were evaluated for inclusion in the ML models include demographics such as age and gender, echocardiographic data such as left ventricular ejection fraction, left ventricular end-systolic diameter, left ventricular end-diastolic diameter, mitral stenosis, mitral regurgitation (MR), aortic stenosis, right ventricular size, right ventricular function, clot status in left atrium, clot status in left atrium appendage, etiology, and surgery type. Other variables included preoperative laboratory values such as blood urea nitrogen, hematocrit, hemoglobin, creatinine, platelet count, white blood cell, and fasting blood sugar. The variables considered for inclusion are listed in [Table 1]. The study protocol was approved by the Research Review Board and the Ethics Committee of Iran University of Medical Sciences (code: IR.IUMS.REC.1399.681). All the patients signed the informed consent form.
Preprocessing
Statistical analyses were undertaken by using functions were developed in Python. By using a function, patients with missing values were excluded. A one-hot encoding function was applied to transform the categorical variables into numeric inputs. The final study population (n = 1200) was allocated into two groups: the train and test. The main target of our study was all-cause in-hospital mortality.
Classifier
We use five ML classification algorithms were trained with 70% of the data (training) and finally evaluated with the remaining 30% of the unseen data (testing). Five ML algorithms were used, including LR, linear discriminant analysis (LDA), support-vector machine (SVM), K-nearest neighbors (KNN), and multilayer perceptron (MLP). The target to predict was deceased, representing in-hospital mortality after IMVR. Models were constructed in Python version 3.7 (Python Software Foundation). [Figure 1] shows the analysis workflow schematically.
Figure 1: Computational methods. Schematic workflow for the construction of classification models including data preprocessing, evaluate the discriminant performance, and resulting interpretation. EvaluationAn open-source package called “scikit-learn” was used to implement the ML and evaluate the models. Choosing an appropriate metric is challenging generally in applied ML but is particularly difficult for imbalanced classification problems. Performance metrics were used include sensitivity, specificity, AUC, and f1-score. However, AUC was the metric that was used to determine the best algorithm and generate the final model but a meaningful sensitivity and specificity are desired.
Out of the 1200 patients included in our study, 50 (4.2%) did not survive hospitalization. The mean age of all patients was 50.16 (±12.37) years, and 68% were females. Of the five different ML methods (LR, LDA, SVM, KNN, and MLP), the LR model for 22 variables had the highest performance in terms of discriminating between survival and in-hospital mortality, with an AUC of 0.68. [Table 2] shows performance metric ML algorithms. [Figure 2], [Figure 3], [Figure 4] show the results of the ML analysis.
Figure 2: ROC AUCs derived from the machine learning models. LR: Logistic regression, LDA: Linear discriminant analysis, SVM: Support vector machine, KNN: K-nearest neighbors, MLP: Multi-layer perceptron, AUC: Area under curve
Figure 3: Sensitivity derived from the machine learning models. LR: Logistic regression, LDA: Linear discriminant analysis, SVM: Support-vector machine, KNN: K-nearest neighbors, MLP: Multilayer perceptron, SEN: Sensitivity
Figure 4: Specificity derived from the machine learning models. LR: Logistic regression, LDA: Linear discriminant analysis, SVM: Support-vector machine, KNN: K-nearest neighbors, MLP: Multilayer perceptron, SPE: Specificity Discussion Compared to traditional methods, to build a clinical prediction model by using LR, ML prediction models have the advantage of higher accuracy and robustness. Traditional algorithms such as LR require researchers to manually select the highly related independent variables X, whereas cutting-edge ML algorithms can find out the relationship between X and Y automatically. In this study, we demonstrated the role of ML approaches in the prediction of in-hospital mortality after IMVR in an Iranian population. We found that ML technique can act better than conventional statistical model, LR, to predict immediate outcomes of IMVR. To the best of our knowledge, this is the first study to implement ML approaches to predict the outcomes in patients undergoing IMVR among the Iranian population. We selected 1200 of patients who underwent IMVR and developed five ML algorithms, including LR, KNN, SVM, MLP, and LDA. The best predictive model in our study was generated by LR (AUC = 0.63). Clinicians and cardiologists, in particular, are becoming overwhelmed by the amount of data that is now being collected and documented about patients, particularly within clinical databases. The aims of developing IMVR risk model include quality monitoring of surgical performance, counseling patients to aid with decision-making, and cost-benefit analysis. These findings suggest that in the future, ML could have an important clinical role in evaluating prognostic risk in patients undergoing IMVR.
In recent studies, several ML-based models have been developed to predict mortality among patients who underwent heart surgery, but its application to IMVR outcomes was rarely.
Daliri et al.[9] developed an Iranian (cardiopulmonary bypass (CPB)) based risk stratification score model. Information of 1920 consecutive patients who underwent elective and emergent surgery was collected. In this study suggests inserting CPB as a determinant variable in predicting mortality. The best model to fit the data was binomial LR. The AUC of this risk stratification was 0.95. Nouei et al.[10] by using a genetic fuzzy system developed a risk assessment of mortality after cardiac surgery. Their study included a group of 1811 patients who underwent a coronary artery bypass grafting (CABG). The developed system leads to 100% sensitivity and 84.7% specificity. Ghavidel et al.[11] developed two new mathematical models for prediction of early mortality risk in CABG surgery. Data of 948 consecutive patients who underwent CABG surgery were selected. An entropy error fuzzy decision tree (EEFDT) and an entropy error crisp decision tree (EECDT) were implemented. The AUC of EEFDT and EECDT was 0.90 and 0.86 consequently.
Zhong et al.[12] build up multiple ML models to predict 30-day mortality, and three complications including septic shock, thrombocytopenia, and liver dysfunction after open-heart surgery. Patients who underwent coronary artery bypass surgery, aortic valve replacement, or other heart-related surgeries. XGBoost, random forest (RF), artificial neural network (ANN), and LR were employed to build the models by utilizing five-fold cross-validation and grid search. Receiver operating characteristic curve, area under curve (AUC), decision curve analysis, test accuracy, F1 score, precision, and recall were applied to access the performance. Among 6844 patients enrolled in this study, 215 patients (3.1%) died within 30 days after surgery, part of patients appeared liver dysfunction (248; 3.6%), septic shock (32; 0.5%), and thrombocytopenia (202; 2.9%). XGBoost, selected to be their final model, achieved the best performance with highest AUC and F1 score. AUC and F1 score of XGBoost for four outcomes: 0.88 and 0.58 for 30-day mortality, 0.98 and 0.70 for septic shock, 0.88 and 0.55 for thrombocytopenia, and 0.89 and 0.40 for liver dysfunction. Penso et al.[13] in “Machine Learning Prediction Models for Mitral Valve Repairability and Mitral Regurgitation Recurrence in Patients Undergoing Surgical Mitral Valve Repair,” analyzed clinical and echocardiographic data of 1000 patients who underwent MV repair. Patients were followed longitudinally for up to 3 years. Endpoints were MV repair surgical failure with consequent MV replacement or moderate/severe MR (>2+) recurrence at 1 month and moderate/severe MR recurrence after 3 years. For intraoperative or early MV repair failure assessment, the best area under the curve (AUC) was 0.75 and the complexity of mitral valve prolapse was the main predictor. In predicting moderate/severe recurrent MR at 3 years, the best AUC was 0.92 and residual MR at 6 months was the most important predictor. Liu et al.[14] used ML models to predict red blood cell transfusion in patients undergoing mitral valve surgery. They retrospectively reviewed 698 cases of isolated mitral valve surgery with and without combined tricuspid valve operation. They used of 13 ML algorithms including CatBoost, Light Gradient Boosting Machine, XGBoost, gradient boosting, extra trees (ET), LR, LDA, RF, AdaBoost, naive Bayes (NB), KNN, decision tree, and quadratic discriminant analysis. A CatBoost classifier had the best result with an AUC of 0.88. Chang Junior et al.[15] for improving preoperative risk-of-death prediction in surgery congenital heart defects used artificial intelligence models. Two thousand and two hundred and forty patients with congenital heart defects were enrolled. The MLP, RF, ET, stochastic gradient boosting, AdaBoost classification, and bag decision trees ML algorithms were used in this study. The RF reached a rate of 0.92 of AUC. Penso et al.[16] retrospectively enrolled 471 patients undergoing TAVI to predict mortality at 5 years after it. They used different ML models such as RF, extreme gradient boosting (XGBoost), MLP, and LR. The best AUC was reached combining LASSO as the feature selection method and MLP as the model, which was 0.79. Hernandez-Suarez et al.[17] for predicting in-hospital mortality for 849 patients who underwent transcatheter mitral valve repair developed five different ML classification algorithms include RF, LR, SVM, NB, and ANN. A NB model had the best discrimination with an AUC of 0.83. Hernandez-Suarez et al.[18] analyzed a total of 10883 data. LR, ANN, NB, and RF ML algorithms were applied to obtain in-hospital mortality after transcatheter aortic valve replacement. The best model was obtained by LR and AUC (0.92) Nowicki et al.[19] used the data of 3150 of mitral valve surgery (repair or replacement) and achieved the AUC (0.79) with a LR in predicting mortality after mitral valve surgery. Most of previous studies only focus on a specific ML algorithm, but we deployed LDA.
We acknowledge several limitations. Data imbalance is frequently encountered in medical applications. Resampling methods can be used in the binary classification to tackle this issue.[20] Our database was imbalances and we should use different resampling methods. Feature selection is the process of finding a subset of relevant features that can be used to develop efficient learning models.[17] We considered all the features. It is better to use feature selection for reducing the number of input variables needed to predict the target variable, removing noninformative or redundant predictors that might add uncertainty, thus degrading the performance of the model.
ConclusionsWe developed a ML-derived model to predict in-hospital mortality in patients undergoing IMVR by using a data mining approach from a nationwide sample. This is a promising model. To improve the model's discriminatory performance and applicability, additional studies with contemporary and more granular data are required.
Ethical clearance
The study protocol was approved by the Research Review Board and the Ethics Committee of Iran University of Medical Sciences (code: IR.IUMS.REC.1399.681).
Financial support and sponsorship
This study was part of a M.Sc. thesis supported by Iran University of Medical Sciences (grant No:IUMS/SHMIS-99-1-37-18291).
Conflicts of interest
There are no conflicts of interest.
References
留言 (0)