1 INTRODUCTION

Neurocognitive disorders (NCDs), including delirium, mild cognitive impairment (MCI), and dementia, pose a grave public health challenge.1, 2 NCDs are commonly prevalent among perioperative patients with sleep disturbance,3 especially those admitted to the intensive care unit (ICU).4, 5 Coupled with the impact of sleep disturbance,6, 7 NCDs in the ICU are associated with increased mortality and even continue to deteriorate cognition and sleep after ICU discharge.8 In the ICU, sleep disturbance is widespread in adult patients, with some studies reporting a prevalence of sleep disturbance in the range of more than 50%.4, 9, 10 Notably, NCDs are closely associated with sleep disturbance and are highly prevalent among perioperative patients in the ICUs.11-13 Adult ICU patients with sleep disturbance have been suffered from serious yet still largely unresolved NCDs.4, 14 Risk assessment tools for ICU delirium are available,15, 16 not specifically for ICU patients with sleep disturbance. Research on MCI and dementia in the ICU, especially their risk assessment, is rarely available, not to mention that in ICU patients with sleep disturbance. There is currently a lack of an overall risk assessment system of NCDs specifically for ICU patients with sleep disturbance, including those perioperatively admitted to the ICU.

Accumulating evidence indicates potential bidirectional regulatory connections between sleep disturbance and cognitive impairment.5, 14, 17-20 Impaired amyloid-β clearance, increased tau levels, aggravation of inflammation, impaired synaptic plasticity, and changes in neurotransmitters, may have important roles underlying the potential association between sleep disturbance and NCDs.18, 21 Additionally, evidence from epidemiological and clinical studies reveals that delirium, MCI, and dementia have a close correlation with sleep disturbance. Besides, sleep disturbance is one of the diagnostic criteria for delirium.22 Although mechanistic studies have identified novel biomarkers with high sensitivity and specificity, they lack popularity in clinical practice. Most importantly, no potential clinical biomarkers associated with NCDs have been identified in patients with sleep disturbance. Sleep disturbance is likely to be a precursory and concomitant symptom of NCDs and is likely to share clinical markers with NCDs. A study on the prediction for postoperative sleep disturbance indicated that gender, midazolam, and sufentanil were important clinical markers independently associated with postoperative sleep disturbance.23 Our previous study on predicting sleep disturbance in ICU patients suggested that some biomarkers from routine laboratory tests also were independent predictors.24 In the absence of NCD-related prediction markers available in patients with sleep disturbance, based on clinical data including the previously established prediction models for sleep disturbance, developing the overall evaluation system of NCDs risk has important clinical value for ICU patients with sleep disturbance. At present, there are no prediction models for NCDs in ICU patients with sleep disturbance, though there have been many kinds of research on neurocognitive consequences of sleep disturbance. This situation may increase the risk of NCDs in ICU patients with sleep disturbance.

As recommended,12 early diagnostic prediction models can effectively assist healthcare providers in estimating the risk of a specific disease or present condition. This provides theoretical support for the early prediction of NCDs in patients with sleep disturbance. Based on the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis,25 our study design specifically focused on the prediction of NCDs in adult patients with sleep disturbances, who were admitted to the ICU. We aimed to develop and internally validate novel models for the prediction of NCDs in adult ICU patients with sleep disturbance using the MIMIC-IV database.

2 MATERIALS AND METHODS 2.1 Data source

This retrospective cohort-based study was conducted using the MIMIC-IV database (version 1.0), the most recent update after MIMIC-III.26 It is a longitudinal, large, single-center database that incorporates contemporary critical care data for over 60,000 patients admitted to ICUs at the Beth Israel Deaconess Medical Center between 2008 and 2019. Patient identifiers in MIMIC-IV were removed to maximize patient privacy protection. MIMIC-IV is a publicly available clinical database that allows for data sharing only after passing the Collaborative Institutional Training Initiative examination. One author who had access to MIMIC-IV from our study group specialized in data extraction from this database. We constructed a clinical dataset of patient hospitalization information, including their demographics, laboratory measurements, medications, and other health-related information, based on the structured query language for data extraction.

2.2 Participants

We included adult patients (age ≥18 years) with sleep disturbance admitted to the ICUs and extracted their data from the MIMIC-IV. According to the third edition of the International Classification of Sleep Disorders, sleep disturbance was divided into seven subtypes including insomnia, sleep-related breathing disorders, central disorders of hypersomnolence, circadian rhythm sleep–wake disorder, parasomnias, sleep-related movement disorders, and other sleep disorders (Appendix S1). Furthermore, sleep disturbance was determined following diagnostic codes of the International Classification of Diseases, 9th revised (ICD-9) and 10th revised (ICD-10) editions. Only patients who met the criteria of the ICU duration of stay greater than 24 h were included.

2.3 Outcomes

The primary outcomes included the major NCDs during ICUs stay. Based on the recommendations in the 5th Edition of the Diagnostic and Statistical Manual of Mental Disorders,22 we included delirium, MCI, and dementia as the major NCDs. The information on included NCDs was extracted according to the ICD-9 code (Appendix S2). Given that neuropsychiatric diseases may also accompany cognitive impairment, we excluded specific neuropsychiatric diseases including craniocerebral diseases, meningitis, encephalitic diseases, epilepsy, cerebrovascular diseases, encephalopathy, mental and neurological disorders, alcoholism or drug abuse, and other common neuropsychiatric disorders (Appendix S3).

2.4 Predictors of NCDs

For the prediction of NCDs, clinical and biological variables were extracted from the MIMIC-IV. For patients who had been admitted to ICUs multiple times, we only used the information at their first ICU admission. Baseline data, vital signs, and system scores only within the first 24 h of ICU admission were included in the analysis. Additionally, only the first recorded data of laboratory measurements were analyzed. The variables included were as follows: (1) patient demographics, including age, gender, admission type, marital status, and ethnicity; (2) vital signs, including heart rate, blood pressure, respiratory rate, temperature, pulse oxygen saturation (SpO2), and partial pressure of carbon dioxide (pCO2); (3) laboratory parameters, including creatinine level, blood urea nitrogen, hemoglobin, platelet count, partial thromboplastin time, international normalized ratio, prothrombin time, white blood cell count, lymphocyte, neutrophils, sodium, potassium, pH, and glucose; (4) comorbidities were identified according to the ICD-9 and ICD-10 codes, including Charlson Comorbidity Index (CCI), hypertension, diabetes, cardiovascular diseases, chronic pulmonary diseases, liver diseases, kidney diseases, and immunodepression; (5) medications, including analgesics and sedative drugs (morphine, midazolam, propofol, etomidate, dexmedetomidine, and haloperidol) and cardiovascular drugs (norepinephrine, epinephrine, dobutamine, and dopamine); (6) system scores, including Simplified Acute Physiology Score II (SAPS II), Sequential Organ Failure Assessment (SOFA) score, and Glasgow Coma Score (GCS); (7) treatment strategies, including mechanical ventilation and renal replacement therapy, along with the length of ICU stay.

2.5 Sample size

After using the inclusion and exclusion criteria, a total of 4,895 eligible patients from MIMIC-IV were enrolled in our cohort. This cohort was randomly divided in a ratio of 7:3 into two groups, namely the primary and validation cohorts. Based on the set standard of 10 events per candidate predictor parameters in the machine learning algorithm,25 the 60 predictors included in this study required at least 600 individuals with their respective events. Also, considering other sample size requirements for developing a clinical prediction model, 3,916 patients in the primary cohort met the standard optimal sample size for further statistical analyses.

2.6 Data cleaning and missing management

Firstly, we searched the “icustays” table in the “icu” module of the MIMIC-IV database. The “icu” module contains data sourced from the clinical information system of MetaVision. The MetaVision table is denormalized to create a star schema, where the “icustays” and “id-items” tables are linked to a set of data tables, all of which are suffixed with an "event". The data recorded in the “icu” module included venous and fluid inputs (input events), patient outputs (output events), procedures (program events), information recorded as dates or times (date time events), and other patient chart information (chart events). All event tables had a “stay_id” column to identify related ICU patients from “icustays,” and an “item-id” column for identification of concepts recorded in “id-items”.26 Subsequently, information for 69,619 patients who were admitted to the ICU was retrieved. Further, we only included the “stay_id” at the first hospitalization and excluded the patients who were repeatedly admitted to the ICU. Finally, a total of 50,048 patients were enrolled in the first ICU admission. Subsequently, we merged and processed other forms according to the patient's “subject_id” number. Throughout the process, we removed patients whose vital signs and laboratory parameter information were missing value more than 10% and lacked the ICD diagnostic codes. Although missing data is frequent in data extraction, MIMIC-IV version 1.0 has addressed this issue by updating patient data and improving the separate and combined uses of module datasets. Finally, for quality control of missing data, a data profiling report (Data S1) was used to analyze all predictor parameters. The percentage of missing values of calcium (2.43%), bun (1.86%), pCO2 (1.65%), and the other variables were <2%. Additionally, we processed the variables with missing values through multiple imputations and filled in the missing data using their predictive values.

2.7 Statistical analyses

Data normality was verified by the Shapiro–Wilk test. Continuous variables were presented as the mean ±standard deviation (for normally distributed data) or median [interquartile range, IQR] (for non-normally distributed data) and as the frequency [percentage] (for categorical variables). All continuous variables in the dataset showed skewed distribution. Baseline characteristics between NCDs and non-NCDs groups in the primary and validation cohorts, respectively, were compared using the Mann–Whitney U test or the Kruskal–Wallis test for continuous variables with non-normal distributions or heterogeneity, and the Pearson Chi-squared test for categorical variables.

According to significant differences (p < 0.05) in baseline characteristics in the primary cohort, potential variables were used in the further multivariate logistic regression and LASSO regression. Odds ratio (OR) and 95% confidence interval (CI) in the logistic regression and coefficient in the LASSO regression were calculated to identify significantly associated independent risk predictors for NCDs. Based on the results of the LASSO regression and multivariate logistic analyses, two nomograms were constructed. The final prediction models were presented as nomograms, which were the main process of developing prediction models for individual NCD diagnoses in this study. Moreover, the calibration, discrimination, and clinical utility of each nomogram were evaluated. The bootstrap resampling procedure with 1,000 repetitions was used to internally calibrate the nomogram in the validation cohort, and a calibration curve was plotted to analyze the accordance between the predicted probability using the nomogram and actual occurrence. The discriminative ability of the nomogram was analyzed using the receiver operating characteristic (ROC) curves and the area under the ROC curves (AUC), which were used along with the calibration curve to evaluate the predictive ability of the prediction models. Decision curve analysis (DCA) was used to assess the clinical utility of the prediction models for decision-making, and we plotted corresponding net benefits for a range of risk thresholds.

Statistical analyses were performed using the R software (version 3.4.3), and statistical significance was defined as a two-tailed p-value <0.05.

3 RESULTS 3.1 Participants

We analyzed clinical data of ICU patients obtained from the MIMIC-IV database, a total of 69,619 individuals between 2008 and 2019. Of the 50,048 patients admitted to the ICU for the first time, 5,582 were diagnosed with sleep disturbances. All patients having craniocerebral diseases (n = 345), mental illness (n = 235), alcoholism or drug abuse (n = 40), age <18 years old (n = 32), and ICU stay <24 h (n = 35) were excluded. Finally, a total of 4,895 patients with sleep disturbance were divided into the primary cohort (3,916 individuals) and the validation cohort (979 individuals) in this study; among them, 1,391 developed NCDs during ICU stay (Appendix S4).

A total of 1,110 patients in the primary cohort and 281 in the validation cohort exhibited NCDs. In the primary cohort, patients in NCDs group were younger than those in non-NCDs group (61.3 (48.5–73.7) vs. 63.7 (53.4–79.0); p = 0.001). Baseline demographics, including gender (female), admission type (emergency), marital status (married), and ethnicity (white) also showed significant differences between the two groups. There were significant differences in all vital signs (all p < 0.001). Notably, patients in NCDs group had lower creatinine levels (1.4 (0.9–1.7) vs. 1.6 (1.0–1.6), p < 0.001), potassium levels (4.1 (3.9–4.1) vs. 4.3 (4.0–4.6), p < 0.001), and glucose levels (123 (102–127) vs. 125 (111–130), p < 0.001); shorter partial thromboplastin time (34.7 (27.6–36.9) vs. 36.9 (28.3–36.9), p = 0.006) and international normalized ratio (1.6 (1.2–1.8) vs. 1.6 (1.2–2.0), p < 0.001); higher blood urea nitrogen levels (26 (17–29) vs. 24 (19–29), p = 0.002), hemoglobin levels (11.7 (10.4–13.4) vs. 11.6 (10.5–12.3), p = 0.001), platelet counts (226 (197–243) vs. 223 (195–232), p = 0.008), and higher pH values (7.41 (7.38–7.44) vs. 7.38 (7.35–7.42), p < 0.001), than those in non-NCDs group. The two groups significantly differed in terms of diabetes and cardiovascular disease incidences, but not for CCI. Compared with the non-NCDs group, patients having greater use of morphine (678 (61.1%) vs. 970 (34.6%), p < 0.001), midazolam (601 (54.1%) vs. 716 (25.5%), p < 0.001), and propofol (256 (23.1%) vs. 561 (20.0%), p = 0.036) were more likely to suffer from NCDs. Moreover, the NCDs group had higher SAPSII scores (36 (31–45) vs. 35 (32–39), p < 0.001) and SOFA scores (9 (3–10) vs. 6 (3–9), p < 0.001), while lower GCS (14 (12–15) vs. 15 (15–15), p < 0.001) values as compared to the non-NCDs group. Baseline characteristics and details at first admission to ICU for all participants are shown in Table 1.

TABLE 1. Characteristics of patients in the primary and validation cohorts Primary cohort Validation cohort NCDs group, n = 1,110 Non-NCDs group, n = 2,806 p NCDs group, n = 281 Non-NCDs group, n = 698 p Age 61.3 (48.5–73.7) 63.7 (53.4–79.0) 0.001 61.6 (49.6–76.1) 63.5 (53.8–72.8) 0.359 Gender, n (%) Female 563 (50.7) 1,152 (41.1) <0.001 152 (54.1) 293 (42) 0.001 Male 547 (49.3) 1,654 (58.9) 129 (45.9) 405 (58) Admission_type (%) Emergency 601 (54.1) 1,390 (49.5) <0.001 151 (53.7) 332 (47.6) <0.001 Observation 296 (26.7) 581 (20.7) 64 (22.8) 159 (22.8) Elective 39 (3.5) 96 (3.4) 9 (3.2) 26 (3.7) Urgent 70 (6.3) 221 (7.9) 32 (11.4) 56 (8.0) Others 104 (9.4) 518 (18.5) 25 (8.9) 125 (17.9) Marital_status (%) Married 455 (41.0) 1,512 (53.9) <0.001 109 (38.8) 377 (54.0) <0.001 Single 452 (40.7) 792 (28.2) 108 (38.4) 203 (29.1) Divorced 101 (9.1) 191 (6.8) 26 (9.3) 55 (7.9) Ethnicity (%) White 857 (77.2) 2,030 (72.3) 0.001 221 (78.6) 512 (73.4) 0.185 Black 122 (11) 434 (15.5) 30 (10.7) 102 (14.6) Others 131 (11.8) 342 (12.2) 30 (10.7) 84 (12.0) Vital signs, [IQR] Heart rate (bpm) 110 (101–123) 104 (100–108) <0.001 98 (95–102) 98 (95–102) 0.434 Diastolic blood pressure (mmHg) 47 (44–50) 45 (42–48) <0.001 44 (41–47) 45 (41–47) 0.725 Systolic blood pressure (mmHg) 94 (81–102) 93 (88–97) <0.001 94 (92–98) 94 (92–97) 0.725 Respiratory rate (bpm) 25 (22–29) 27 (23–30) <0.001 25 (23–28) 26 (23–28) 0.564 Temperature (℃) 37.3 (37.1–37.6) 37.2 (36.9–37.6) <0.001 37.1 (36.9–37.4) 37.2 (36.9–37.4) 0.237 SpO2 (mmHg) 129 (88–133) 123 (99–127) <0.001 122 (88.5–127) 123 (91.8–128) 0.546 pCO2 (mmHg) 46 (40–49) 41 (37–44) <0.001 41 (38–44) 42 (39–44) 0.225 Laboratory parameters, [IQR] Creatinine (mg/dL) 1.4 (0.9–1.7) 1.6 (1.0–1.6) <0.001 1.6 (0.9–1.8) 1.6 (1.0–1.8) 0.674 Blood urea nitrogen (mg/dL) 26 (17–29) 24 (19–29) 0.002 27 (17–30) 28 (19–29.3) 0.399 Hemoglobin (g/dL) 11.7 (10.4–13.4) 11.6 (10.5–12.3) 0.001 11.1 (10.8–12.1) 11.1 (10.9–11.6) 0.417 Platelet (×109/L) 226 (197–243) 223 (195–232) 0.008 228 (218–247) 227 (190–230) 0.001 Partial thromboplastin time (s) 34.7 (27.6–36.9) 36.9 (28.3–36.9) 0.006 32.2 (27.4–36.8) 36.6 (28.4–36.9) 0.002 International normalized ratio 1.6 (1.2–1.8) 1.6 (1.2–2.0) <0.001 1.4 (1.2–1.6) 1.4 (1.2–1.7) 0.090 Prothrombin time (s) 14.5 (12.3–15.4) 14.7 (12.5–15.4) 0.872 14.3 (12.2–15.4) 14.7 (12.5–15.5) 0.059 White blood cell count (×109/L) 11.1 (8.2–13.5) 11.5 (8.8–12.2) 0.678 11.0 (8.6–11.2) 11.0 (8.8–11.2) 0.606 Lymphocyte (%) 21.6 (15.3–26.7) 21.6 (15.0–26.0) 0.710 21.6 (15.8–26.7) 21.6 (14.0–25.1) Neutrophils (%) 69 (62.5–75.4) 69 (63.7–75.5) 0.468 69 (62.5–75.0) 69 (63.7–77.5) Sodium (mmol/L) 138 (137–139) 138 (138–138) 0.206 136 (134–139) 137 (134–139) 0.550 Potassium (mmol/L) 4.1 (3.9–4.1) 4.3 (4.0–4.6) <0.001 4.2 (3.9–4.4) 4.2 (3.8–4.3) 0.240 pH 7.41 (7.38–7.44) 7.38 (7.35–7.42) <0.001 7.39 (7.36–7.41) 7.38 (7.36–7.40) 0.645 Glucose (mg/dL) 123 (102–127) 125 (111–130) <0.001 123 (107–127) 123 (108–127) 0.992 Comorbidity, n (%) CCI 4 (2–6) 4 (3–6) <0.001 4 (3–7) 4 (3–7) 0.768 Hypertension 608 (54.8) 1,595 (56.8) 0.253 146 (52.0) 388 (55.6) 0.321 Diabetes 268 (24.1) 924 (32.9) <0.001 66 (23.5) 241 (30.7) 0.001 Cardiovascular diseases 225 (20.3) 661 (23.6) 0.028 55 (19.6) 181 (25.9) <0.001 Chronic pulmonary diseases 310 (27.9) 718 (25.6) 0.136 81 (28.8) 180 (25.8) 0.119 Liver. diseases 81 (7.3) 213 (7.6) 0.788 14 (5.0) 54 (7.7) 0.781 Kidney. diseases 148 (13.3) 439 (15.6) 0.074 38 (13.5) 129 (18.5) 0.074 Immunosuppressive 15 (10.4) 315 (11.2) 0.461 23 (8.2) 80 (11.5) 0.136 Medications, n (%) Analgesic and sedative drugs (%) Morphine 678 (61.1) 970 (34.6) <0.001 132 (47) 274 (39.3) 0.031 Midazolam 601 (54.1) 716 (25.5) <0.001 135 (48) 273 (39.1) 0.012 Propofol 256 (23.1) 561 (20.0) 0.036 111 (39.5) 298 (42.7) 0.390 Etomidate 215 (19.4) 563 (20.1) 0.657 96 (34.2) 217 (31.1) 0.364 Dexmedetomidine 189 (17.0) 516 (18.4) 0.333 116 (41.3) 298 (42.7) 0.721 Haloperidol 129 (11.6) 365 (13.0) 0.262 48 (17.1) 106 (15.2) 0.497 Cardiovascular system drugs (%) Norepinephrine 74 (6.7) 173 (6.2) 0.560 15 (5.3) 45 (6.4) 0.559 Epinephrine 13 (1.2) 49 (1.7) 0.255 1 (0.4) 9 (1.3) 0.297 Dobutamine 7 (0.6) 20 (0.7) 1.000 0 (0) 3 (0.4) 0.562 Dopamine 17 (1.5) 34 (1.2) 0.436 4 (1.4) 6 (0.9) 0.485 Score system, [IQR] SAPSII 36 (31–45) 35 (32–39) <0.001 34 (31.5–37) 34 (32–38) 0.695 SOFA 9 (3–10) 6 (3–9) <0.001 6 (3.5–9) 6 (4–9) 0.764 GCS 14 (12–15) 15 (15–15) <0.001 13 (11–14) 13 (11–15) 0.612