A retrospective propensity-matched cohort study was performed at a single tertiary referral academic medical center in Asheville, North Carolina (Mission Hospital / HCA Healthcare). Mission is an acute-care hospital with 853 licensed beds, including 147 ICU beds (87 adult / 60 pediatric/neonatal). The 87 adult ICU beds allocated to the care of patients included in this study comprise medical, surgical/trauma, cardiovascular, cardiothoracic surgery, neurological, and neurosurgical intensive care. The hospital has an affiliated ACGME-accredited residency and fellowship program which covers more than 170 trainees in 12 different training programs. The hospital’s primary service area comprises 18 counties in western North Carolina, providing the region’s only Level II trauma center, comprehensive stroke center, and children’s hospital. The study time-window was January 1, 2022, to December 31, 2022. Inclusion criteria consisted of all adult patients ≥ 18 years of age admitted to an ICU for a minimum of 4 h. The ICU admissions included medical or surgical indications for elective or urgent/emergent conditions. Exclusion criteria were patients < 18 years of age, downgrade to a labor/delivery unit, less than 4 h ICU length of stay, and a delayed readmission to ICU beyond 7 days. Analysis cohorts were identified based on the presence or absence of a return to the ICU during the inpatient stay. A return to the ICU was defined as a patient downgrade from the ICU to a routine or intermediate level of care setting with subsequent return to the ICU more than one hour and less than 7 days following the downgrade. The patient selection flowchart is depicted in Fig. 1.

Patient selection flowchart

This study was reviewed by the HCA Healthcare Institutional Review Board (IRB) and was deemed exempt from IRB oversight (ID# 2023 − 1146).

The Rothman Index (Spacelabs Healthcare, Snoqualmie, WA, USA) is a real-time, composite measure of medical acuity for hospitalized patients which serves as a predictive analytics model designed to provide an objective measure for continuous monitoring of a patient’s clinical status and improvement or deterioration over time [14]. The RI is automatically generated in real-time and calculated by measuring deviation from a minimum risk value of the defined clinical variables, with a maximum score of 100 representing no deviation from minimum risk, and a deterioration in RI score reflecting deterioration in a patient’s clinical status. The RI was designed to be applicable to any patient with any underlying condition, independent of the specific diagnosis, type of treatment or intervention, and respective environment [14]. At Mission Hospital, the RI has been adopted as a tool with dual intent that allows (1) clinical monitoring and (2) appropriate decision-making for appropriate downgrades in the acuity level of care and safe discharge planning at daily multidisciplinary rounds (MDR).

Figure 2 demonstrates the RI score grading thresholds pertaining to the respective decision-making recommendations used by clinical staff at Mission Hospital.

Rothman Index chart for clinical and patient downgrade/discharge decision-making at Mission Hospital (Asheville, North Carolina)

The primary outcome measure was in-hospital mortality or discharge to hospice for end-of-life care. Secondary outcome measures included overall hospital length of stay, ICU length of stay, and 30-day readmission rates. In-hospital mortality and discharge to hospice were defined using administrative discharge status codes. Cerner EHR Admission, Discharge and Transfer (ADT) system data were used to calculate both overall and ICU-specific length of stay. Hospital length of stay was defined as the number of days from admission to discharge, calculated to the hour. Length of ICU stay was calculated based on entry and exit date/times from the ICU. Readmissions were identified as any inpatient hospital visit to the same facility more than 6 h and less than 30 days from the time of inpatient discharge. Readmission analyses excluded patients with a discharge status of expired or discharged to hospice.

Propensity matching was used to control for differences across cohorts to better estimate the impact of ICU returns on the primary and secondary outcomes. Matching covariates included features which could influence either return to ICU or the primary and secondary outcomes, specifically: patient age, gender, and admission type, medical/surgical classification for hospitalization using MS-DRG grouping, the first RI score during the visit, and the Charlson Comorbidity Index. The Charlson Comorbidity Index uses patient comorbidities to predict long-term mortality and is a commonly used algorithm to assess chronic conditions in hospitalized patients [23]. Logistic regression techniques were used to identify the cumulative probability of a return to ICU using the matching covariates. Cases of patients returning to the ICU were then matched to controls based on these probabilities using a 1:1 Greedy propensity matching algorithm and requiring at least a four decimal place match between the case and control. This algorithm attempts to match cases with the highest precision match first and continues to perform matches until no additional matches are found thereby minimizing the number of incomplete and inexact matches. Baseline demographics including patient characteristics (e.g., age, gender, race, ethnicity), visit characteristics (e.g., admission type, discharge status), and clinical features representing clinical status (e.g., first RI score, Charlson Comorbidity Index, medical vs. surgical care, COVID-19 diagnosis) were reported and compared across cohorts before and after the matching process. Counts and percentages were used to report and compare categorical outcomes including mortality, discharge to hospice and 30-day readmissions across cohorts while mean, median, and standard deviation were used to compare overall and ICU-specific length of stay. Chi-square tests were used to analyze differences between cohorts for categorical variables with Fisher’s exact test used for comparisons with small sample sizes. For continuous variables, ANOVA was used to analyze differences between cohorts with Mann-Whitney tests used for non-normal distributions. Multivariable regression models were used to estimate the impact of a return to the ICU on primary and secondary outcomes. Logistic regression models were used for in-hospital mortality, discharge to hospice, and readmission. General linear regression models with negative binomial distributions were used to evaluate length of stay overall and in the ICU. Model confounders included patient and visit characteristics, patient comorbidities, and clinical features indicating severity of illness and physiological status. All statistical tests were conducted using SAS version 9.4 (SAS Institute, Cary, NC).

A p-value < 0.05 was considered statistically significant.

A sensitivity analysis evaluated the impact of time to ICU return on patient outcomes and hospital length of stay and to mitigate bias from the selection of a cut-point of seven days in the primary analysis. Outcomes were assessed and reported separately for returns to the ICU within three days and five days.

A descriptive analysis was conducted to evaluate the effectiveness of several features constructed from the RI to differentiate between patients with and without a return to the ICU. Analysis cohorts were identified based on the first ICU admission for both patients with and without a return to the ICU during the stay. ICU returns were defined using the methodology previously described. RI variables included the RI score at the time of the downgrade from the ICU to a lower level of care, the difference in the RI score between ICU entry and exit, the decrease in the RI score over the 24 h prior to downgrade and a binary indicator identifying if the patient was in a RI-generated warning at the time of downgrade. Configurable warnings based on either the RI score value or change in scores over time are part of the RI system functionality and serve an important role in operationalizing the RI for clinical decision support [24].

Propensity score matching with a 1:1 Greedy matching algorithm requiring at least a four decimal place match between the case and control was used to control for differences across cohorts similar to the primary analysis. For this subgroup analysis, matching variables included patient age, sex, type of ICU (i.e., medical vs. surgical/trauma), level of care in downgrade unit (i.e. routine vs. step-down), and first RI score in the ICU. A multivariable logistic regression model was used to evaluate the performance of the RI features on predicting returns to the ICU.