Introduction

Long-term care (LTC) residents require assistance with activities of daily living due to physical and/or cognitive impairment due to dementia, among other comorbidities.1 Care for these frail and vulnerable residents must be guided by the residents’ wishes. However, the typical medical paradigm of providing interventions to prolong life can have the paradoxical effect of causing discomfort or even harm, and are often incongruent with the goals that are most important to the resident. Residents, family members and even clinicians often do not understand or feel equipped to predict the ‘natural’ or expected downward trajectory in function and cognition (eg, loss of desire to eat and subsequent cachexia) that occurs for many LTC residents—the majority of whom are dying of the projected frailty trajectory (ie, as opposed to those dying of terminal illness such as cancer, where there is a defined terminal phase).2

Information about future health-related quality of life can support person-centred medical decision-making. Decisions—including transitions of care (eg, transfer to hospital to insert an IV line for hydration or feeding tube for those who are cachectic)—often do not consider the natural progression of disease nor the resident’s values and goals,3 4 resulting in actions that harm residents and are stressful for caregivers. Clear and accurate personalised predictions about functional and cognitive disability can support conversations between residents, caregivers and the medical team about how future changes might impact residents’ quality of life and when life-prolonging treatment should be pursued versus prioritising and supporting a natural death. Predictive models for functional and cognitive disability can also guide treatment recommendations. For example, if being able to dress and toilet are a critical part of a residents’ sense of dignity and personhood, then a high probability of not being able to do these activities after an illness could support and empower residents to remain at the LTC home instead of transferring to hospital when they become ill. The physician will be more confident to recommend the treatment plan that is most aligned with the resident's values, resulting in better care quality and resident satisfaction.

Existing models to predict physical disabilities have focussed on community-dwelling older adults.5 In a systematic review by Grootven and Achterberg that examined 43 studies and conducted 167 evaluations of predictive models, high risk of bias was found in 86 evaluations within the domain of participant recruitment and 158 evaluations within the study analysis domain. In particular, the authors stated that the participant recruitment strategy was not often specified, and missing data was not handled for analysis. In addition, predictor selection was often done using a data-driven approach based on p values, and censoring due to death was not accounted for in the models.4 Prior models that have been developed to predict cognitive ability in older adults used machine-learning approaches by reviewing neuroimaging results (eg, MRI and PET).5 However, this limits the models to older adults with previous neuroimaging and makes it challenging to use in LTC settings. To our knowledge, no model has been built for LTC residents—a growing population with increasing care needs—to predict resident-centred outcomes such as time to loss of functional or cognitive abilities and time spent in the state of disability.

As the foundation for this work, we consulted with patient partners, LTC resident councils and patient and family advisory councils to select resident outcomes for prediction that are meaningful and help guide decision making. Together we have selected (1) complete dependence for activities of daily living, (2) inability to communicate, remember or make decisions and (3) time alive in these states of disability.

Our objective is to derive and validate time to event models for LTC residents who experience (1) complete dependence for all activities of daily living, (2) permanent loss of all cognitive abilities required to communicate, interact and remember and (3) time alive in complete dependence for those who entered the outcome state starting from the new index date. Our models will be used to support care planning, prognostication and medical decision-making. This protocol and the reporting of the model will follow the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement for the transparent reporting of multivariable prediction models for individual prognosis.6

MethodsPatient and public involvement

Patient and public involvement was ensured by engaging caregivers of LTC residents in the design and conduct of the study, and selection of the outcome measures. To identify meaningful research questions and outcomes, we worked with caregivers of LTC residents in focus group meetings. Caregivers were invited to research meetings to discuss the study design and research methods and will continue to contribute to the ongoing discussions that will shape the course of our study. We will review the study findings to gain their perspectives and ensure that our research is applicable in long-term care settings and aligns with what they value and supports decision making.

Data sources and cohort creation

We will use population-based secondary health data from all LTC homes in Ontario, Canada to construct the derivation cohort and internal validation cohort. We will construct our derivation and internal validation cohort using data from the ICES (formerly known as the Institute for Clinical Evaluative Sciences). ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyse healthcare and demographic data, without consent, for health system evaluation and improvement. For this study, we will use the Continuing Care Reporting System (CCRS), and the Registered Persons Database (RPDB). These datasets are linked at ICES using unique encoded identifiers and will be analysed at ICES. Data for all predictors and outcomes including demographics, comorbidities, care-related data and functional measures will be obtained from the Resident Assessment Instrument-Minimum Data Set 2.0 (RAI-MDS 2.0) assessment in the CCRS. The RAI-MDS 2.0 is a comprehensive assessment with more than 160 items, which are categorised into domains including cognitive patterns, physical function and structural problems, psychosocial well-being, disease diagnosis, etc.7 The RAI-MDS 2.0, along with the questions designed for each domain, is well-validated in LTC settings and demonstrates high data quality and consistency across assessments.8–12 It is administered to every LTC resident on admission, every 3 months and when there is a change in their medical status. Deaths will be captured using provincial vital statistics data in the RPDB, which is updated monthly. Participants will be followed until death or the end of the study, whichever occurs first. External validation will be conducted within a cohort of LTC residents from Alberta, Canada, within the same timeframe. This province also has access to RAI-MDS 2.0 from their provincial data warehouse within Alberta Health Services. The final predictive model will similarly be assessed for measures of predictive accuracy, discrimination and calibration in the Alberta validation cohort to test model fit.

We will derive the model using the first 4 years of accrual data from newly admitted residents between April 2013 and March 2016 in all LTC homes in Ontario. We will then validate the model using the remaining 2 years of accrual data, residents admitted from 2017 to 2018. All residents are followed for up to 5 years, with follow-up data available through 31 March 2023, which represents the most recent data. This temporal split approach simulates a prospective validation and ensures that the model is tested at future time points, reflecting time-varying changes and trends that may not be seen in the derivation cohort. Although splitting does not use all available data for derivation, this is not a concern in our large (>1 00 000) population-based sample. Additionally, we will consider using a stratified approach to allow the model to be tested and validated in meaningful subgroups (ie, sex and age). We will use all data to fit the final model, which will be externally validated in the Alberta cohort. The planned start date for this study is 1 February 2024, and the anticipated end date is 1 February 2026.

Participants

We will identify an incident cohort of LTC residents aged 65 years or older newly admitted between 2013 and 2018 allowing up to a 5-year follow-up period for all residents. We will exclude individuals who met any of the following criteria: invalid date of birth; non-Ontario residents and residents with the outcomes of interest on admission to LTC.

Outcomes

Outcomes for prediction will be (1) time until complete physical dependence, (2) time until entering a state of severe cognitive impairment, (3) time alive with complete physical dependence for those who entered the outcome state starting from the new index date and (4) time alive with complete cognitive dependence for those who entered the outcome state starting from the new index date. Complete physical dependence (Activities of Daily Living Self-Performance Hierarchy score=6) will be defined as total dependence in performing personal hygiene, toilet use, eating and locomotion with no improvement in any domain on any subsequent assessments once the outcome is achieved. Severe cognitive impairment (Cognitive Performance Scale≥5) is defined as being comatose, or severely impaired decision-making skills and total dependence in eating with no improvement in any domain in subsequent assessments once the outcome is achieved. Our patient partners and clinical experts have advised that these are meaningful outcomes that could support decision making. Variables defining each outcome can be found in table 1.

Table 1

Physical and cognitive outcomes

Sample size

Our team’s previous work describing prevalence and incidence of functional and cognitive disabilities in LTC residents during a 5-year follow-up found that the combined derivation and internal validation cohort includes 120 238 participants with an average follow-up time of 2.3 years. At baseline, there were 3390 (2.8%) participants with complete dependence in all activities of daily living, and 10 408 (8.7%) with permanent loss of all cognitive abilities. At the end of the follow-up, there were 19 101 (15.9%) participants with complete dependence in all activities of daily living and 32 426 (26.9%) with permanent loss of cognitive abilities. Our sample will have more than sufficient power to support our predictive models using Riley et al’s sample size calculation method. For outcomes with lower outcome incidence (13.1%), even with a low R2 of 0.05, we will only require 5249 participants.13

Identification of predictors

We have identified predictor variables through a literature review of factors associated with loss or decline of physical and cognitive function in institutionalised older adults,14–19 the team’s previous work in developing predictive models for life expectancy20–22 and subject-matter expertise. LTC residents and care partners are actively involved in this process to ensure our predictor variables are resident-important. No prespecified predictors were given more importance than others to avoid introducing bias into the model. Each predictor has been mapped to data elements available in the RAI-MDS 2.0. Variables will be excluded if they meet any of the following criteria: more than 20% missingness; and variables with high multicollinearity. Multicollinearity will be assessed using variance inflation factors. We identified 50 variables to predict the loss of independence in physical function: three sociodemographic, one home-level characteristic, 42 health characteristics and four care characteristics (online supplemental table A1). We have identified 58 variables to predict the loss of independence in cognitive function: three sociodemographic, one home-level characteristic, 54 health characteristics and six care characteristics (Table A2). We have identified 36 variables to predict the time spent in the state of dependence: three sociodemographic, one home-level characteristic, 29 health characteristics and three care characteristics (Table A3). Lists of predictors are in online supplemental file 1. All predictors have been prespecified to avoid type 1 error. We will consider including interaction terms with age to improve model performance for subgroups.

Continuous variables will be examined using histograms and box plots to visualise data distribution. Continuous predictors with highly skewed distributions will be log-transformed. Descriptive analysis will be used to examine missing values. Invalid values will be set as missing. Categorical, ordinal and binary variables will be examined using frequency tables and bar graphs. When coding variables, we will avoid categorisation to maximise retention of information and avoid loss of detail and introduction of bias that can occur with this approach.23

Missing data

We will use simple imputation to impute missing data to facilitate implementation in settings that may also have missing data. Given that this is an admission cohort, missingness will be near zero, we will impute the mean, median or mode from the population, to avoid loss of statistical power and prevent introducing bias using traditional complete cases analysis.24 For each model, we will exclude cases if the outcome of interest is missing.

Model estimations

We will use Fine-Gray subdistribution hazard models to predict the time until permanent loss of independence in physical or cognitive function with death as a competing risk. Competing risks are events that prevent or change the possibility of outcome occurrence.25 26 Fine-Gray models account for competing risk of death using the cumulative incidence function, which estimates the probability of the event occurring over time, unlike traditional Cox proportional hazard models, which treat competing risks as censoring events. Fine-Gray models will provide an accurate risk estimation when analysing clinical outcomes in the long-term care population with high morbidity and mortality. We chose the Fine-Gray model to predict the time until permanent loss of independence because (1) this approach provides a better estimation of risk compared with the Kaplan-Meier and Cox models, particularly when competing risks are present,27 (2) Fine-Gray models have been previously demonstrated to perform well for clinical prediction tasks.28–31

We will then use Cox proportional hazard models to predict time spent in each state of dependence among those who entered the outcome state starting from the new index date with death as the outcome. The Cox proportional hazards model is a widely used approach for time-to-event analysis. We chose Cox proportional hazard models because death is our direct outcome and it allows us to model the association between predictors and the instantaneous risk of death over time. This enables us to study the duration spent in states of permanent impairment.

The model assumptions for each predictor variable will be assessed using stratified plots of the log cumulative hazards and by generating and reviewing raw and smoothed scaled Schoenfeld residuals expected to vary by time.

Model specification

We will fit separate models for each outcome using prespecified forms of the predictor variables. To avoid assumptions about predictor-outcome relationships, age will be modelled using 5-knot restricted cubic splines as recommended by Harrell.32 Restricted cubic splines are cubic functions of the predictor between knots placed at quantiles of the distribution. The functions are equal at the knots, to avoid discontinuity and are linear above the uppermost and below the lowermost knot. Knots will be placed at 5th, 27.5th, 50th, 72.5th and 95th centile. An initial degree of freedom will be allocated for each predictor variable, with 20 (4k−p) for age and k−1 for categorical variables. Ordinal variables will be coded as either linear or categorical depending on their expected association with the outcomes. The initial models prespecified in Tables A1–A3 include a total of 112 df for physical function outcome prediction, 146 df for cognitive function outcome prediction and 84 df to predict time spent at the state of dependence.

Assessment of predictive performance

Predictive performance will be assessed in both derivation, and validation cohorts using overall measures of predictive accuracy, discrimination and calibration. For the derivation cohort, all predictive performance measures will be reported after bootstrap optimism correction. Accuracy will be measured using Nagelkerke’s R2 and Brier score. Discrimination will be assessed using Harrell’s concordance statistic (c-stat), by calculating the ratio of correctly ordered pairs to all possible pairs and time-dependent receiver operating characteristic (ROC) curves. Time-dependent ROC curves will be computed from censored data to visually assess how well the model discriminates between those who do and do not experience the outcomes at specific time points. Calibration will be assessed using the Integrated Calibration Index and observed versus expected plots with locally estimated scatterplot smoothing. We will compare measures of fit and calibration in meaningful risk groups including institutional level (eg, for-profit status, size and rurality) and resident level characteristics (eg, age and sex). Differences between observed and expected across risk groups and calibration slope will be tested with Wald or likelihood ratio tests.

Estimation of the final models

We will fit a fully prespecified model as described above. Next, we will use the stepdown approach described by Ambler et al and Harrell et al to simplify our model without sacrificing predictive performance.32 33 The approach involves determining the contribution of each predictor to the model’s R2. Predictors with the least contribution to the R2 are removed sequentially until removing the next one would drop the model’s R2 to less than 95% of the original model’s R2. We will select the final model (either the full or reduced model) based on calibration slopes and calibration in specific subgroups (male and female sex). If discrimination and calibration are similar in the validation cohorts, we will combine the derivation and validation cohorts to estimate final regression coefficients. This will allow us to maximise follow-up time for our outcomes.

Model presentation

We will present results for the derivation, validation and combined cohorts. We will present beta estimates, hazard ratios and 95% CIs. We will show the estimated effect of each predictor variable using graphs (eg, predicted versus actual plots and coefficient plots). We will present the final model as a regression formula. We will also create a web-based calculator to visualise the model output in consultation with our partners and experts to create output that is interpretable and supports residents, caregivers and physicians in making decisions.

Risk groups

Risk groups will be created to facilitate meaningful interpretation of results. The regression formulas will generate individual risk scores as an estimated measure of individual risk. We will calculate risk scores for all people in the cohort and then create 30–40 risk groups based on quantiles of risk scores. For each risk group, we will present the median and IQR of time until or in the outcome.

In conclusion, our predictive model will support medical decision making by focussing on supporting residents and families in recognising the trajectory of illness and providing risk estimates that could eliminate futile interventions, facilitate useful interventions and encourage resident-centred treatment decisions.

Ethics and dissemination

Ethics approval was obtained through Bruyère Research Institute Ethics Committee on 21 June 2023 (REB#M16-23-030). Study findings will be disseminated through conferences, seminars and in published journals. The predictive algorithm will be publicly available for the general population.