Development and validation of a hospital frailty risk measure using Canadian clinical administrative data [Research]

Abstract

Background: Accessible measures specific to the Canadian context are needed to support health system planning for older adults living with frailty. We sought to develop and validate the Canadian Institute for Health Information (CIHI) Hospital Frailty Risk Measure (HFRM).

Methods: Using CIHI administrative data, we conducted a retrospective cohort study involving patients aged 65 years and older who were discharged from Canadian hospitals from Apr. 1, 2018, to Mar. 31, 2019. We used a 2-phase approach to develop and validate the CIHI HFRM. The first phase, construction of the measure, was based on the deficit accumulation approach (identification of age-related conditions using a 2-year look-back). The second phase involved refinement into 3 formats (continuous risk score, 8 risk groups and binary risk measure), with assessment of their predictive validity for several frailty-related adverse outcomes using data to 2019/20. We assessed convergent validity with the United Kingdom Hospital Frailty Risk Score.

Results: The cohort consisted of 788 701 patients. The CIHI HFRM included 36 deficit categories and 595 diagnosis codes that cover morbidity, function, sensory loss, cognition and mood. The median continuous risk score was 0.111 (interquartile range 0.056–0.194, equivalent to 2–7 deficits); 35.1% (n = 277 000) of the cohort were found at risk of frailty (≥ 6 deficits). The CIHI HFRM showed satisfactory predictive validity and reasonable goodness-of-fit. For the continuous risk score format (unit = 0.1), the hazard ratio (HR) for 1-year risk of death was 1.39 (95% confidence interval [CI] 1.38–1.41), with a C-statistic of 0.717 (95% CI 0.715–0.720); the odds ratio for high users of hospital beds was 1.85 (95% CI 1.82–1.88), with a C-statistic of 0.709 (95% CI 0.704–0.714), and the HR of 90-day admission to long-term care was 1.91 (95% CI 1.88–1.93), with a C-statistic of 0.810 (95% CI 0.808–0.813). Compared with the continuous risk score, using a format of 8 risk groups had similar discriminatory ability and the binary risk measure had slightly weaker performance.

Interpretation: The CIHI HFRM is a valid tool showing good discriminatory power for several adverse outcomes. The tool can be used by decision-makers and researchers by providing information on hospital-level prevalence of frailty to support system-level capacity planning for Canada’s aging population.

Health systems need to make concerted efforts to strengthen care for their aging populations.1 A particular challenge that jurisdictions face is in the identification and management of age-related conditions, including frailty. Frailty is a health state characterized by increased vulnerability to stressors and a high risk of adverse health-related events.2 People living with frailty often have lower quality of life and reduced functional independence, which can lead to a variety of negative outcomes.37 Tools that allow for the identification and measurement of frailty are needed to help health systems address this challenge. Various jurisdictions have begun to use administrative health data to identify and prepare for the increasing number of older adults living with frailty.811 A common approach has been to use a frailty index, which measures the proportion of deficits present in an individual out of the total number of age-related health variables considered.1214

Although several frailty measures have been developed, 811,1518 no standard measure using routinely collected administrative data from patients admitted to acute care hospitals has been available in Canada. In response, the Canadian Institute for Health Information (CIHI) sought to develop the Hospital Frailty Risk Measure (HFRM).19 The aim of this paper is to describe the development and validation of the CIHI HFRM.

MethodsStudy design

The CIHI HFRM is based on the deficit accumulation approach1214 and modelled on existing frailty measures.810,16 We developed the measure using retrospectively identified cohorts of patients in Canadian hospitals. We used a patient’s total number of age-related health deficits to determine their risk of frailty.

We developed the CIHI HFRM in 2 phases. Phase 1 involved construction of the measure, including identification of age-related conditions. Phase 2 involved refinement of the measure and assessment of its predictive validity. Throughout development, consultations were held with an expert group — comprising 9 Canadian and international clinical, policy and research experts knowledgeable about frailty (including D.B.H., C.J.M., J.M., K.R., S.S., O.T.), as well as Canadian coding specialists — to inform development and address content validity.20

In Phase 1, we iteratively constructed the preliminary measure by identifying and refining available age-related conditions and underlying diagnoses. We performed a literature review to assess existing frailty measures used in multiple care settings (Appendix 1, Appendix A1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.220926/tab-related-content). We identified potential conditions and diagnostic codes and examined them in consultation with the expert group, informed by a standardized procedure for developing a frailty index (e.g., statistical analyses evaluating condition prevalence in data, correlation with age)21 and for assessing coding practices across Canadian jurisdictions over time. The expert group reviewed the condition list for clinical relevance and content validity. The intent of this phase was to identify a minimum of 30 deficits from wide-ranging domains that included the presence of select morbidities, functional impairment, sensory loss, cognitive issues and mood disorders.10,21 The expert group selected the final list of conditions by consensus.

Phase 2 included refinement of the measure to establish meaningful risk groups and evaluation of construct, convergent and predictive validity (e.g., comparisons with existing measures and ability to predict frailty-related outcomes).2224

We designed the CIHI HFRM to report on risk of frailty in 3 formats to ensure usability for a diverse group of stakeholders, including as a continuous risk score ranging from 0 to 1, which is most suitable for health researchers; as membership in risk groups, for use by health system planners and administrators targeting smaller at-risk groups; and as a binary risk measure (i.e., the percent of older patients discharged from hospital who were at risk of frailty v. not at risk), which can be used as a first-step crude measure for overall health system planning and monitoring.

We report this study according to the Reporting of Studies Conducted Using Observational Routinely-collected Health Data (RECORD) Statement checklist.25

Population

We included data from patients aged 65 years and older who were discharged alive from all Canadian teaching and community acute care hospitals (as defined in CIHI databases) from Apr. 1, 2018, to Mar. 31, 2019, to develop this measure.26 For patients with multiple discharges, we used the last discharge as the index case. We linked patient records over a 2-year look-back period (to fiscal year 2016/17) to identify deficits, as a 2-year period was shown to increase the accuracy of frailty risk assessment and predictive ability compared with a 1-year period.27

Since frailty is predominantly seen among older adults, we designed the CIHI HFRM to target patients aged 65 years and older. Restriction to an older population is consistent with most frailty indices, as the underlying mechanism for development of frailty may differ in younger populations.9,28

To determine the discriminatory power of the CIHI HFRM, we used patients from the development cohort to predict future adverse health outcomes. For this, we linked the cohort to future hospital visits (for the full cohort from Phase 1) and new admissions to long-term care (LTC) (for a subset of the cohort from jurisdictions where LTC data were available) in fiscal years 2018/19 and 2019/20. To ensure that LTC admissions were new, we excluded patients with LTC admissions in the previous 5 years (back to fiscal year 2013/14).

Data sources

The CIHI HFRM uses Canadian administrative health data that are routinely submitted to CIHI. Analytical staff from CIHI had full access to the data. We constructed the development cohort using inpatient records from the Discharge Abstract Database (DAD) and Hospital Morbidity Database (HMDB),29,30 which offer complete coverage of all acute care hospital discharges for all provinces and territories across Canada. We used the National Ambulatory Care Reporting System (NACRS) and all DAD-HMDB records to identify patients’ frailty-related conditions, which offer complete coverage of inpatient hospitalizations and day surgeries for all provinces and territories across Canada, as well as partial coverage of visits to emergency departments and to outpatient and community-based clinics in Ontario, Alberta, Prince Edward Island, Nova Scotia, Saskatchewan and Yukon.31 Codes from the Canadian version of the International Classification of Diseases, 10th Revision (ICD-10-CA) are used to capture diagnoses in these databases.32 We used the Continuing Care Reporting System to support validation for a subgroup of acute care patients who used LTC services in Alberta, British Columbia, Newfoundland and Labrador, Ontario and Yukon.33 Finally, we identified deaths that occurred after the index hospital admission and other hospital-based outcomes such as readmissions using the DAD and NACRS databases; thus, we did not identify out-of-hospital deaths. As shown in reports of database quality and re-abstraction studies, CIHI ensures high quality of the information in its data holdings.2931,33

Statistical analysisPhase 1: Identification of frailty conditions

We used a standardized procedure developed by Searle and colleagues21 to guide the inclusion of frailty deficits. We used a cut-off (> 1% prevalence) to determine whether certain conditions or categories would be kept. Since some ICD-10-CA diagnoses are highly specific and interrelated, we used factor analysis (with factor loading > 0.5) to group similar diagnostic codes together into relevant frailty deficit categories (Appendix 1, Appendix A2).34 We also plotted prevalence against age to filter out conditions that saturated too early (a criterion for frailty indices).21

Phase 2: Refinement and validation

We developed and refined 3 formats of the CIHI HFRM. We calculated the continuous risk score for each patient by dividing the number of deficits present by the maximum number considered.

To develop the risk groups format, we explored several ways of grouping the continuous measure into meaningful risk groups (e.g., using results obtained to create 5, 7 or 8 risk groups from lowest to highest values) and assessed the discriminatory ability (C-statistic) of the groupings in predicting poor outcomes. We examined overlapping confidence intervals (CIs) of odds ratios (ORs) and hazard ratios (HRs) to assist in collapsing adjacent risk groups. We chose the final risk groups based on the expert group’s assessment of which groups had the strongest discriminatory power and meaningful actionability.

For the binary risk measure, we used the stratum-specific likelihood ratio (SSLR) method to determine the cut-off point to identify those at risk of frailty. This method has been previously applied to frailty indices to identify risk groups with significant differences in the likelihood of adverse outcomes.23 These ratios represent the likelihood that patients in each frailty stratum (e.g., number of deficits) will have an adverse outcome (relative to their likelihood of not having such an outcome).35 We conducted separate SSLR analyses for each outcome. We used the stratum with an SSLR exceeding 1.0 that was significantly different (at p < 0.05) from the preceding stratum for all outcomes as the cut-off point.

We evaluated the association of the CIHI HFRM continuous risk score with the United Kingdom Hospital Frailty Risk Score (HFRS)9 using the Pearson correlation coefficient to assess convergent validity. In addition, we tested the predictive validity of the CIHI HFRM on a range of frailty-related adverse health outcomes (Box 1); we also compared these results with those of the UK HFRS. These outcomes included death and subsequent health service use, which we selected because they are often reported in the frailty literature and have direct relevance to resource planning.4,811,15

Box 1: Frailty-related adverse health outcomes for predictive modelling

In-hospital death*: All-cause death within 1 year of the index hospital discharge (in-hospital deaths available only).

Hospital readmission: 30-day and 1-year readmission to an inpatient hospital after the index hospital discharge.

Long length of stay (LOS)§: Inpatient hospitalization with cumulative LOS of 30 days or more within 1 year of the index hospital discharge.

High users of hospital beds: Patients with 3 or more hospitalizations with a cumulative LOS of more than 30 days, within 1 year of the index hospital discharge.

New admission to long-term care (LTC)**: 90-day or 1-year new admission to long-term care (LTC) after the index hospital discharge (i.e., no previous LTC admissions within 5 years of the most recent hospital index discharge).

* Methodology derived from Canadian Institute for Health Information (CIHI)’s 30-Day Acute Myocardial Infarction In-Hospital Mortality indicator.36

† Index hospital discharge is the last hospital discharge in the index fiscal year; the index and subsequent fiscal year is used to identify future outcomes of interest.

‡ Methodology based on CIHI’s All Patients Readmitted to Hospital indicator.37

§ Methodology based on that used by Gilbert and colleagues.9

¶ Methodology based on CIHI’s High Users of Hospital Beds indicator.38

** Methodology derived from CIHI’s New Long-Term Care Residents Who Potentially Could Have Been Cared for at Home indicator.39

Outcomes and variables

For in-hospital death, only deaths in DAD-HMDB and NACRS were available for analysis. For all other models, we removed or accounted for death as a competing risk.40,41 We used logistic regression models and Cox proportional hazards models to determine the risk of these outcomes, adjusting for the patient’s age, sex and Charlson Comorbidity Index.42 We added the latter as a categorical variable (i.e., 0–1 as Charlson group 1; 2–4 as Charlson group 2; and ≥ 5 as Charlson group 3), and included it in models to validate that the CIHI HFRM was a measure of frailty rather than a comorbidity score.43 We conducted a sensitivity analysis by fitting models without the Charlson variable and calculated its variance inflation factor to ensure it could be included in the same model.

Missing data

Acute inpatient and day surgery institutions are mandated by all provincial and territorial ministries to submit data to CIHI’s clinical administrative databases. The record-level non-response rate was 0.003% for the DAD-HMDB in 2018/19. Item nonresponse for core mandatory data elements is typically less than 0.1%.29,30 We did not perform any imputation as the percent of missing mandatory elements (including diagnosis codes) was less than 5%, and the proportion of non-mandatory diagnosis codes is unknown. In addition, we did not perform any imputation for jurisdictions that do not submit NACRS data, since the underlying mechanism for missingness is unknown, without a viable imputation method that could be validated.

We performed all analyses in SAS version 9.4.

Ethics approval

The Canadian Institute for Health Information receives health data from all jurisdictions. Its use of these data does not require review by a research ethics board.

ResultsPhase 1: Identification of frailty conditions

A cohort of 788 701 patients aged 65 years or older was available to develop the CIHI HFRM (Table 1). Using an initial list of 55 conditions (Appendix 2, Appendix B1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.220926/tab-related-content), we redefined, regrouped or deleted groups of deficits according to expert feedback, prevalence rates and factor analysis results. The final CIHI HFRM includes 36 deficit categories and covers a wide range of frailty-related deficits including morbidity, function, sensory loss, cognition and mood (Box 2). The final list of deficits with corresponding 595 ICD-10-CA codes is available in Appendix 2, Appendix B2.

Table 1:

Characteristics and outcomes of the Canadian Institute for Health Information Hospital Frailty Risk Measure cohort

Box 2: The 36 condition categories* of the final Canadian Institute for Health Information Hospital Frailty Risk Measure

Morbidity

Anemia

Cardiac

Cerebrovascular

Diabetes

Gastrointestinal

Hypo- and hypertension

Incontinence

Renal

Respiratory

Thrombosis and embolisms

Function

Instrumental activities of daily living

Arthritis and inflammation

Movement and immobility

Fatigue

Functional dependence

Fractures and osteoporosis

Musculoskeletal

Machine dependence

Edema

Sensory loss

Cognition and mood

Delirium

Delusions and hallucinations

Dementia and Alzheimer disease

Other cognitive disorders

Mood disorders

Other conditions

Endocrine

Epilepsy

History of medications

Infections

Nutrition and wasting

Pain

Organ transplants and ostomies

Other frailty conditions and diseases

Other injuries

Ulcers and soft tissue disorders

Phase 2: Refinement and validation

Eight groups were included in the final version of the risk group format of the CIHI HFRM (Table 2).

Table 2:

Characteristics of 8 risk groups of the Canadian Institute for Health Information Hospital Frailty Risk Measure

For the binary risk measure, a cut-off point of 6 deficits had SSLRs exceeding 1.0 that were significantly different from the preceding stratum for all outcomes (Appendix 1, Appendix A3). Therefore, we chose a cut-off of 6 or more deficits (or risk score ≥ 0.167) for those considered to be at risk of frailty.

Among the cohort, the median continuous risk score was 0.111 (equivalent to the presence of 4 deficits) (Table 3), and 35.1% were at risk of frailty using the binary risk measure. The scores ranged from 0 to 0.778 (maximum of 28 deficits), with the distribution heavily skewed to the right (Appendix 1, Appendix A4). Higher frailty scores were seen with increasing age, and scores were similar for males and females (Table 4). The median frailty risk score was consistently higher among patients who had adverse outcomes.

Table 3:

The 3 formats of the Canadian Institute for Health Information Hospital Frailty Risk Measure

Table 4:

Descriptive statistics for the continuous risk score and 8 risk groups, by characteristics and outcomes

As with the continuous risk score, the proportion of patients with poor outcomes increased with increasing levels of risk in the 8 risk groups format and in the at-risk group of the binary risk measure. Comparable patterns were found for all 3 formats of the CIHI HFRM for all outcomes for a subset of patients admitted to LTC (Appendix 1, Appendices A5–A7).

We observed a strong positive correlation between the CIHI HFRM’s continuous risk score and the UK HFRS (Pearson correlation coefficient 0.829, 95% CI 0.828–0.829). A scatter plot confirmed linear correlation (Appendix 1, Appendix A8). The CIHI HFRM’s continuous risk score had similar discriminatory power as the UK HFRS in predicting health outcomes, with slightly higher C-statistics for hospital-based events and slightly lower C-statistics for LTC admissions (Appendix 1, Appendix A9).

When examining the predictive ability for frailty-related adverse outcomes, the models for all 3 formats of the CIHI HFRM had reasonable discriminatory power based on C-statistics. The inclusion of other characteristics (age, sex and Charlson group) improved discrimination for all 3 formats of the measure for most outcomes and showed that higher frailty risk was predictive of poor outcomes (Table 5, Table 6, Figure 1, Figure 2; Appendix 1, Appendices A10–A13). Sensitivity analysis showed that the CIHI HFRM offered an additional contribution independent of Charlson group and was a more powerful tool for prediction (Appendix 1, Appendices A14–A16). The variance inflation factor for Charlson group was 1.11, indicating minimal concern for collinearity.

Table 5:

Unadjusted and adjusted odds ratios (ORs) and hazard ratios (HRs) for the continuous risk score (unit = 0.1)

Table 6:

C-statistics for unadjusted and adjusted logistic regression and Cox proportional hazards models for continuous risk score, 8 risk groups and binary risk measure

Figure 1:Figure 1:Figure 1:

Regression results for 8 risk groups predicting (A) in-hospital death within 1 year; (B) 30-day- and (C) 1-year hospital readmission; (D) high users of hospital beds (≥ 3 hospitalizations, with a cumulative length of stay [LOS] > 30 d); and (E) long LOS (≥ 30 d). Note: CI = confidence interval, Ref. = reference category. Odds ratio (OR) reported for logistic regression models; patients who died upon discharge excluded. Hazard ratio (HR) reported for Cox proportional hazards models. All models were adjusted for sex, age and Charlson comorbidity group.

Figure 2:Figure 2:Figure 2:

Regression results for 8 risk groups predicting (A) 90-day- and (B) 1-year new admission to long-term care. Note: CI = confidence interval, Ref. = reference category. Hazard ratio (HR) reported for Cox proportional hazards models. Models were adjusted for sex, age and Charlson comorbidity group.

Table 6 shows that the continuous risk score performed reasonably well, especially for predicting in-hospital death (C-statistic 0.717, 95% CI 0.715–0.720) and high use of hospital beds (C-statistic 0.709, 95% CI 0.704–0.714). It did not perform as well for predicting hospital readmission (C-statistics for 30-d and 1-yr readmission were 0.660, 95% CI 0.656–0.665, and 0.621, 95% CI 0.620–0.622, respectively). The format using 8 risk groups discriminated best for predicting a new LTC admission (C-statistics for 90-d and 1-yr admission were 0.815, 95% CI 0.812–0.817, and 0.802, 95% CI 0.799–0.804, respectively).

The format using 8 risk groups had similar discriminatory ability to that of the continuous measure format. The binary risk measure had the weakest discriminatory ability among the 3 formats.

Interpretation

We used a large national cohort of older patients who had been discharged from an acute care hospital to develop the CIHI HFRM, which includes 36 deficit categories that cover domains of morbidity, function, sensory loss, cognition and mood. The CIHI HFRM has good predictive validity and reasonable goodness-of-fit for a range of frailty-related adverse health outcomes, such as death, readmission, high use of hospital beds and admission to LTC. The CIHI HFRM, which describes the level of frailty in hospitals, can be used to support system-level capacity planning for Canada’s aging population. For example, it can inform service provision for health care organizations (e.g., models of care for intake and assessment, programming, staffing, discharge and end-of-life planning; estimation of need for specialized geriatric services) and support research and quality improvement initiatives.

Although most frailty indices present a continuous risk scale with few cutpoints, the CIHI HFRM has 3 formats, which helps to ensure its utility for diverse stakeholder groups, including health system planners and decision-makers, care delivery managers and researchers. The binary risk measure, despite its slightly weaker discriminatory abilities, can be used as a first-step crude measure for overall health system planning and monitoring. For health care managers wishing to target smaller at-risk groups, the format using 8 risk groups is useful. The continuous risk score is the most granular measure and is likely most suitable for health researchers.

Another advantage is its accessibility.44 Similar scales using this approach are critiqued for their complexity, making them difficult to replicate.6,7,45 The CIHI HFRM results are pre-calculated based on routinely collected data and updated on an ongoing basis at the national, jurisdictional and hospital levels, readily available on the CIHI website (https://www.cihi.ca/en/frailty-among-hospitalized-seniors).44 At a system level, this provides a benefit compared with frailty tools that require additional and potentially costly data collection. This consideration includes the Clinical Frailty Scale, which typically requires both individualized evaluation of the patient and introductory training for health care practitioners in classifying patients onto a 9-point scale.22,46

The discriminatory ability of the CIHI HFRM is comparable to other validated measures of frailty that are considered good predictors of future health outcomes, such as death and institutionalization. 4,811,15 Although several approaches have been proposed, a review found that a deficit accumulation approach results in higher agreement with comprehensive geriatric assessments. 47 Compared with the UK HFRS, the CIHI HFRM showed similar discriminatory performance. However, the UK HFRS performed slightly better in discriminating LTC admissions, which may be owing to higher weights given to conditions like dementia. The CIHI HFRM adopted an equal-weight algorithm for each condition included in the measure, as derivation of weights is data-driven and may lack transferability to different patient populations. The equal-weight algorithm aligns with usual practice in deficit accumulation models and is arguably more easily transferrable to international populations, as weights are likely country-specific and not readily available.

Although the UK HFRS (developed using patients aged ≥ 75 yr) has been used in Canada,48,49 the CIHI HFRM was developed and validated using a Canadian cohort with a broader age band (including patients aged 65–74 yr, an age group that represented more than 40% of the older inpatient population in our study and who potentially have a different pattern of frailty-related conditions than older patients), and considered Canadian-specific coding practices and standards. Given health system differences between Canada and the UK, particularly in diagnostic coding practices and standards,32 as well as age differences in the development cohort, the CIHI HFRM is better suited to characterize the risk of frailty in Canada for this population.

We are planning a second publication to describe further validation of the CIHI HFRM against comprehensive geriatric assessments that are considered the reference standard in clinical practice.22,50 In addition, the recent adoption of the 11th edition of the ICD by the World Health Organization has implications for the measure. This new coding standard will likely be implemented into CIHI databases in the future. When this happens, all ICD-10-CA codes and associated measures using these codes (such as the CIHI HFRM) will be mapped and updated to reflect necessary changes. This is part of CIHI’s ongoing work to assess and implement ICD-11 for health system use in Canada.

Limitations

Regional differences in collection and coding practices limit the ability to compare across jurisdictions. Not all regions participate in the collection of data for NACRS, which can be used as an additional data source to identify age-related conditions. The inclusion of NACRS data provides a better approximation of the risk of frailty, as additional analysis showed that using DAD-HMDB data alone underestimated the level of frailty among patients admitted to hospital (Appendix 1, Appendix A17). This means that, for jurisdictions that do not participate in NACRS, the level of frailty based on the CIHI HFRM is likely underestimated and cannot be compared with NACRS-participating jurisdictions; CIHI HFRM results reflect the minimum level of frailty that needs to be planned for.

Missing data, in the form of frailty deficits not captured in patient records, may be present, as some diagnostic codes included in the CIHI HFRM are not mandatory for reporting. Since the level of missing non-mandatory diagnosis coding cannot be estimated, we did not employ any imputation or other techniques to account for potential missing nonmandatory codes. Cognitive and functional impairments are also poorly documented in hospital-based databases,51 which can lead to underestimation of true frailty risk. Improved data collection and coding may improve the utility of the measure.

Recent research has shown that use of ICD-10 codes from administrative data to delineate frailty has resulted in underestimation, 52 but this measure has value at the system level since information on frailty levels across hospital systems is currently lacking. Although trends in mortality rates have been shown to be the same regardless of whether in-hospital deaths or all deaths are measured, we did not capture out-of-hospital deaths.53 Thus, some misclassification in the predictive models of death is present; for other predictive models, we could not remove or account for out-of-hospital death as a competing risk. Finally, the CIHI HFRM does not replace a comprehensive geriatric assessment in the provision of clinical care and is not designed for use by clinicians for assessment and care of individual patients.

留言 (0)

沒有登入
gif