WHAT IS ALREADY KNOWN ON THIS TOPICWHAT THIS STUDY ADDS

In this national, parallel, cluster-randomised trial including 40 hospitals and 155 362 patients, an intervention using control charts to monitor surgical outcomes paired with feedback to surgical teams was associated with significant reductions in hospital bed-days per patient within 30 days following surgery, and hospital costs reimbursed for this care per patient by the insurer.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICYIntroduction

Many countries devote a substantial proportion of their financial resources to healthcare expenses. Hospitals often represent the largest spending category, accounting for almost 40% of total healthcare expenditures on average in Organisation for Economic Co-operation and Development countries.1 For this reason, stakeholders, including hospitals, oversight agencies and policy makers, are coming together in an effort to improve healthcare quality and safety while reducing costs.2–4 Surgical complications are a major cause of increased hospital expenses.5 6 Therefore, any perioperative strategy that decreases complications would significantly reduce associated healthcare costs.6

Quality improvement methodologies have been developed over the last century in the manufacturing sector to reduce variation and error, increase the reliability of production processes and improve product quality, while also reducing costs.7 The adaptation and implementation of these approaches in a variety of healthcare environments has become increasingly common over the last several decades.7–10 However, the economic effects of these interventions remain unclear as few high-quality studies have evaluated the potential economic benefits for healthcare systems.9 Control charts are a specific measurement tool from the quality management sciences that have shown broad applicability in healthcare, including for monitoring adverse events.11 These graphic tools enable users to monitor the behaviour of an indicator, analyse the variations and ultimately implement corrective actions to reduce variability and improve quality.12 13

The findings of the SHEWHART trial showed that the implementation of an intervention using control charts paired with feedback on indicators to surgical teams was associated with reductions in major adverse events and patient death.14 Ideally, in order to increase value in healthcare, interventions should improve health outcomes, and avoid increasing healthcare costs.15 Because the potential impact of control charts on hospital resource consumption remains unknown, we aimed to prospectively evaluate the effects of implementing surgical outcome monitoring using control charts on hospital bed-days per patient within 30 days following surgery, and hospital costs reimbursed for this care per patient by the insurer.

MethodsStudy design and participants

We performed an analysis using data from the SHEWHART trial, which has been described at length (protocol available at http://shewhart.univ-lyon1.fr).14 The SHEWHART trial was a nationwide parallel cluster-randomised trial, designed to determine the impact of using control charts to monitor surgical outcomes on patients’ major adverse events. Following a 2-year pre-implementation period (1 January 2014–31 December 2015), participating hospitals were randomised into two cluster groups by the health data department of Hospices Civils de Lyon using a computer-generated randomisation schedule. Because this trial concerned an open-label intervention and involved local investigators, it was not possible to mask hospital staff, although patients were masked to study group allocation. Next, during the 2-year implementation period (1 January 2017–31 December 2018), the hospitals allocated to the intervention group began using control charts to monitor surgical outcomes. No specific actions were implemented in the control hospitals. A difference-in-difference approach was used to compare economic outcomes between periods and between intervention and control hospitals to determine whether the intervention improved outcomes.

Forty surgical departments located in distinct hospitals across the country were included in the study on a first-come first-served basis.14 For patients, inclusion criteria were undergoing digestive surgery (hernia repair, cholecystectomy, appendectomy, bariatric, colorectal, hepatopancreatic or oesophageal and gastric surgery) in the participating surgical departments. Patients who were under the age of 18 years, who received inpatient care, who were hospitalised for <24 hours or who received care in the scope of palliative care or organ transplant were excluded.

Intervention

In intervention hospitals, we implemented an intervention based on monitoring surgical outcomes using control charts (details at http://shewhart.univ-lyon1.fr and in the online supplemental appendix).14 For each operative procedure, a set of Shewhart p-control charts, including indicators of postoperative death, intensive care stay, reoperation and severe complications, were provided. In the charts, the data points represented the indicator measure for each quarter, and the central line depicted the mean indicator value for each individual hospital. The control and warning limits were calculated based on binomial distribution, and set at 99.7% (3 SD) and 95.5% (2 SD) around the central line.12 Special cause variation was defined as either a single point outside the control limits, or two of three successive points outside the warning limits. Based on these rules, the risk of false positive was 8.9% for a 20-point Shewhart control chart.16 Thus, special causes were characterised by considerable changes in patient outcomes, attributed to unforeseen phenomena occurring during care delivery that deserved further investigation. The control charts were displayed on posters hung on the wall of the operating room each quarter. Charts were discussed during quarterly team meetings in an effort to improve understanding of variations in surgical outcomes. In the context of deteriorating outcomes, special attention was paid to the identification and resolution of root causes, and specific actions aimed at care improvements were tested.

To facilitate control chart implementation, study champion partnerships consisting of a surgeon and a collaborator (surgeon, anaesthetist or nurse) were established at each site. Pairs were responsible for conducting control chart review meetings and maintaining a logbook where they recorded all modifications to care processes. Furthermore, the pairs from all of the intervention hospitals met during three 1-day trainings held at 8-month intervals. These sessions aimed to provide skills to use control charts appropriately, lead review meetings with effective cooperative and decision-making processes, identify special cause variations and design improvement plans. Compliance of each hospital with the programme implementation was measured as previously described.14 Details can be found in the online supplemental appendix.

Outcomes and data sources

The economic outcomes analysed for this analysis were the number of hospital bed-days per patient within 30 days following surgery, and the hospital costs reimbursed for this care per patient by the insurer. Hospital bed-days included both the index stay associated with the surgery and any acute care readmissions related to the occurrence of major adverse events within the 30 days following the initial surgical procedure, regardless of the hospital. Major adverse events were defined as inpatient death, extended intensive care stay (at least two nights in intensive care or five nights in critical care), reoperation (open or laparoscopic digestive tract procedure) or severe complications (cardiac arrest, pulmonary embolism, sepsis or surgical site infection). Hospital costs reimbursed for this care by the insurer was determined based on the total amount of expenditures reimbursed by France's health insurance system (Assurance Maladie) for each patient stay.

This study was based on data from the French Medical Information System (Programme de Médicalisation des Systèmes d’Information (PMSI), source Agence Technique de l'Information sur l'Hospitalisation (ATIH)). The PMSI is a nationwide database routinely implemented for care reimbursement, and updated weekly with data from all hospitals in France. The data are prospectively collected, and the database relies on a coding system with strict variable definitions. A subset of records is audited on a regular basis to avoid coding errors. Due to its accuracy and exhaustive data collection, no patients were lost to follow-up during the study period. Inpatient stays were recorded as standard discharge abstracts containing compulsory information about the patient and their primary/secondary diagnoses using the International Classification of Diseases, 10th revision, and detailed procedural codes associated with the care provided. From the PMSI, we extracted patient demographics, comorbidities according to the Elixhauser algorithm, emergency admission, date and operative procedure, main diagnosis, surgical procedure complexity, hospital bed-days within 30 days following surgery and hospital costs reimbursed for this care by the insurer.17 Patients’ socioeconomic status (median household income) was based on patients’ residential codes provided by the National Institute of Statistics.

Statistical analysis

We computed multivariable generalised linear mixed models with a log link to estimate the impact of control chart implementation on economic outcomes while accounting for patient clustering within hospitals (see model specification in online supplemental appendix). A negative binomial distribution was used to model the mean number of hospital bed-days per patient within 30 days following surgery, and a gamma distribution was used to model the mean hospital costs reimbursed per patient for this care by the insurer. Adjusted rate ratios (RR) or cost ratios (CR) were defined as the ratio of hospital bed-days or hospital costs, respectively, between implementation (2017–2018) and pre-implementation (2014–2015) periods, and were estimated in intervention and control hospitals. Using a difference-in-difference approach, we used the interaction between study groups and period to estimate the adjusted ratio of rate ratios (RRR) or ratio of cost ratios (RCR) with 95% CIs, which compared changes in outcomes from the pre-implementation with the implementation period between the intervention and control hospitals. A RRR or RCR value <1 indicates that using control charts in the intervention hospitals reduced healthcare consumption compared with control hospitals, after adjusting for temporal variations. For each group and period, we used estimated regression coefficients obtained from these models and the marginal standardisation method to determine the standardised rates of hospital bed-days per patient within 30 days following the surgery and standardised hospital costs reimbursed for this care per patient by the insurer (see details for marginal standardisation method in online supplemental appendix), difference of absolute rates or costs difference and difference of relative rates or costs difference for both outcomes. The corresponding 95% CIs were computed from non-parametric bootstrap based on 1000 replications. The total number of avoided hospital bed-days within 30 days following surgery, and hospital costs reimbursed for this care by the insurer were estimated from the difference of relative rates or costs difference for surgical patients in intervention hospitals during the implementation period.

All of these models were adjusted for potential confounders based on a patient resource consumption score introduced as a categorical variable (quartiles), death status within 30 days following surgical procedure and their interaction. The patient resource consumption score predicted the mean expected amount of resource consumption, and was previously developed from patient data from the randomisation period. A specific resource consumption score was established for each outcome and operative procedure using multivariable generalised linear models with a log link and a negative binomial (for the number of hospital bed-days per patient within 30 days following surgery) or a gamma (for hospital costs reimbursed for this care per patient by the insurer) distribution, considering age, sex, comorbidities, emergency admission, date and operative procedure, main diagnosis, surgical procedure complexity, median household income for patient-level covariates and status for hospital-level covariates.

We conducted a sensitivity analysis by imputing missing household incomes based on the mean value for each study group and period.

P values were two-sided and considered significant if <0.05. Data were analysed using SAS V.9.4 (SAS Institute, Cary, North Carolina, USA).

Results

Among the 40 participating hospitals across the country, 17 (43%) were academic, 14 (35%) were non-profit and 9 (23%) were private-for-profit. Of the 159 688 patients who underwent operative procedures during the pre-implementation and implementation study periods, 97.8% (156 133) were eligible for trial inclusion, of whom 99.5% (155 362) were included in analyses (figure 1). After hospital randomisation, 75 047 patients were included in the intervention group and 80 315 were included in the control group. Patient characteristics between study groups and periods are presented in table 1. Detailed data reporting the compliance of each hospital with the implementation of the programme, and examples of control charts from the study with improvement actions made or corrective actions following a special cause variation can be found in the online supplemental appendix.

Figure 1

Flow chart for the SHEWHART cluster randomised trial. During the study period, 159 688 patients were enrolled in the 40 participating hospitals. A total of 4326 patients (2.7%) were excluded; the final study population included 155 362 patients, of whom 79 127 were assigned to the 2014–2015 pre-implementation period (37 579 patients in intervention hospitals vs 41 548 in control hospitals) and 76 235 to the 2017–2018 implementation period (37 468 vs 38 767).

Table 1

Patient characteristics by study group and period

During the study period, the mean (SD) number of hospital bed-days within 30 days following surgery per patient was 7.0 (6.8) days, and the mean (SD) of hospital costs reimbursed for this care by the insurer per patient was €5209 (€5603). The 40 hospitals consumed a total of 1 084 434 bed-days and spent a total of €809 214 404 (online supplemental table 2). Table 2 shows the changes in economic outcomes from the pre-implementation period to the implementation period for hospitals in the control and intervention groups. The adjusted RR and CR analyses showed that hospital bed-days per patient within 30 days following surgery, and hospital costs reimbursed for this care per patient by the insurer reduced in both the intervention group (RR 0.91; 95% CI 0.90 to 0.92 for hospital bed-days; CR 0.95; 95% CI 0.95 to 0.96 for hospital costs) and the control group (RR 0.94; 95% CI 0.93 to 0.95 for hospital bed-days; CR 0.97; 95% CI 0.96 to 0.97 for hospital costs). However, the adjusted RRR and RCR revealed that, following the introduction of the control charts, both hospital bed-days per patient within 30 days following surgery (RRR 0.97; 95% CI 0.95 to 0.98; p<0.001) and hospital costs reimbursed for this care per patient by the insurer (RCR 0.99; 95% CI 0.98 to 1.00; p=0.01) were significantly reduced in the intervention compared with the control groups. The results from our sensitivity analysis including patients with missing household incomes were consistent with our primary analyses (online supplemental table 2).

Table 2

Comparison of economic outcomes by hospital group

The standardised rates of hospital bed-days per patient within 30 days following surgery and standardised costs reimbursed for this care per patient by the insurer during the pre-implementation and implementation periods are presented by study group in figure 2. When comparing intervention hospitals with control hospitals, the absolute amount of hospital bed-days per patient was reduced by 0.2 (95% CI 0.1 to 0.3 bed-days), which was equivalent to a reduction of 3.3% (95% CI 2.1% to 4.6%). Furthermore, the hospital costs reimbursed for this care per patient by the insurer were reduced by €73 (95% CI €7 to €144) or 1.3% (95% CI 0.0% to 2.6%) in the intervention group compared with the control group (figure 2 and table 3). Overall, the consumption of a total of 8910 hospital bed-days (95% CI 5611 to 12634 bed-days) and €2 615 524 (95% CI €32 366 to €5 405 528) were avoided in the intervention hospitals after the control chart implementation.

Figure 2

Standardised rates of hospital bed-days per patient within 30 days following surgery and standardised costs reimbursed for this care per patient by the insurer, by study group and period. The bar charts represent the standardised rates of hospital bed-days per patient within 30 days following surgery on the left, and standardised costs reimbursed for this care per patient by the insurer on the right in each group (control and intervention hospitals) and period (pre-implementation and implementation). These standardised rates and standardised costs were determined using estimated regression coefficients obtained from the generalised linear mixed models and marginal standardisation method (see details for marginal standardisation method in online supplemental appendix). The corresponding 95% CIs were computed from non-parametric bootstrap based on 1000 replications. Differences above grey dotted brackets indicate absolute differences in standardised rates or standardised costs between implementation and pre-implementation periods in each group (control and intervention hospitals). Differences above black solid line brackets indicate difference between the intervention and control hospitals of absolute differences in standardised rates or in standardised costs from implementation to pre-implementation periods in each hospital group. These differences of absolute rates or costs differences capture the control chart impact by comparing the change in economic outcomes from pre-implementation with implementation periods between the control and intervention hospitals. Asterisks indicate significant differences as follows: *p≤0.05; ***p≤0.001 (p values computed from the generalised linear mixed models).

Table 3

Estimated absolute and relative rate of hospital bed-days or costs differences

Discussion

This national cluster randomised trial demonstrated that, in surgical teams, the implementation of prospective outcome monitoring coupled with indicator feedback using control charts was associated with reductions in hospital bed-days per patient within 30 days following surgery, and hospital costs reimbursed for this care per patient by the insurer. These findings suggest that the regular use of control charts could support reductions in hospital resource expenditures.

Our study is the first in our knowledge to evaluate the economic impacts of establishing a control chart-based intervention in surgical departments. A recent systematic review reported that the number of studies using control charts to monitor performance in the context of surgery has steadily grown over the last two decades.18 Although some of these studies have focused on economic indicators, including length of stay and costs, none evaluated the economic effects of implementing a control chart-based programme for surgeon performance monitoring. In fact, the majority either aimed to adapt control chart methodology and test the utility or feasibility of their application in surgical departments, or used control charts to measure the impact of quality improvement efforts in surgical departments.18

Past research has suggested that reducing hospital bed-days after surgery could benefit healthcare systems by increasing hospital bed turnover. This can alleviate pressure on hospital staff and liberate beds to match demand with capacity for interhospital transfers, intensive care unit care and emergency admissions.9 In addition, shorter hospitalisation periods after surgery tend to be associated with reduced costs of postoperative care during the index hospitalisation, and are not necessarily linked to increased medical expenses once the patient has returned home.10 Reducing postoperative hospital bed-days can also benefit patients by decreasing their risk of exposure to infection and other adverse events during their hospital stay.11 However, reductions in hospitalisation days should be done cautiously and progressively as evidence suggests that reduced length of stay can be related to worse postoperative outcomes in some contexts.19

The costs of quality improvement methodologies are often cited as a barrier to their implementation.7 Although the correct construction of control charts requires some expertise, they do not require costly equipment to be created.11 They are especially low cost when they use data that is already routinely collected. Our results provide further support that control charts based on routinely available hospital data could represent an economical quality improvement solution because they were associated with reduced resource expenditure. In our study, our highly trained team used the centralised relevant data from the various surgical centres, and generated the control charts that were then distributed to the participating hospitals. However, hospitals could also automatize control chart feedback by integrating data into local computerised decision support systems that trigger monitoring alerts for abnormal surgical outcomes. Future studies could evaluate how the automation of control charts in-house could facilitate their implementation and transform the way surgeons monitor patients.

When applying a control chart intervention, healthcare providers should recognise that the feedback alone will not result in improvement. In our intervention, control charts were actively reviewed by surgical teams during regularly scheduled meetings. Discussion of the control charts encouraged behavioural and organisational changes, and helped surgical teams determine actions that could be taken to improve their outcomes. Therefore, achieving the desired results depends on the participation of surgical teams, and requires dedicated leaders on-site, regular meetings, productive communication and a willingness to identify and implement new solutions.14 The satisfactory compliance of surgical teams in the intervention hospitals was a major strength of this study.

This study used a rigorous cluster randomised design to avoid the risk of contamination. Additionally, our robust statistical analysis approach enabled us to adjust for potential confounding factors that were measured, including median household income, which was significantly different between study groups. The results of sensitivity analysis including patients with missing household incomes were consistant with our intial results. Nevertheless, this trial does have several limitations. First, we cannot rule out the possibility that residual confounders influenced our findings due to potential inherent inaccuracies or unmeasured variables (linked to patient case-mix, surgeon, surgical team and the operating room) in the medico-administrative data. However, as most of the information used are critical for billing purposes, we expect they are accurately collected in hospital claims databases. Furthermore, we would assume the data quality to be equivalent between the two groups, due to randomisation. Second, we did not consider the costs related to implementing the control charts, which should be subtracted from the savings obtained following the intervention. Beyond the control chart development and diffusion, most of the intervention costs were related to the human resources mobilised to coordinate the scientific project. Additional costs would be related to the time local champions spent during training days and while conducting the control chart review meetings in every hospital. Third, the study was not designed to identify the mechanisms by which implementing control charts impacts hospital bed-days per patient as within 30 days following surgery, and hospital costs reimbursed for this care per patient by the insurer. Therefore, we cannot draw any causal conclusions or determine that any specific actions in the intervention group impacted the measured outcomes. Fourth, the results of our analyses of costs saved should be interpreted prudently as the CIs are wide. Finally, the present study was conducted in France, which limits the generalisability of our findings to other countries. The economic impact of applying the control chart intervention in a different context remains unknown.

Conclusions

Although the value of control charts for reducing the occurrence of adverse events has been proven, the potential economic benefits of such interventions are not well-understood. This study compliments the previous findings of the SHEWHART control trial, by demonstrating that prospective outcome monitoring in surgical departments has the potential to support improved patient outcomes, and benefit hospitals by decreasing hospital bed-days per patient within 30 days following surgery and hospital costs reimbursed for this care per patient by the insurer. Control charts are a promising tool because they are relatively simple, low cost and could be implemented in hospitals around the world to improve surgical outcomes, while potentially reducing pressure on resources in hospital systems.

Data availability statement

No data are available.

Ethics statementsPatient consent for publication

Not applicable.

Acknowledgments

We thank the members of the SHEWHART Trial Group. The SHEWHART Trial Group: Françoise Bourgoin, Housseyni Holla, and Sandra Steunou (Agence Technique de l’Information sur l’Hospitalisation); Clotilde Naudot (Assistance Publique Hôpitaux Paris - Louis Mourier); Isabelle Lacombe and Jérémie H Lefèvre (Assistance Publique Hôpitaux Paris - Saint Antoine); Matthew J Carty (Brigham and Women’s Hospital); Jean-Marc Arimont and Charles Foulkes (Centre Hospitalier Alpes-Leman); Patrice David and Laurence Neyer (Centre Hospitalier Colmar - Louis Pasteur); Clément Gayet, Sandrine Hemet, and Loïc Le Menn (Centre Hospitalier Cornouaille - Laennec); Valérie Serra-Maudet (Centre Hospitalier Le Mans); Emeric Abet (Centre Hospitalier La Roche-sur-Yon); Matthieu Poussier (Centre Hospitalier Montélimar); Jérôme Broli, Domenico Papaleo, and Jan Martin Proske (Centre Hospitalier Sud Francilien - Jean Jaurès); Valérie Filippi and Davide Mazza (Centre Hospitalier Toulon - Sainte Musse); Bénédicte Fraleu Louër, Dominique Gratien, and Hélène Poirier (Centre Hospitalier Versailles - André Mignot); Béatrice Alves-Neto, Kévin Fixot, and Matthieu Hournau (Centre Hospitalier Régional Metz Thionville - Bel Air); Jean-Marc Regimbeau (Centre Hospitalier Universitaire Amiens Picardie - Nord); Nicolas Bouviez (Centre Hospitalier Universitaire Besançon - Jean Minjoz); Yoann Marion (Centre Hospitalier Universitaire Caen Normandie - Côte de Nacre); Anne Dubois, Sophie Perret-Boire, and Denis Pezet (Centre Hospitalier Universitaire Clermont Ferrand - Estaing); Christophe Mariette (Centre Hospitalier Universitaire Lille - Huriez); Laurent Brunaud and Adeline Germain (Centre Hospitalier Universitaire Nancy - Brabois); Juliette Podevin and Edwige Riegler (Centre Hospitalier Universitaire Nantes - Hôtel Dieu); Tarek Debs (Centre Hospitalier Universitaire Nice - Archet); Riccardo Gauzolino (Centre Hospitalier Universitaire Poitiers - La Miletrie); Reza Kianmanesh (Centre Hospitalier Universitaire Reims - Robert Debré); Amine Brek (Centre Hospitalier Universitaire Saint Etienne - Nord); Sylvain Kirzin (Centre Hospitalier Universitaire Toulouse - Purpan); Benoît Bourdet and Bertrand Suc (Centre Hospitalier Universitaire Toulouse - Rangueil); Dorothée Brachet and Dumitru Cojocarasu (Centre Médico Chirurgical du Mans - Pôle Santé Sud); Philippe Granger (Clinique Belledonne); Serban Bageacu and Michel Bourbon (Clinique Mutualiste Chirurgicale – Saint Etienne); Walter Bertolaso (Clinique Pont de Chaume); Pierre Caillon (Clinique Tonkin); Hector Ormando (EM Lyon Business School); Renato Lupinacci and Olivier Oberlin (Groupe Hospitalier Diaconesses - Croix Saint Simon); Axèle Champault and Monique Sigismond (Groupe Hospitalier Paris - Saint Joseph); Pascal Frileux and Alexandre Rault (Hôpital Foch - Suresnes); Raphaël Bourdariat (Hôpital Privé Jean Mermoz - Lyon); Antoine Lamblin and Christine Leclercq (Hôpital Privé La Louviere - Lille); Bernard Pol (Hôpital Saint Joseph - Marseille); Mathilde Adam, Gilles Poncet, and Catherine Valette-Lagnel (Hospices Civils Lyon - Edouard Herriot); François Chollet, Cyrille Colin, Antoine Duclos, Keitly Mensah, Philippe Michel, Léa Pascal, Cécile Payet, Stéphanie Polazzi, and Sarah Skinner (Hospices Civils Lyon - Lacassagne); Sébastien Couraud, Jean-Christophe Lifante, Guillaume Passot, Jean-Louis Peix, and Vincent Piriou (Hospices Civils Lyon - Lyon Sud); Cédric Beau (Polyclinique Jean Villar - Bruges); Philippe Benand and Benjamin Brugiere (Polyclinique Poitiers); Dine Koriche (Polyclinique Vauban - Valenciennes)