The data used for this study were derived from the Dutch Audit for Treatment of Obesity (DATO). DATO is a nationwide, mandatory quality registry for metabolic and bariatric surgery in the Netherlands that collects data on patient characteristics, procedures, complications, and follow-up since 2015 [18, 19]. On-site data verification has proven high validity of the data [20]. All Dutch bariatric clinics participate in this registry, thereby gaining valuable insights in the quality of bariatric care in everyday clinical practice. Healthcare quality is monitored through indicators that provide national benchmarks including the percentage of patients achieving at least 20% TWL during follow-up from 1 up to 5 years after surgery.

The study protocol was approved by the DATO scientific committee. In accordance with Dutch regulations informed consent was not obtained, as DATO is an opt-out registry. The current study was performed in accordance with the ethical standards as stated in the declaration of Helsinki of 1964 and its later amendments.

Weight loss expressed in %TWL at 5 years was the basis for the primary analysis, in line with the objective to achieve the best long-term outcomes. The outcome %TWL is calculated as [weight at screening – weight at follow-up] / weight at screening × 100%. All patients who underwent primary bariatric surgery between October 1, 2016, and September 30, 2017, with registered weight at baseline and at 5 years were considered eligible for analysis, which resulted in 15 hospitals being analyzed. Follow-up years are defined in DATO with an approximation of + / − 3 months, meaning that any follow-up between 9 and 15 months is considered a 1-year follow-up moment, and any follow-up between 57 and 63 months is considered a 5-year follow-up moment, thereby taking follow-up until January 1, 2023, into account. As national policies and regulations do not permit patient-linkage between hospitals, potential revisional surgery after the primary surgery could not be accounted for.

Hospital performance and outlier status were compared between a funnel plot around the median %TWL, and binary funnel plots using two different cutoff points, i.e., achieving at least 20% and 25% TWL. The cutoff 20% is commonly used with 25% added from a perspective of continuous quality improvement, as done in previous studies [13, 17, 21]. Outlier status means performing either significantly better (outperformer) or worse (underperformer) than the national benchmark.

First, the %TWL distribution at 5 years was analyzed by plotting a histogram, which was also used to determine the nationwide median and percentage of patients achieving 20% TWL and 25% TWL. Histograms were also created for each hospital separately, to explore possible differences in distributions. Second, a funnel plot around the median was created which compared the median %TWL of each hospital to the nationwide median, with outliers given a color according to their position with respect to the 95% control limits. Hospitals positioned below the 95% control limit (underperformers) were colored red, hospitals above the 95% control limit (outperformers) were colored green, and hospitals within the control limits were performing conform the nationwide median and therefore colored grey (see appendix for statistical code to create the funnel plot around the median). The median rather than the mean %TWL per hospital was chosen because of its better representation of the overall distribution. Third, the binary funnel plot for achieving at least 20% TWL (yes/no) was created, and hospitals were depicted in this funnel plot using the colors reflecting their performance from the funnel plot around the median as described above. In this way, it is shown how hospitals with a significantly worse (i.e., lower) %TWL distribution would have been missed, i.e., considered performing conform the nationwide benchmark in the binary funnel plot, and thereby missed the incentive to investigate and start improvement initiatives. Fourth, the same analyses were repeated with the binary funnel plot for achieving at least 25% TWL (yes/no).

Decisions on procedure type may explain differences in the %TWL distribution, which may be based on hospital preference rather than patient-mix, as shown in a previous study [13]. Sleeve gastrectomy (SG) and gastric bypass procedures (i.e., Roux-en-Y gastric bypass (RYGB), banded RYGB, or one anastomosis gastric bypass (OAGB), depending on the hospital’s preference), are the two types of surgery that are practiced most. Therefore, the proportion of these procedures performed per hospital was included as the independent variable in a linear regression analysis for the outcome %TWL. As RYGB is the most commonly performed surgery in the Netherlands [4], the proportion of this type of gastric bypass was analyzed separately. This approach will provide insight whether a difference in %TWL distribution may be driven by the choice in procedure type, which could be among the things for underperforming hospitals to investigate. In case of an identified association, funnel plots were separately constructed for SG and RYGB as well to explore whether hospital variation remains within patients undergoing these procedures.

As feedback with funnel plots supports local improvement cycles, it could be preferable to have feedback on outcomes that are achieved by more recent treatment strategies, such as 1-year outcomes. Therefore, similar funnel plots as in the primary analysis were constructed using the outcome %TWL at 1 year for patients operated in the same period (i.e., October 1, 2016, until September 30, 2017). In this way, it was possible to examine whether the same hospitals are identified as outliers in the funnel plot for 1- and 5-year outcomes, thereby exploring whether the performance at 1 year is predictive for their performance at 5 years. The same approach as in the primary analysis was used for analyzing choice of procedure type as an explanatory factor.

To validate the performance of the median-based funnel plot in a different patient cohort, we created funnel plots for the outcome %TWL at 1 year including all patients receiving primary surgery in 2021 (i.e.,operated between October 1, 2020, and September 30, 2021). All statistical analyses were performed using RStudio version 2023.06.1 (R Foundation for Statistical Computing, Vienna, Austria).