We conducted a segmented time-series analysis to evaluate changes in carbapenem usage following the implementation of a hospital pharmaceutical formulary. Additionally, a non-inferiority test with propensity score matching was performed to assess the changes in treatment efficacy. The study period spanned from September 1, 2016, to October 31, 2020. The effects of the formulary introduction were assessed by comparing usage before (September 1, 2016, to September 30, 2018) and after (October 1, 2018, to October 31, 2020) the intervention, including changes in the target areas of usage. We examined the impact on the effectiveness of infectious disease treatment from two perspectives.

In October 2018, we formulated the standard use of carbapenem antibiotics in our hospital and administered them throughout the hospital. At the time this formulary was implemented, 5 carbapenem antibiotics were available on the Japanese market: meropenem (MEPM), biapenem (BIPM), imipenem/cilastatin (IPM/CS), panipenem/betamiprone (PAPM/BP), and doripenem (DRPM). Our hospital has used all 5 types. Before the formulary intervention, the selection of carbapenem antibiotics was not restricted, and the choice among carbapenems (e.g., MEPM, DRPM, and IPM/CS) was made at the discretion of the prescribing physician. At our institution, all carbapenem usage requires submission of a usage report to the Infection Control Department for monitoring purposes.

Based on evidence regarding therapeutic efficacy and safety, drug prices, usage history, and other factors, 3 types of carbapenem antibiotics are recommended as standards (Appendix 1). The first recommendation was MEPM, which is highly recommended by various guidelines in the field of infectious diseases, and has a relatively low cost due to its status as a generic drug. Despite limited evidence and its higher cost, DRPM was chosen as the second most recommended drug. Additionally, IPM/CS was selected as the second recommended drug due to its extensive clinical experience and pharmacokinetic data in pediatric and neonatal populations in Japan [6,7,8], was selected as the second recommended drug. This formulation was approved by the Pharmaceutical Affairs Committee of Yokohama City University Hospital and was issued throughout the hospital. After the system was made public, a list of standard recommended drugs popped up when entering a prescription in the electronic medical record.

The subjects were patients who received injectable antibiotics (MEPM, DRPM, and IPM/CS) during hospitalization at Yokohama City University Hospital during the survey period. There were no exclusions based on the patient characteristics. The monthly usage of formulary-eligible injectable antibiotics (DRPM, IPM/CS, MEPM) administered to target cases was calculated from the electronic medical records of Yokohama City University Hospital. The total amount of carbapenem used was calculated as a reference value.

Given the Japanese pharmaceutical pricing system, where drug prices are periodically reduced, and the potential for total drug costs to be influenced by the allocation of multiple product specifications (e.g., 0.5 g/vial and 1 g/vial), economic evaluations based on drug prices may be influenced by systemic factors. To avoid this potential bias and provide a more practical and accurate assessment, we evaluated economic efficiency based on antibiotic usage volume.

The amount used was calculated using the ATC/DDD system published by WHO [9] and the AUD was calculated. The following formula was used to calculate the AUD: The World Health Organization’s ATC index version 2022 was used for DDD.

$$\beginAUD&=\frac&Monthly\:usage\:amont\:of\:Targeted\:Antibiotics\,(g)\end}&DDD(g/person\:per\:day)\times\:total\:number\:of\:inpatient\:days(person\:per\:day)\end}\times\:1000\end$$

DDD: Defined Daily Dose.

The evaluation item was whether there was a decrease in the use of DRPM, the second most recommended drug, due to the introduction of the hospital formulary. The calculated AUD was analyzed using a generalized linear model to perform a split time-series analysis before and after the intervention [10]. The only factors included in the generalized linear model were time course and formulary intervention factors. Model selection was performed by comparison with Akaike’s information criterion (AIC) [11, 12]. A generalized linear model was employed with a gamma distribution and an inverse link function to model the relationship between DRPM usage and formulary intervention. For autocorrelation evaluation, residuals, autocorrelation, and partial autocorrelation were checked.

The subjects were hospitalized patients with intra-abdominal infections who received injectable antibiotics for 3 or more days during the study period. Target diseases included peritonitis (including pelvic inflammatory disease), intra-abdominal abscess, and hepatic/biliary tract infections (e.g., cholecystitis, cholangitis, and liver abscess). From the electronic medical records of the Yokohama City University Hospital, we identified patients who met these criteria. Patients with intra-abdominal infections were selected as the target population based on the usage patterns of carbapenem at our institution. Prior to the formulary intervention, DRPM was frequently used in gastrointestinal surgery, while IPM/CS had limited usage. Given the frequent use of DRPM and abundant evidence supporting its efficacy in intra-abdominal infections, this domain was deemed appropriate for evaluating changes in usage patterns and clinical outcomes associated with the formulary intervention.

Patient selection was further refined based on the exclusion criteria listed in Appendix 2. This exclusion criterion is based on the definitions outlined in Clinical Evaluation Methods for Antibacterial Drugs for Intra-Abdominal Infections by the Ministry of Health, Labour, and Welfare in Japan, which specifies clinical evaluation standards for antibacterial agents used in intra-abdominal infections [13]. Additionally, cases in which antibiotic treatment was not intended for the management of intra-abdominal infections were excluded.

Treatment effectiveness was assessed using the following indicators: (1) body temperature < 37 °C, (2) white blood cell count within the normal range, (3) C-reactive protein (CRP) level below 3.0 mg/dL, (4) improvement in abdominal findings, and (5) resolution of abnormal imaging findings. The criteria for evaluating treatment effectiveness were based on the evaluation items used in phase III clinical trials of antibacterial treatments for intra-abdominal infections conducted in Japan [14]. These criteria were adjusted to include items that could be observed in medical records. If 4 of these 5 indicators were met at 14 days after the initiation of antibiotic therapy or at the final treatment day (whichever occurred earlier), the response was considered “excellent”. A response meeting 2–3 indicators was classified as “effective”, while responses meeting only ≤ 1 were deemed “ineffective”. If ≥ 3 were not measured, the outcome was classified as “indeterminate”. Cases classified as “indeterminate” were excluded from the analysis to ensure the reliable assessment of treatment outcomes.

The proportion of patients categorized as “effective” or better was compared before and after formulary introduction using a non-inferiority test [15]. Propensity score matching (PSM) was applied to control for baseline covariates, including age at admission, sex, BMI, cancer status, and baseline blood test results (WBC, RBC, albumin, AST, ALT, ALP, and CRP) [16, 17]. Propensity scores were calculated using logistic regression, and covariates with a standardized mean difference (SMD) of ≥ 0.15 were adjusted in the final analysis. We employed 1:1 nearest-neighbor matching without replacement, using a caliper width of 0.1 of the standard deviation of the logit of the propensity score.

The primary outcome of the non-inferiority test was the proportion of patients categorized as “effective” or better. To ensure that the formulary change did not lead to a clinically significant reduction in effectiveness, a non-inferiority margin of 10% points was selected based on clinical relevance and previous studies [18,19,20]. We used the likelihood method to evaluate non-inferiority with a one-sided significance level of 0.05. Non-inferiority was established if the lower bound of the 95% confidence interval (CI) for the difference in effectiveness exceeded a pre-specified margin.

Additionally, if physician comments regarding the adverse effects of antibiotic use were present in patient records, they were documented and counted.

All statistical processing and figure creation were performed using R (ver. 4.3.2) [21]. The following libraries were used: tidyverse, tableone, ggplot2, MatchIt, mice, cobalt, and gridExtra [22,23,24,25,26,27,28]. P values of < 0.05 were considered to indicate statistical significance.