Trial design

A two-arm, parallel, cluster-randomized trial will assess the effect of the ROAD Home strategy on days of antibiotic overuse at discharge. We will recruit at least 40 hospitals from within the Michigan Hospital Medicine Safety Consortium (HMS), a statewide 69-hospital collaborative consisting of diverse hospitals focused on improving the care of hospitalized patients. HMS hospitals that agree to participate will undergo covariate-constrained randomization to improve balancing of hospital characteristics between groups with 1:1 allocation to the ROAD Home strategy vs. a “usual stewardship” control. In the 12-month intervention period, hospitals will implement their selected stewardship interventions while we assess days of antibiotic overuse at discharge and patient outcomes. During the intervention period and in the 12-month postintervention period, we will conduct a mixed-methods process evaluation to evaluate barriers, facilitators, and implementation outcomes across hospitals.

Study setting

HMS is supported by Blue Cross/Blue Shield of Michigan (BCBSM) and, since 2017, has focused on improving the care of hospitalized patients treated for UTI or CAP. Trained abstractors at each hospital collect data via medical record review of a pseudo-random sample of hospitalized patients with a positive urine culture or CAP within each hospital [7, 36, 37]. There are tri-annual HMS meetings (2 in person, 1 virtual) to share data and best practices. HMS was initially voluntary (pre-2020) but has become required for all eligible hospitals participating in BCBSM’s value-based partnership program. HMS has established pay for performance antibiotic use metrics which promote improvement. However, HMS does not provide financial support for antibiotic stewardship interventions nor specify how hospitals should achieve improvement. To date, there remains variation in performance on antibiotic performance metrics across HMS hospitals [36, 37]. Recruitment from HMS is highly pragmatic due to existing infrastructure which allows patient-level data collection, hospital survey administration, in person group meetings, and inclusion of diverse hospitals from across the state of Michigan. As of January 2023, 69 (75%) of the 92 non-critical access, non-federal hospitals in the state of Michigan participate in HMS. Table 1 shows characteristics of HMS hospitals.

Table 1 HMS hospital characteristics, n = 69 hospitalsParticipants

Stakeholders working in acute care hospitals who implement stewardship interventions (inner setting implementation leads) are the primary targets of the strategy being evaluated in the ROAD Home trial. These inner setting implementation leads will be the main point of contact between the ROAD Home study team and hospitals randomized to receive the ROAD Home strategy. Each hospital within HMS is represented by at least one data abstractor (usually a nurse with quality improvement training) and one physician champion. Many hospitals also include quality improvement representatives or leaders, administrators, additional physicians, and pharmacists involved in antibiotic stewardship. Given the variation in stewardship stakeholder composition that naturally exists in HMS, we will allow hospitals to self-select the individual(s) to represent them as implementation lead(s) within the trial.

Because the ROAD Home strategy is intended to increase the uptake of antibiotic stewardship interventions known to reduce antibiotic overuse at hospital discharge, patients cared for in intervention arm hospitals who are being treated for UTI or CAP are secondary targets, in that increased uptake of evidence-based antibiotic prescribing is expected to translate into improved clinical outcomes.

Inclusion/exclusion criteria

All hospitals participating in HMS are eligible to participate.

Recruitment

Hospital recruitment occurred in November 2023 at an in-person HMS collaborative meeting. Hospitals unable to attend the in-person meeting were contacted for a separate, virtual meeting in December 2023. Representatives from eligible hospitals were invited to an in-person recruitment session led by the ROAD Home investigative team. To improve trial retention and commitment to study procedures, we employed an adapted Methods Motivational Interviewing approach [38] for recruitment where we (a) set clear participant expectations, (b) introduced the trial including rationale for key elements of study design, and (c) diffused ambivalence about trial participation through small group discussion. This process has been shown to improve study retention in behavioral clinical trials [39]. Recruitment is still ongoing. The recruitment goal, based on power calculations (see below), was at least 40 hospitals; given the goals of HMS, we did not turn away hospitals interested in participating once our goal of 40 was reached. As of December 20, 2023, 50 hospitals have agreed to participate.

Randomization

The trial will randomize study hospitals in a 1:1 allocation ratio to two arms: “ROAD Home strategy” intervention vs. “usual stewardship” control (see Fig. 1 for CONSORT Diagram). Eligible hospitals that agree to participate will undergo covariate-constrained randomization to improve balancing of critical hospital characteristics between groups. Covariate-constrained randomization allows for balancing of multiple pre-specified characteristics of interest when the number of characteristics is high compared to the number of clusters being randomized [40]. Characteristics that will be included in the constrained randomization process include (a) baseline antibiotic overuse at discharge; (b) year of entry into HMS; (c) social needs of population being served by the hospital (classified as rural hospital) (defined as either somewhat or very rural using Rural Urban Continuum Code (RUCC) score > 3) [41], Minority Serving Hospital or Safety Net Hospital (defined as hospital with RUCC score of 1 or 2 and either top 25% non-white and/or Hispanic population or top 25% Medicaid or uninsured population, respectively) [42] or neither; (d) composition of ASP leadership (is the ASP team led by both an ID physician and an ID pharmacist or not); and (e) baseline number of stewardship interventions already implemented (weighted using our published ROAD Home framework) [35]. We will select from randomization sequences that have a balance of ± 10% or ± 2 sites between arms for continuous and categorical variables, respectively. All hospitals will be allocated to intervention or control arms simultaneously, negating the need for allocation concealment (see Fig. 2 for SPIRIT flow diagram). Due to the need for investigators to participate in the delivery of the ROAD Home strategy, neither investigators nor hospitals will be blinded to their assignment.

Fig. 1

CONSORT diagram. HMS, Michigan Hospital Medicine Safety Consortium; MMI, methods motivational interviewing. aHospitals will undergo covariate-constrained randomization to improve balancing of critical characteristics between groups with 1:1 allocation

Fig. 2

SPIRIT flow diagram: schedule of enrollment, interventions, and assessments

Intervention hospitalsDevelopment of ROAD Home

ROAD Home is a multicomponent implementation strategy that combines (a) evaluative techniques to understand hospital context, (b) tailoring of stewardship interventions to that context, (c) external facilitation, and (d) active participation from hospitals to address known barriers to the implementation of discharge antibiotic stewardship. We developed ROAD Home based on extensive formative research on the current discharge stewardship landscape, approaches to reduce discharge overuse across hospital contexts, and quantitative associations of different stewardship interventions with antibiotic overuse at discharge [7, 35, 43, 44]. This research demonstrated three primary findings that informed the creation of ROAD Home. First, the stewardship intervention that has demonstrated efficacy and is most recommended (prospective audit and feedback on discharge antibiotic prescriptions by a clinical or ID pharmacist) [15] to improve discharge prescribing is resource intensive and not feasible or sustainable for most hospitals, and thus rarely used [7, 16, 44,45,46,47]. Second, there appear to be multiple pathways to improve discharge antibiotic prescribing through combinations of stewardship interventions that vary widely in feasibility [4, 35]. Third, antibiotic stewardship interventions that are implemented using a participatory approach are more likely to be successful than “top down” approaches because they promote stakeholder engagement and tailoring of strategies to context [30, 48,49,50,51,52,53].

Considering these findings in concert with the integrated-Promoting Action on Research Implementation in Health Services (iPARIHS) framework, we selected a group of implementation strategies that are likely to promote the tailoring of stewardship interventions to improve discharge antibiotic prescribing [54]. iPARIHS suggests that successful implementation is a function of the quality of the evidence to be implemented and the way the stakeholders perceive this evidence; the characteristics of the setting or context in which implementation occurs; and the way that evidence is introduced or facilitated into practice [55]. Facilitation is central in iPARIHS, which posits that facilitation enables successful implementation by connecting action around new practices with the people who need to implement the new practice in a context-adaptive manner [56]. ROAD Home is intended to support inner setting implementation leads tasked with quality improvement and/or antibiotic stewardship to implement interventions to improve antibiotic use at hospital discharge. An additional file includes a table that specifies each strategy included in ROAD Home (see Additional file 1) [57]. Figure 3 depicts our Implementation Research Logic Model [58] elaborating the connections between implementation determinants, strategies, and outcomes addressed in the ROAD Home Trial.

Fig. 3

ROAD Home Trial Implementation Research Logic Model. CFIR, Consolidated Framework for Implementation Research, Damschroder et al. Implementation Science (2022). ERIC, Expert Recommendations for Implementing Change, Powell et al. Implementation Science (2015). HMS, Michigan Hospital Medicine Safety Consortium. Implementation Research Logic Model Template from Smith et al. Implementation Science (2020). Color coding indicates the connection between the ERIC strategy and mechanism proposed for how it impacts implementation outcomes (underlined). +  = facilitator.—= barrier

The ROAD Home strategy

Hospitals randomized to receive ROAD Home will undergo (1) a baseline needs assessment to create a tailored suite of discharge stewardship interventions, (2) supported decision-making in selecting interventions to implement, and (3) external facilitation following an implementation blueprint (Fig. 4). We describe each strategy below.

Fig. 4

ROAD Home strategy. HMS, Michigan Hospital Medicine Safety Consortium

Baseline needs assessment and development of customized suite of stewardship interventions

Hospitals will undergo a baseline assessment including an audit of baseline antibiotic use at discharge to identify areas of high overuse and a needs assessment. The needs assessment—conducted via HMS annual survey—will assess antibiotic stewardship interventions currently in use (and, if in use, how frequently, on what patient populations, and by whom), existing resources, infrastructure, hospital priorities, and anticipated barriers to implementation. Based on the findings of this local needs assessment, study investigators will generate a tailored suite of potential stewardship interventions for each hospital.

The tailored suite created for each hospital randomized to receive ROAD Home is based on a framework we developed that specifies a three-tiered suite of evidence-based discharge antibiotic stewardship interventions [4, 11, 35]. Tier 1 includes critical stewardship infrastructure (e.g., guidelines); Tier 2 includes inpatient-focused stewardship interventions (e.g., inpatient-focused audit and feedback); and Tier 3 includes discharge-focused stewardship interventions (e.g., audit and feedback of discharge prescriptions). While Tier 3 interventions individually have been shown to be most efficacious [15, 17, 19, 35], we found in a retrospective analysis [35] that implementing three Tier 1 interventions, or one Tier 1 and one Tier 2 intervention, had a similar effect on antibiotic prescribing at discharge as implementing a single Tier 3 intervention. Thus, rather than suggesting all hospitals implement the same intervention to improve discharge prescribing, ROAD Home supports hospitals in selecting intervention(s) to implement based on their existing priorities, resources, and infrastructure [35]. Interventions have been given a point value (each Tier 1 intervention = 1 point, each Tier 2 intervention = 2 points, each Tier 3 intervention = 3 points). The only requirement is that hospitals select 3 points worth of interventions to implement, which they can achieve by selecting a single Tier 3 intervention, or a combination of Tier 2 and Tier 1 interventions, or three Tier 1 interventions.

Each hospital will receive a tailored ROAD Home suite with a personalized graphic (see Fig. 5) that specifies ROAD Home interventions at each tier placed in one of four groups: (a) already doing well (no need to change); (b) intervention we recommend adding/changing; (c) intervention your hospital could implement, but may be unnecessary given performance; or (d) intervention your hospital could implement, but there are barriers. Recommended interventions will add up to 3 points but can be changed based on hospital preference.

Fig. 5

Example tailored ROAD Home suite

Supported decision-making to select and tailor interventions

After intervention hospitals review their tailored suite, they will meet virtually with study investigators to ask questions, provide clarifying information, and suggest edits to which 3 points they choose from the suite. Stewardship interventions not included in the original suite may also be considered; novel interventions will be given a point value determined by study investigators based on the most appropriate Tier fit. With assistance from the study investigators, intervention hospital implementation lead(s) will select a total of 3 points of interventions to implement. While study investigators will facilitate a discussion of the anticipated benefits, drawbacks, barriers, facilitators, and resources needed for each potential intervention, the hospital implementation lead(s) will make the final decision on which interventions to include.

External facilitation following an implementation blueprint

Once interventions have been selected, study investigators will facilitate a discussion about implementation, starting with the hospital implementation lead(s) selecting a start date within a 3–6-month window and creating an implementation blueprint. The blueprint will map out the specific steps, a timeframe with milestones, responsible people (or groups), and resources needed to implement ROAD Home interventions. An additional file shows a sample implementation blueprint (see Additional file 2) [59]. The hospital implementation lead(s) will be provided with a blueprint template, instructions to fill it out, and a request to submit it for review within 1 month. Study investigators will review the blueprint for completeness and ask clarifying questions if necessary. As intervention hospitals prepare to implement their selected interventions, they will have access to adaptable tools from study investigators to support implementation of the selected interventions. For example, if a hospital decides (for 1 point) to create and implement institution specific UTI guidelines, we will provide a guideline template that can be tailored (e.g., adding their hospital logo, utilizing context-specific language).

As part of external facilitation, all hospitals will have 4 additional meetings with study investigators and other intervention hospitals during HMS meetings. The first, just prior to the intervention period, will be a “kick-off” meeting where study investigators will facilitate a discussion among hospital implementation leads including a high-level presentation of themes that cut across the implementation blueprints related to anticipated barriers. The purpose of this meeting is to engage meaningfully with intervention hospitals prior to launch, build enthusiasm for the approach, and develop a sense of camaraderie across sites. During the intervention period, hospitals will have 2 “check-in” sessions and 1 close out session during HMS collaborative wide meetings. These meetings will be led by the study investigators who will facilitate a discussion about study progress, challenges, and strategies to overcome challenges. These meetings have three primary functions to facilitate implementation: (1) foster a learning climate among hospitals so that implementation leads can share with each other strategies they have used to address barriers; (2) promote accountability for follow through on the planned ROAD Home interventions, which will increase fidelity to the implementation blueprints; and (3) increase implementation leads’ self-efficacy to make change at their hospital.

Control hospitals

The study’s primary aim is to understand whether the ROAD Home strategy results in fewer days of antibiotic overuse at discharge compared to “usual stewardship.” With this goal in mind, hospitals randomized to the control group will continue usual HMS activities and their existing stewardship interventions. As described previously [36, 37], usual HMS activities include hospital benchmarking of key quality metrics (e.g., pneumonia antibiotic duration), sharing best practices during 3 times per year meetings, and pay-for-performance metrics. These activities may touch on discharge prescribing (e.g., during best practices discussion) but discharge prescribing will not be the focus (i.e., pay-for-performance metrics will not be discharge antibiotic use). Individual hospital activities are not specific to HMS but hospitals often use HMS data to improve antibiotic prescribing. It is possible that some control hospitals may decide to focus on improving discharge antibiotic use during the intervention period. However, they will not receive any of the ROAD Home strategies including baseline data analysis or needs assessment, tailored suite, supported decision-making in selecting interventions to implement, an implementation blueprint, adaptable stewardship tools, or external facilitation. We will assess ongoing and new stewardship interventions (including discharge-focused interventions) implemented in both intervention and control hospitals via annual required HMS surveys to identify which, if any, control hospitals worked on improving discharge antibiotic use during the intervention period and conduct a per protocol sensitivity analysis that excludes control hospitals that started a new discharge-focused stewardship intervention.

Study outcomesPrimary outcome

Our primary study outcome will be baseline-adjusted days of antibiotic overuse at discharge, defined by our extensively tested, guideline-based method (with minor updates applied to reflect national guideline changes) [4, 11, 35]. Antibiotic overuse at discharge includes a composite of unnecessary antibiotic use (e.g., antibiotics for asymptomatic bacteriuria [ASB]), excessive antibiotic duration (e.g., > 3–5 days for most patients with CAP), and avoidable fluoroquinolones (e.g., ciprofloxacin use for a patient with uncomplicated cystitis who has no allergies).

Balancing metric

Prior data suggest that excessive antibiotic use does not improve patient outcomes [60]. To ensure no unintended patient harm occurs from reducing antibiotic overuse at discharge, we will use a non-inferiority framework to analyze composite 30-day post-discharge patient outcomes including mortality, readmission, or urgent visit (i.e., urgent care or emergency care visit).

Secondary outcome

Our composite clinical secondary outcome is antibiotic-associated adverse events defined using our prior metrics [4, 7, 61] including common side effects such as gastrointestinal upset and candidiasis as well as potentially life-threatening side-effects such as Clostridioides difficile infection (CDI). Side effects will be gathered through a combination of chart review and 30-day post-discharge patient phone calls.

Data collection, completeness, and quality

Antibiotic overuse at discharge—including the clinical data needed to define “overuse”—will be collected as part of HMS’ ongoing quality improvement initiatives. In HMS, discharge antibiotic prescriptions and discharge summaries are reviewed by trained clinical abstractors who collect antibiotic name, start, and stop dates, discharge antibiotic duration, number of pills dispensed, indications, dose, and frequency. All other patient data, including symptoms, diagnostic testing results, and inpatient antibiotic treatment will be obtained via HMS medical record review as described previously [4, 7, 37]. Thirty-day post-discharge outcomes will be obtained through HMS via a combination of medical record review and patient phone calls. In HMS, all patients who survive to hospital discharge have 30-day outcomes obtained via medical record review. In addition, patients eligible for a discharge phone call (i.e., those not discharged to hospice, prison, or a skilled care facility or those not known to have died or been rehospitalized) are called 30 days after hospitalization (3 attempts made) to obtain information on antibiotic-related adverse effects and any additional 30-day outcome data that might be missing from the medical record. HMS has collected data on mortality, readmission, need for additional care (i.e., urgent/emergent visit), and antibiotic-associated adverse events, via this method since 2017 [7, 11, 61].

Statistical analysisSample size

Over the study period, we anticipate including ~ 8000 patients in the intervention arm and ~ 8000 patients in the “usual stewardship” control arm (~ 400 patients/hospital/year). We estimated sample size and power based on the number of clusters (20 per arm) and using previously published data on antibiotic overuse at discharge, including in HMS hospitals [11, 35, 62], which found (a) baseline 2.2 days of antibiotic overuse at discharge per patient (or 4.4 days per patient with overuse); (b) an intraclass correlation coefficient (ICC) for our primary outcome (days of antibiotic overuse at discharge) of 0.03 to 0.035; and (c) the average number of patients included per hospital per year (400). Estimates of effect size were determined based on the minimal clinically important difference (MCID) needed to improve distal public health outcomes (20%) [63,64,65,