Study protocol is published elsewhere [18]. A brief description is as follows:

Study design

The design was a 2-arm, 3-year pragmatic hybrid type 3 [19, 20] mixed method randomized controlled trial conducted at 18 waiver sites in the state of Michigan [21,22,23,24]. A hybrid design was chosen as it examines the effects of implementation strategies and intervention effectiveness for beneficiary outcomes [20]. The knowledge to action [24] model underpinned examination of outcomes and the Consolidated Framework for Implementation Research [25]-guided implementation.

Study setting and participants

The settings were 18 Medicaid home and community-based waiver (HCBW) sites in Michigan that used the same electronic health record (EHR) system. Two Michigan sites using a different EHR were excluded. The waiver supports low-income (at or below 300% of the federal poverty level), nursing home eligible, disabled, and older adults in the community to avoid institutionalization. Sites care for 400 to 2500 beneficiaries and employ 10 to 125 clinicians. Sites were contracted, and clinicians were recruited via email at each site. Beneficiaries were recruited during usual care by clinicians, using a pocket aid to examine beneficiary needs in regard to the intervention. Beneficiaries could opt out and continue to receive care provided by the site, but their data were not extracted from the EHR or analyzed.

Randomization and blinding

To assure similarity of two trial arms, sites were paired in blocks using quality assessment scores (2015–2017) and number of beneficiaries. A coin was flipped to determine arm assignment for each pair. Clinicians and beneficiaries were blinded to arm assignment.

Power analysis

Given 18 sites (9 in each arm) available in Michigan, in the comparison of site-level outcome of adoption and sustainability, the detectable effect size with power of 0.80 in two-sided tests at .05 level of significance was Cohen’s d = 1.41. Given the sample size of clinicians of 539, the average cluster size was approximately 45, and with an assumed intra-class correlation coefficient (ICC) of 0.01, the design effect factor was 1.45, and the detectable effect size was Cohen’s d = 0.29. Given the sample size of 7030 beneficiaries, the average cluster size was approximately 390, and with an assumed intra-class correlation coefficient of 0.01, the design effect factor was 4.9, and the detectable effect size was Cohen’s d = 0.15 for power of 0.80 in two-sided tests at .05 level of significance. These effect sizes are below d = 0.33–0.5, the thresholds commonly used for clinical significance; therefore, the study was powered to detect any meaningful differences between arms on clinician and beneficiary outcomes [26, 27].

Usual care

Usual 1915(c) HCBW services care includes annual assessments, case management, and supports coordination by RNs and social workers (SWs) via home visits and phone calls [28]. Nineteen services are provided as needed and include adult day care, chore services (e.g., cleaning, laundry), community health worker, community transportation (e.g., to doctor’s appointment), counseling, environmental modifications, and a fiscal intermediary. In addition, goods and services, home delivered meals, nursing services, personal emergency response system, private duty nursing/respiratory care, specialized medical equipment and supplies, training, personal care, medication management, lawn care, snow removal, cleaning, grocery shopping, and laundry are provided as needed.

Intervention

In addition to the usual care that was provided, the intervention [3,4,5], which was previously adapted [29] to fit the Michigan HCBW, was implemented. RNs, SWs, and OTs conducted up to 10 additional home visits over 16 weeks and provided assistive devices (e.g., commode) and home modifications and alterations (e.g., installing devices or widening doorways) [29]. RNs, SWs, and OTs consulted with the individual receiving the care (beneficiary) to identify daily activity goals (e.g., taking a shower and walking to the bathroom) and evaluate barriers to achieving the goals to attain their desired outcomes, and then, care was provided. OTs assisted beneficiaries to carry out ADLs and IADLs that were challenging, such as meal preparation, bathing, and dressing. RNs targeted pain and mood management, fall prevention, incontinence prevention, and medication management. SWs addressed social and behavioral needs and issues and coordinated community resources.

Implementation strategies

As shown in our published protocol paper [18], 9 strategies were included in the implementation bundle. A formal relationship was established by memorandum of understanding, delineating the role and duties of study staff, HCBW site, internal facilitators, and clinicians, and an informal relationship was built among those parties via monthly meetings (virtual). Readiness to implement, leadership, clinician attitude toward use of evidence, and self-efficacy were examined. Training of clinicians on use of the intervention with beneficiaries and of internal facilitators and the external facilitator on facilitation occurred. A coalition of internal facilitators met monthly (virtual) to share best practices (implementation and intervention). IF and EF are described below. Fidelity to implementation strategies was monitored, and feedback was provided to internal facilitators and the external facilitator.

Facilitation

Internal facilitators acted as “change agents” and “champions,” utilizing the implementation strategy bundle to support RNs, SWs, and OTs’ use of the intervention with beneficiaries (IF arm and IF + EF arm). They were identified by management teams at the sites based on the following criteria: experienced HCBW manager/supervisor (RN or SW); organized, understands the needs of others, and clear communication skills; and believed the intervention was effective. Internal facilitator training included 9-online modules on the role and tasks, problem-solving, feedback, reflection, counseling, motivational interviewing, and remediation and a 60-min session (synchronous) with the study team covering competencies [30] and the implementation plan.

Internal facilitators’ tasks (IF arm and IF + EF arm) prior to implementation included the following:

Clarified purpose and role of internal facilitator with study staff.

Integrated the intervention within existing clinical programs and services.

Engaged local leadership to support implementation and use of the intervention (e.g., sign agreement).

Reviewed the structured implementation toolkit and products (e.g., posters, emails, scripts).

Conducted an implementation needs assessment with key stakeholders to identify potential barriers.

Set expectations based on the local needs assessment (e.g., need to hire OT).

Developed a localized plan (e.g., hiring OT, scheduling clinician visits) and timeline for implementation of the intervention.

Internal facilitators’ tasks (IF arm and IF + EF arm) during implementation included the following:

Deployed the localized implementation plan (e.g., timing of training, use of OT).

Assured RNs, SWs, and OTs completed training (online).

Participated (virtual) in a learning collaborative (monthly) to share best practices on implementation with other internal facilitators.

Reviewed monitoring data (weekly) on implementation status and intervention usage.

Provided feedback to the RNs, SWs, and OTs on training and use of the intervention (weekly).

Conducted counseling and remediation with clinicians, as needed.

The external facilitator acted as a “change agent” and was a “super-champion” at 9 HCBW sites (IF + EF arm), supporting internal facilitators to implement the intervention with clinicians. The external facilitator was an OT with 3 years of experience in facilitation and implementation of the intervention [21] in the HCBW program. External facilitator training was two-phased. First, the external facilitator completed the internal facilitator training (phase 1). Then, the external facilitator completed an online module on the external facilitator role and tasks, challenges an internal facilitator may face, and implementation barriers identified by other facilitators [31], followed by a 90-min session (synchronous) with the study team (phase 2).

During implementation, the external facilitator conducted monthly (or as needed) phone calls for up to 9 months with the 9 internal facilitators in the IF + EF arm. The external facilitator performed the following tasks with the internal facilitators:

Mentored internal facilitators in implementation strategies, transferring the knowledge and skills necessary to support ongoing intervention sustainment.

Reviewed the localized implementation plan and provided suggestions for uptake.

Reviewed barriers to implementation and problem solved strategies to overcome.

Reviewed how to apply the implementation toolkit by sharing examples used previously in the setting, so that strategies could be tailored to the local setting.

Set expectations for use of clinicians, provided examples on differences in duties compared to usual care, and reviewed cases where the intervention was being used.

Assisted with engagement of key stakeholders in sites to support hiring of OTs

Monitored and provided feedback on progress in achieving implementation goals.

Monitored use and impact of identified solutions for problems and barriers.

Remained accessible by phone or email, as needed

It is important to note that although internal facilitators and external facilitator activities were described in a linear-phased process, the facilitation process was actually dynamic and iterative, with activities overlapping and repeating to continually monitor and adjust localized implementation processes to maximize potential for success.

Measures

Site, clinician, and beneficiary level data were collected using the Stages of Implementation (SIC) [32, 33] (sites), Organizational readiness for change (TCU-ORC) [34], Evidence-Based Practice Attitude Scale (EBPAS) [35], general self-efficacy (GSE) [36], (clinicians), and Minimum Data Set-Home Care (MDS-HC) [37] (beneficiaries) tools as described in detail in the protocol paper [21]. We also measured clinician and beneficiary characteristics and training completion.

Outcomes

Primary outcome was adoption and sustainability of the intervention measured via the SIC scores. Scoring of the SIC tool (range 0–100) is described in detail in the protocol paper [18]. Secondary outcomes were clinician attitudes toward evidence-based practice and self-efficacy and beneficiary ADLs (sum of 11 ADL items), IADLs (sum of 8 IADL items), pain (sum of 4 self-reported pain items), depression (sum of 3 self-reported mood items), and number of falls, ED visits, and hospitalizations. Intervention fidelity was not collected due to difficulty extracting data from the EHR.

Study procedures

Internal review board approvals were obtained, contracts (site, state, and EHR company) were executed, and internal facilitators (each site) and the external facilitator were selected and trained. Data (quality assessment scores, number of beneficiaries) were obtained from the state, and sites were randomized. Clinicians were recruited and consented, and baseline data (online survey; characteristics, EBPAS, GSE, TCU-ORC) were obtained. Clinicians were trained, and the intervention was provided to beneficiaries. Clinician EBPAS and GSE were collected again at 9 months after completion of training (June 30, 2020). We planned to collect the SIC (phone surveys) data monthly for 12 months; however, due to COVID, data were not collected in April through August 2020, with the exception of 3 surveys from some of the sites. Beneficiary data were extracted from the EHR, which included the last assessment prior to the clinician training (before October 1, 2019) and assessments following clinician training (October 2, 2019, to June 30, 2020).

Data analysis

Stages of implementation scores were compared between trial arms using t-tests, and effect sizes (Cohen’s d) were estimated as differences between means expressed in standard deviation units. The cutoffs for the interpretation of Cohen’s d are 0.2 (small), 0.5 (medium), and 0.8 (large) [38]. Characteristics of clinicians and beneficiaries were summarized by trial arm at baseline. Because of the turnover of clinicians at each site, constrained longitudinal model, with measures at baseline and at 9 months and a constraint of equality of means at baseline due to randomization, was used for the analysis of clinician data. With this analytical technique, data from all clinicians who completed baseline only, 9 months only, or both surveys were used. Random effects were used to account for nesting of clinicians within sites. For all beneficiaries, baseline data were available, and characteristics of beneficiaries without post-intervention data were compared by trial arm to evaluate potential bias due to missing values. Post-intervention data were analyzed in relation to trial arm with the adjustment for baseline version of the outcome, age, sex, and any baseline factors that differed in terms of missing values.