Out of 28 invited participants, 16 completed interviews (57% response rate). Participant characteristics are provided in Table 1. To maintain participant anonymity, quotes are attributed to care delivery leaders (care delivery leaders and LICSW managers), LICSWs, primary care providers (PCPs), and implementation team members (medical assistants [MAs] and practice facilitator). Additional quotes are provided in Table 2. Nine participants were directly involved in the implementation of reSET and reSET-O including two of the care delivery leaders, three LICSWs, one PCP and all three of the implementation team members. All participants besides the implementation team members had been a part of the implementation of Thrive and the apps available via the health plan’s patient portal website (Calm and myStrength). All but one of the LICSWs and PCPs were from two KPWA medical centers in Seattle, Washington. Care delivery leaders and implementation team members worked across KPWA.

Table 1 Participant characteristicsTable 2 Interview themes with exemplar quotes

Insights from interviews were grouped into six key themes (presented from more general to more specific): (1) general support for implementing digital therapeutics (n = 16), (2) general implementation strategy and workflow recommendations (n = 16), (3) app design and target population will determine implementation needs (n = 14), (4) implementation adaptations for app-based AUD treatment may not need to be extensive (n = 12), (5) implementation adaptations for app-based AUD treatment to accommodate high patient volume (n = 10) and (6) implementation adaptations for app-based AUD treatment to accommodate variation in AUD severity, motivation to change, and treatment goals (n = 10). Key themes were well represented across provider types (i.e., color-coded sticky notes on the virtual affinity diagramming board did not reveal any patterns between roles, and all of the themes included content from multiple provider types).

General support for implementing digital therapeutics

Participants were supportive of offering digital therapeutics for SUD generally and AUD specifically, describing it as “an extra tool in the toolbox” (#11, care delivery leader) and a way to meet high treatment demand. Participants also said that digital therapeutics were a good fit for this context because clinicians in the health system were already using apps to treat anxiety and depression.

General implementation strategy and workflow recommendations

Reflecting on the recent pilot implementation of a SUD digital therapeutic, 5 participants said that the partnership between care delivery leaders and researchers was a successful strategy because researchers evaluated the evidence-base of the apps and research funding brought in additional resources like practice facilitation, health coaching, and electronic health record programming. Participants also recommended involving clinical leaders and clinical champions in implementation efforts and making sure those responsible for implementation had dedicated time to address clinician questions and problem-solve around implementation barriers.

Participants gave advice on approaches for sharing information about newly implemented digital therapeutics and increasing clinician knowledge about them. Participants advocated for training to describe the evidence-base for the app and information about who is most likely to benefit. They also suggested that managers provide dedicated time for clinicians to test and become familiar with the app. Participants advised clinician-facing information about digital therapeutics should come in multiple forms including email “blasts” (#1, LICSW and #9, care delivery leader), documents that contain a concise written overview of the treatment on a single page, announcements in meetings, and information from clinical champions. Participants also suggested clinicians be given an “elevator pitch” (#7, LICSW), meaning a concise verbal description of the app, that they can share with patients.

To ease implementation, participants suggested clear, simple workflows and electronic health record supports to make it easy to connect patients to the app. For example, during Thrive and reSET and reSET-O implementations, a programmer created auto-populating text about the apps for clinical notes; this made it easy for clinicians to share information about the apps and for patients and clinicians to access the information in the future.

App design and target population will determine implementation needs

Participants explained how an app’s design, and specifically, its target population, would determine the number and characteristics of patients who might be offered the app and the supports they may need, which in turn would determine the ideal implementation strategies. For example, a digital therapeutic might be designed for patients with unhealthy alcohol use (likely a large population) or eligibility may be limited to patients with a clinically recognized AUD diagnosis (likely a much smaller population); successful implementation for these unique target populations would likely require different implementation strategies and supports for app delivery.

In general, participants preferred flexible and inclusive eligibility criteria for digital therapeutics. Participants expressed feeling challenged when digital therapeutics had eligibility criteria that restricted its use to a specific population of patients which they perceived as unnecessarily narrow. One participant remarked,

One of the challenges we've come across so far… is that when patients present and they have alcohol use disorder, it has to be paired with another substance [for them to be eligible for reSET]. Which is really hard, because there are so many patients who have presented that have problems with alcohol use, they want support around it, and then we review and see—oh wait, they don't have another substance they're using so I can't offer them reSET. (#11, care delivery leader)

Although reSET was designed to treat SUD but not AUD alone, participants articulated a preference to make their own determination about who to prescribe the app to. Nine participants said they thought the principles behind treating AUD and other SUDs were similar enough that it would be appropriate to prescribe reSET to patients with AUD who do not use other drugs, even though the app is not indicated for patients who solely use alcohol [30].

Whatever the eligibility criteria for an app-based treatment may be, participants stressed that messaging during implementation should make it clear to clinicians which patients are eligible and best suited for the app, especially if care teams have access to multiple digital therapeutics.

Implementation adaptations for app-based AUD treatment may not need to be extensive

Most participants (12/16) said that few, if any, modifications would be necessary to use the existing implementation and workflow strategies developed for prior implementations of other apps to implement a digital therapeutic for AUD. Specifically, implementation strategies (e.g., clinician training materials, electronic health record note templates), and workflows for identifying patients and connecting them to the digital therapeutic (e.g., PCP identification of potentially eligible patients and referral to an LICSW) were identified as applicable for the implementation of digital therapeutics for AUD. Participants also felt that procedures for treatment and follow-up needed few modifications to implement a digital therapeutic for AUD as opposed to SUD. For instance, participants thought that an app could be used as an adjunct to usual treatment for AUD, like it had been for SUD.

Though participants generally endorsed the applicability of past implementation strategies, many also recommended changes to account for the large number of patients with AUD and the unique treatment needs of patients with varying AUD severity, motivation to change, and treatment goals. These recommendations are described in detail below.

Implementation adaptations for app-based AUD treatment to accommodate high patient volume

Most participants (10/16) expected more patients would be eligible for and interested in app-based treatment for AUD compared to SUD because AUD is more prevalent. Participants recommended adaptations to help care teams manage higher patient volume. To improve a health system’s capacity to offer the app to more patients, several participants advocated for a ‘no wrong doors’ approach where any care team member could connect the patient to the app. One participant shared,

We talk about clients having rapport with their PCP... that's not always the case. It may be the nurse or the social worker or the therapist who has far more contact with the client… I believe this is something all clinicians should have awareness of in their toolbox, so to speak, so that if they have rapport with their particular client, that they feel comfortable discussing and offering it. (#9, care delivery leader)

On the other hand, others thought that it would be easier to manage high patient volume if there was one dedicated person (e.g., a centralized LICSW or MA) who would be responsible for connecting patients to the app for multiple clinics. This would help ensure that patients could be reached even if clinicians in the local clinic were too busy to offer the app.

Some participants suggested patients should be able to access the app without going through a clinician and had specific ideas for how a digital therapeutic could be paired with existing wellness or treatment resources. For example, patients at the study site can complete an annual health profile online that includes an alcohol screening instrument. When patients are screened for unhealthy alcohol use, they could be offered the app algorithmically based on a positive screening score result. Another participant suggested,

If somebody is prescribed a medication to reduce cravings, [we could] also hand them this brochure [with information about an app]… at minimum, give them this brochure, at maximum have a quick conversation about here's something else we could pair with medication. (#7, LICSW).

The potential for a high volume of patients caused participants to express doubt care teams would have the capacity to actively monitor patient app use. To support care teams in working with patients using digital therapeutics, participants recommended giving clinicians additional dedicated time to care for these patients, including time to view and process information the app collects (if applicable). Participants also suggested adding supports for patients who are engaging in the app such as tech support or access to a health coach who could monitor patient app use.

Implementation adaptations for app-based AUD treatment to accommodate variation in AUD severity, motivation to change, and treatment goals

Interview participants recommended digital therapeutics be offered as one of many options for AUD treatment, depending on the individual patients’ needs. Different treatment options are needed to account for individual patients’ AUD severity, motivation to change, and treatment goals in terms of whether they want to stop versus reduce their drinking. One participant remarked,

I think a key question is if the app is designed and targeted for people who have alcohol use disorder versus just unhealthy alcohol use. If it's just unhealthy alcohol use, that's a huge population and there would need to be something that is completely self-directed and available on the Web…. [But] the population with a use disorder would benefit from having some staff who are supporting people in using the app and connecting to other care providers and supports if they are identifying a need and a desire for that. Because again, you're talking about a group of people who have a use disorder with a lot of morbidity associated with it and even a highly effective app is not likely to be effective in and of itself for most people. (#12, care delivery leader)

Two participants hypothesized that app-based treatments would be best for patients with mild to moderate AUD who do not need formal treatment. One LICSW shared,

There's a lot of people who get in touch with their provider, their provider gets in touch with social worker because they've started to have the conversation around 'maybe I'm drinking a little bit too much, but I'm not drinking so much that I need treatment or that I need to be connected to a substance use therapist, but maybe I just need a little bit of something to help me get back on track with my goals around a healthy relationship with alcohol.' And so, I think those are patients who would be particularly receptive to app-based care. Because to them it doesn't feel like it's a major problem. It's like the level of treatment fits the level of problem. (#6, LICSW)

On the other end of the spectrum, participants were skeptical about the effectiveness of the app for patients with more severe AUD and expressed concerns about what would happen in a crisis. Some conveyed that dangerous withdrawal symptoms are of greater concern with AUD than SUD, and participants stressed that patients at risk of severe withdrawal symptoms should not rely on digital therapeutics alone for AUD treatment. One PCP said, “I'd probably just want to screen for more of those medically concerning signs of withdrawal, so then they could be encouraged to seek medical care if they are happening…” (#10, PCP).

Participants also hypothesized that patient motivation to change is a determining factor in whether a digital therapeutic would be effective. In general, participants did not think app-based treatment would be useful to patients with low motivation to change. For example, one social work manager shared, “I think that the people who are heavily drinking… I think the app might not be powerful enough” (#5, care delivery leader). Another LICSW compared patients with low motivation to change to patients with high motivation to change:

I think there are some patients that don't want to do anything about their drinking. They either don't acknowledge that it's a problem, or they acknowledge it’s a problem, but they don't want to do anything about it. There are other patients that are like 'yeah, I recognize that I have a problem, but I don't want to go to inpatient, I don't want to go to a facility, I don't want to do all of that, I want to be able to do it on my own.' So those patients I think would be appropriate. Yeah, that's great - you have that motivation, you're driven, you want to make a difference, but you want to be able to have it be a little less intensive. I think a resource like that could be really beneficial. (#16, LICSW)

Participants were supportive of offering app-based treatments to patients who wanted to reduce but not stop their drinking, and a few participants emphasized the importance of allowing for goals besides abstinence. One care delivery leader shared,

You would want a tool that would allow patients to have different goals in terms of what they're looking at with their alcohol use, both from a patient-centered standpoint but also from an effectiveness standpoint, because again we know that just helping people significantly reduce their alcohol use has tremendous health benefits for people. (#12, care delivery leader)

Participants were supportive of having multiple apps on hand to offer to patients with AUD (6/16), as long as it is clear to clinicians “who goes where” (#2, PCP), or which patients to connect to which apps.