Trial design

This study was a 2X2 factorial randomized controlled trial where all participants received access to an 8-week ICBT program, and either an assessment interview (Factor 1), guidance (Factor 2), a combination of these (Factor 1 and 2), or neither of these. The trial was conducted at the Online Therapy Unit (OTU; www.onlinetherapyuser.ca), based at the University of Regina, Saskatchewan, Canada. The OTU routinely offers ICBT to residents of Saskatchewan free of charge, financed by the Saskatchewan Ministry of Health. The study was registered at www.clinicaltrials.gov (NCT03984786) and approved by the University of Regina Ethics Review Board (approval number 2019-058). The protocol has been published [21].

Participants

Participants were recruited through a variety of methods, including Google and Facebook ads across Canada; emails and posters distributed to primary care physicians in Saskatchewan; and emails sent to Canadian organizations. Interested participants were directed to the OTU webpage, where they could complete a consent form and questions regarding contact information, background information (e.g., demographics, medical history, mental health history etc.), alcohol use, depression, and anxiety. Applicants meeting initial inclusion criteria scheduled a telephone screening call with OTU staff. In the call (conducted within 1–2 weeks of survey completion), applicants were asked follow-up questions to the screening questions to confirm eligibility, and were asked to verbally consent to participation. To be included in the trial, participants had to (a) be 18 years or older; (b) be a Canadian resident; (c) have access to the internet; (d) score ≥ 8 on the Alcohol Use Disorder Identification Test (AUDIT) [22] indicating at least hazardous drinking; and (e) have consumed ≥ 14 drinks in the preceding week. Applicants were excluded from the trial if they presented with (a) > 24 on the Patient Health Questionnaire 9-item (PHQ-9) [23]; b) suicidal ideation; (c) unmanaged bipolar disorder or schizophrenia; (d) > 24 on the Drug Use Disorder Identification Test (DUDIT) [24] or severe substance use problems as assessed in the telephone enrollment call; (e) low motivation to engage with online treatment as assessed in the telephone enrollment call; (f) frequent visits with a mental health professional (i.e., more than twice a month); or (g) hospitalization for mental health reasons in the past year. Ineligible applicants were referred to appropriate services or were offered the course without being included in the trial.

Randomization

Immediately after eligibility had been confirmed and verbal consent had been obtained in the telephone screening call, screening staff randomized participants to one of the four treatment conditions. The randomization sequence was pre-generated on http://randomization.com using blocks of 16 and uploaded to the survey system used for data collection. The sequence was hidden from the staff randomizing and participants were blinded to the factors investigated. At the end of screening calls, all participants received a username and temporary password, along with instructions for accessing the course. All participants were given access to the ICBT program on the second Monday after randomization. This treatment delay was chosen to allow a minimum period of nine days to evaluate changes in drinking and motivation to change resulting from the assessment interview.

Intervention: the alcohol change course

The Alcohol Change Course (ACC) is an ICBT program originally developed in Switzerland [25,26,27] and translated to English [28] to target alcohol misuse and depression in young adults. For the purpose of this study, the program was adapted for use by adults, with relevant information about alcohol use in Canada (prevalence, guidelines etc.), abstinence, and the impact of alcohol on physical health added to Lesson 1. Further, the program content was restructured to be consistent with the OTU’s other ICBT programs; each lesson consisted of information provided in a slide show format, combined with case stories, worksheets to practice skills, and quizzes and exercises related to each lesson. Participants could download worksheets for use at a later date. The 12 lessons were delivered consecutively over the span of 8 weeks. The adaptation process was completed by a patient-oriented working group consisting of four patients, two clinicians, two managers, two trainees, and two group facilitators. The program was pilot-tested with nine participants before formal start of the factorial trial.

Experimental factorsFactor 1: assessment interview

The assessment interview was administered in the screening call immediately following the randomization. The purpose of the assessment interview was to increase participants’ insight into their own alcohol habits and their consequences through a supportive conversation guided by the AUD module of the Structured Clinical Interview Diagnostic Statistical Manual 5 – Research Version (SCID-5 RV) [29]. In total, three screeners were involved in the screening. The screeners ranged in background training and included Bachelor’s degrees in social work (n = 2) and a Master’s degree in counselling psychology (n = 1). Screeners received a SCID-5 training session as well as training and supervision provided by authors CS and MN. Since the goal of the interview was not to establish a diagnosis, participants were not informed about how many criteria they screened positive for. Screening calls with the assessment interview were 36.0 min (SD = 11.2), while screening calls without the assessment interview were 18.4 min (SD = 6.4), (t = − 14.771, p =  < 0.001, Cohen´s d = 1.87, CI: 1.572–2.169).

Factor 2: guidance

Two clinicians provided guidance. Clinicians held graduate degrees in counselling psychology (MEd) and social work (MSW) and had been practicing for 1 and 13 years, respectively, at the time of study onset. They were blinded to whether participants had been randomized to the assessment interview or not and were instructed to spend approximately 15 min per week communicating with each participant, primarily through messages on the treatment platform. In these messages, clinicians were to answer participants’ questions, reinforce module completion, and boost motivation. In rare cases, clinicians contacted participants by telephone (i.e., if a participant requested a call, if there was increased suicide risk, or to address misunderstanding).

Across all four conditions, participants received automated, weekly emails with information about new lesson content. Those randomized to a group without guidance did not receive any other regular contact, but a member of the research team reviewed weekly survey responses to check for significant clinical issues requiring attention (i.e., major increase in drinking, sudden increase in depression symptoms or suicidal ideation). If any participant was deemed at risk, they were contacted and offered referral to appropriate health care. However, they were only discontinued from the trial if they requested it. All participants could contact the OTU if they experienced technical difficulties related to the treatment platform or wanted to discontinue the intervention.

Monitoring of participants

Each week throughout the 8-week course, all participants who logged in to the platform were asked to complete two questions about past week alcohol use; (1) How many drinks have you had in the past week; and (2) Over how many days did you consume these drinks? They also completed the Patient Health Questionnaire-4 (PHQ-4), a brief questionnaire assessing depression and anxiety [30], and item 9 from PHQ-9 assessing suicidal ideation [23]. These weekly questionnaires were not intended as outcome measures, but allowed systematic monitoring of client symptoms as a safety measure. Participants also responded to reflection questions asking them to list challenges they had with the exercises and to provide examples of what they had learned.

Measurements

Participants were asked to complete online questionnaires with outcome measures at screening, pre-treatment, mid-treatment (4 weeks into the treatment), post-treatment (8 weeks), and 3-month post-treatment (6 month and 1 year follow-up data collection is ongoing). Participants who did not complete questionnaires were contacted via telephone and/or email as a reminder to complete measures, with a maximum of three reminders per follow-up period. The primary outcomes were number of drinks and number of heavy drinking days (HDD; defined as ≥ 4 drinks per day for women and ≥ 5 drinks per day for men) in the preceding week. Secondary measures were the AUDIT [22], the Penn Alcohol Craving Scale (PACS) [31], and the Brief Situational Confidence Questionnaire (BSCQ) [32]. Additional measures included daily functioning (Sheehan Disability Scale; SDS) [33], depression (the Patient Health Questionnaire-9; PHQ-9) [23], and anxiety (Generalized Anxiety Disorder-7; GAD-7) [34]. Initial motivation, as well as motivational changes between screening and treatment start were assessed using the Readiness to Change Questionnaire –Treatment Version (RCQ-TV) [35]. Finally, treatment credibility was assessed at mid-treatment using the Credibility/Expectancy Questionnaire [36]. Questions pertaining to treatment evaluation and negative effects were assessed at post-treatment and have previously been reported for the overall sample [37]. Treatment engagement was assessed via: number of lessons accessed, days between first and last access to the website, and mean number of website log-ins.

Sample size and power analysis

The trial aimed to recruit 300 participants to the four conditions (75 participants per group). To estimate this sample size, we used the Factorial PowerPlan provided in the R package MOST [38]. Regarding effect size estimates of factor 1 (assessment interview) we had no available studies to draw on, and therefore pragmatically decided to estimate the effect size as 0.35, as this was the minimum effect size that would indicate this factor to be worth implementing, considering the time spent by staff conducting the interview. Regarding factor 2 (guidance), we estimated the effect size as 0.35, based on previously published studies [7, 8]. Power was set at 80%, alpha at 0.05, and we assumed a correlation of 0.5 between pre-and post-test measurements, and an attrition of 30%

Statistical analyses

At least one primary or secondary outcome measure was missing for 27% of the sample at post-treatment and 37% at 3-month follow-up. Missing responses were strongly associated with fewer lessons completed (p < 0.001) and with the self-guided course (p = 0.002). To attempt to control for possible bias from differential response rates we used a multiple imputation procedure to replace missing outcome measures. Fifty imputed datasets were created with the MICE package in R [39]. The imputation models controlled for lesson completion, treatment factors, observed values of that measure at other observation times, and interactions between lesson completion, treatment, and observed values.

To evaluate changes in primary and secondary measures, we modeled responses using generalized estimating equations (GEE) [40] using the geepack package, version 1.3.2. in R. The GEE models used Gamma distributions to accommodate skewed response distributions and a log-link function to model changes as proportional to pre-treatment severity. We specified an exchangeable working correlation matrix within individuals to address within-subject correlations, estimated standard errors with robust “sandwich” estimates, and tested whether interactions between Time and Factor were significant using the multivariate Wald test pooling results from the multiply imputed datasets. To evaluate whether the assessment interview had an effect on RCQ-TV change between screening and pre-treatment we planned to use a multinomial logit model. However, as virtually all participants were in the same “readiness to change” stage at screening (i.e., contemplation stage) and at pretreatment (i.e., action stage) respectively, we did not conduct this analysis. To evaluate changes in drinking between screening and pre-treatment, two-way ANOVA’s were used, and to evaluate program engagement, credibility, satisfaction and negative effects by group, one-way ANOVA’s were used. SPSS 25 was used for descriptive statistics and to analyze changes between screening and pre-treatment, while R version 4.1.0. was used for all outcome analyses.