The study design was a retrospective descriptive observational investigation via chart review using a fixed available sample from datasets within existing electronic health records and databases for clients admitted to ongoing care by VODP Case Management between April 1, 2017 and March 31, 2020. This initiative met ethics review requirements as specified by Alberta Health Services.

The dataset was derived from information from two electronic databases: ASIST (Addiction & Mental Health System for Information and Service Tracking) and REDCap [32]. ASIST maintains records of utilization and reports at an aggregate level (admissions, discharges, active clients, length of stay). For clinical outcomes related to the study sample the outcome measurement tool was the ODP Admission Assessment and Re-assessment Project managed using REDCap electronic data capture tools hosted by AHS. This project serves as an ongoing clinical data collection tool to monitor client status at admission to Case Management in the VODP and at 3 months, 6 months, and at 12 months for clients while in treatment. REDCap questionnaires were administered by the most responsible VODP Case Manager or delegate by telephone and responses were entered into REDCap. All clients are encouraged to complete questionnaires but they are not mandatory. Clients may decline to complete individual items, may terminate the assessment session at any time, or refuse to participate in the follow-up assessments entirely.

Clients in ongoing care with the VODP were 18 years of age or older and were living in communities across Alberta. The main study sample was identified by first extracting all available records of REDCap questionnaires for clients admitted to the program between April 1, 2017 and March 31, 2020 (N = 1522). Only clients retained for ongoing care by the VODP were included in the study sample; as many as two thirds were served by alternate arms of the VODP such as the Transitional Treatment Service, the Same Day Start Service, or are discharged. These clients were not followed over time as support through transition can be targeted and brief, and thus were excluded from the study. Client records were included in the sample for analysis if they evidenced a completed admission assessment and at least one complete re-assessment. A total of 440 unique client IDs (identifiers removed) were available for inclusion. The steps used to create the final sample are outlined in Fig. 1.

Flow chat outlining steps to create study sample

Clients in the original group who were assessed for admission to the VODP but who did not go on to complete further re-assessments may have done so for various reasons. They may have continued in treatment but declined to participate in follow-up assessments, as clients are not required to participate and there is no change made to their care if they choose to opt out. They may, in alternative situations, have been in the program for up to 3 months and then elected to not continue in treatment. Others may have transitioned to a provider in their local community. Finally, a small number of clients would have been triaged as Harm Reduction, in which case re-assessments are not completed due to contact challenges.

Program characteristics for the VODP over three years were extracted from ASIST. Program characteristics describe all clients admitted to the VODP, including those engaged in ongoing care by VODP ongoing care but not exclusively those individuals, as the VODP offers multiple service arms supporting client engagement in OAT. The number of active clients refers to the number of individual clients who were active with the program during the indicated time period.

Outcome measures related only to clients retained by VODP Case Management in ongoing care. Treatment retention was calculated at 6 months and at 12 months for clients in the study sample. Outcomes were extracted from the ODP Admission Assessment and Re-assessment Project maintained in REDCap on admission and at 3 months, 6 months and 12 months while the client was in treatment. The primary outcome measure was Drug Use, and the questionnaire item was worded, “Are you using this drug?” with binary response options (i.e., yes or no). Drug Use follow-up referred to behaviors during the previous 3 months. Secondary outcomes included the number of categories of substances used by each client within 30 days of assessment, with a score out of 7 calculated with one point added per category (Category 1: Fentanyl/Heroin; Category 2: Other Opioids-Codeine/Tylenol 3, 4; Hydromorphone; Oxycodone; Methadone; Morphine & Other); Category 3: Cocaine/Crack Cocaine; Category 4: Amphetamines; Category 5: Methamphetamines (Crystal Meth); Category 6: Benzodiazepines/Tranquilizers; Category 7: Other Drugs). Of note, when in the pilot stage, the VODP did not provide methadone initiation and as a result methadone was included as one of the substances of interest. To maintain data integrity for this evaluation methadone was retained as a substance of interest despite the fact that methadone was added as a treatment option once initial pilot safety monitoring and expansion were completed.

Additional outcomes included the client's self-reported history of accidental and intentional overdose, with questionnaire questions worded, “Have you accidently overdosed?” and, “Have you intentionally overdosed?” The time frame for follow-up was over the previous three months and response options were binary (i.e., yes or no). Clients' ratings of their level of pain during the 30 days leading up to admission and each re-assessment were also assessed. The variable question was, “Please select one of the following choices to rate your level of pain in the past 30 days: ‘no pain’ is associated with 0, and ‘worst pain possible’ is associated with 10”. The severity of symptoms related to opioid use was also identified as an outcome, derived from diagnostic criteria for Opioid Use Disorder found in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.) [33]. This item was scored based on the checklist described in the DSM-5, and the final scale for this item was None (0–1 symptom), Mild (2–3 symptoms), Moderate (4–5 Symptoms), and Severe (6 + symptoms). The final response for this item was scored and recorded by the client’s Case Manager or delegate who administered the interview.

Items from the Brief Treatment Outcome Measure [34, 35] Social Functioning Scale (BTOM-SFS) were embedded in the project, and scores were extracted at admission, 3 months, 6 months and 12 months. The 6 item social functioning scale includes measures of personal and social well-being related to the client’s levels of financial hardship, conflict in relationships with spouses/partners, other relatives and employers/school staff and students, time spent living with person(s) who use drugs and time spent with friends who do not use drugs. Responses related to how often or how much time clients reported that they spent engaged in these behaviors during the previous 3 months. A composite score was calculated for each of six sub-scales for each client based on Likert scale responses between 0 and 3, with lower scores reflecting better social functioning.

Clients' ratings of satisfaction with using telehealth in support of their OAT care on a 10 point scale ranging from 1 (Completely Dissatisfied) to 10 (Completely Satisfied) were identified as outcomes and were available for extraction at 3 months, 6 months, and 12 months within re-assessments, as were client ratings of satisfaction with the program (ranked on an ordinal scale with five response options: Very Helpful, Somewhat Helpful, Undecided, Not So Helpful, Not At All Helpful). Open-ended responses describing what clients found most helpful and least helpful about the VODP were also extracted at 3 months, 6 months, and 12 months at each re-assessment, and were were exported verbatim.

Data were analyzed using SPSS (version 25) and Microsoft Excel (2013). Descriptive statistics summarized quantitative responses including measures of central tendency and frequencies across four assessment periods. Descriptive statistics were presented for dichotomous variables (i.e., yes or no), and non-parametric Friedman tests of differences over repeated assessments were performed for ordinal variables (Likert Scale or numerical ratings) and non-normally distributed continous variables. Where variables were ordinal but missing data precluded completion of Friedman tests of differences, descriptive statistics were presented. Responses to open-ended questions were pooled across re-assessments and analyzed using NVivo (version 12, 2019) to allow for coding and organization into themes. A small core group of coding categories were developed a priori, with additional elements emerging from the analysis.

There were two main sources of missing data identified within the study sample. Variations in workload demands on program staff and unavailability or unwillingness of clients to complete assessments in a timely fashion resulted in gaps in data collection leading to missed or incomplete assessments within the series for a number of client IDs. For example, for the 226 admission assessments associated with one re-assessment, 133 (59%) were paired with 3 month re-assessments, 72 (32%) were connected with 6 month re-assessments, and 21 (9%) were associated with 12 month re-assessments. Secondly, within individual assessments, singular items were left uncompleted. Due to the retrospective nature of the study, no explanations were available for the origin of those missing values, and patterns of individual missing responses were assumed to be missing at random as they were distributed across re-assessments. No imputation was used to address missing data in the statistical analysis, and listwise deletion was employed for Friedman tests when variables were ordinal, therefore only responses for client IDs with all four assessments completed were analyzed.